Customizing environment definitions

As Admin or Editor of a project, you can change the name, the description, and the hardware configuration of an environment definition that you created. You cannot change the language of an existing environment definition.

You can customize the software configuration of Jupyter notebook environment definitions through conda channels or by using pip. This can be done by providing a list of conda packages, a list of pip packages, or a combination of both. When using conda packages, it is also possible to provide a list of additional conda channel locations through which the packages can be obtained.

You can’t customize the software configuration of a Spark environment definition you created.

To customize an environment definition you created:

  1. Click your project’s Environments tab.
  2. In the Active environment runtimes section, check that no runtime is active for the environment definition you want to change.
  3. In the Environment definitions section, click the environment definition you want to customize.
  4. Make your changes.

    For a Juypter notebook environment definition, select to create a customization and specify the libraries to add to the standard packages that are available by default. You can also use the customization to upgrade or downgrade packages that are part of the standard software configuration.

    The libraries that are added to an environment definition through the customization aren’t persisted; however, they are automatically installed each time the environment runtime is started. Note that if you add a library using pip install through a notebook cell and not through the customization, only you will be able to use this library; the library is not available to someone else using the same environment definition.

    If you want you can use the provided template to add the custom libraries. There is a different template for Python and for R. The following example shows you how to add Python packages:

    # Modify the following content to add a software customization to an environment.
    # To remove an existing customization, delete the entire content and click Apply.
    # Add conda channels below defaults, indented by two spaces and a hyphen.
      - defaults
    # To add packages through conda or pip, remove the comment on the following line.
    # dependencies:
    # Add conda packages here, indented by two spaces and a hyphen.
    # Remove the comment on the following line and replace sample package name with your package name:
    #  - a_conda_package=1.0
    # Add pip packages here, indented by four spaces and a hyphen.
    # Remove the comments on the following lines  and replace sample package name with your package name.
    #  - pip:
    #    - a_pip_package==1.0

    Important when customizing:

    • Before you customize a package, verify that the changes you are planning have the intended effect.
      • conda can report the changes required for installing a given package, without actually installing it. You can verify the changes from your notebook. For example, for the library spaCy:
        • In a Python notebook, enter: !conda install --dry-run spacy=2.0.16
        • In an R notebook, enter: print(system2("conda", args="install --dry-run spacy=2.0.16", stdout=TRUE))
      • pip does install the package. However, restarting the runtime again after verification will remove the package. Here too you verify the changes from your notebook. For example, for the library spaCy:
        • In a Python notebook, enter: !pip install spacy
        • In an R notebook, enter: print(system2("pip", args="install spacy", stdout=TRUE))
    • If you can get a package through conda from the default channels and through pip from PyPI, the preferred method is through conda from the default channels.
    • Conda does dependency checking when installing packages which can be memory intensive if you add many packages to the customization. Ensure that you select an environment with sufficient RAM to enable dependency checking at the time the runtime is started.
    • To prevent unnecessary dependency checking if you only want packages from one Conda channel, exclude the default channels by removing defaults from the channels list in the template and adding nodefaults.
    • If you only add packages through pip to the customization template, you must make sure that dependencies is not commented out in the template.
    • When you specify a package version, use a single = for conda packages and == for pip packages. Wherever possible, specify a version number as this reduces the installation time and memory consumption significantly. If you don’t specify a version, the package manager might pick the latest version available, or keep the version that is available in the package.
    • You cannot add arbitrary notebook extensions as a customization because notebook extensions must be pre-installed. The only notebook extensions which can be enabled are the Esri ArcGIS extension and its prerequisites. When you create a runtime environment definition with the Python 3.6 software configuration, you can select to enable Esri ArcGIS. Note that you can only use the pre-installed version.
  5. Apply your changes.

Learn more

Installing custom packages through a notebook