Installing Custom Packages in Kubeflow Notebooks at Runtime

Describes how to install custom packages in existing Kubeflow notebooks that persist between restarts.

You can only install custom packages in a Kubeflow notebook in connected HPE Ezmeral Unified Analytics Software environments for two types of packages – conda packages and pip packages.

You cannot install custom packages in air-gapped environments. Packages installed to the base environment do not persist; the packages are removed after the notebook restarts.

By default, the base conda environment is activated for all notebook users. All notebook users can perform the following tasks:
  • Install packages to the base environment
  • Create and install your own conda environment
  • Use the conda environment of another notebook user, if permitted by the environment owner
You can install packages that persist (save between restarts). You can also install packages that do not persist (do not save between restarts).
  • If packages do not have to persist between restarts, install the packages to the base conda environment. This applies to both single-user and multi-user modes.
  • If packages must persist between restarts, create an individual conda environment. This applies to both single-user and multi-user modes.
The following sections describe how to create an individual conda environment where you can install packages that persist between restarts in single and multi-user modes:
NOTE
Run commands in the notebook terminal.

Single-User Mode

Complete the following steps in the notebook if you want to install custom packages that persist between restarts in single-user mode:
  1. Create an individual conda environment:
    conda create --prefix ~/.conda/envs/kf-users-env --clone base
  2. Activate the conda environment:
    conda activate kf-users-env

Multi-User Mode

Any user with access to the notebook, typically the owner, can create the conda environment. The conda environment is shared with other users (between contributors). All users get equivalent permissions. Users can use the existing packages, as well as install and remove the packages.

Complete the following steps in the notebook if you want to install custom packages that persist between restarts in multi-user mode:
  1. Create the conda environment:
    umask 0000 && conda create --prefix ~/.conda/envs/kf-users-env --clone base
  2. Activate the conda environment:
    conda activate kf-users-env
  3. Add users (contributors) to the conda environment:
    conda config --append envs_dirs  /home/<notebook_owner_username>/.conda/envs
  4. Activate the conda environment for users:
    conda activate kf-users-env
  5. Install the conda package:
    conda install package-name=<version>
  6. Install the PIP package:
    pip install package-name==<version>