Using the Air Gap Utility

Describes how to use the air gap utility to download files in an air-gapped HPE Ezmeral Runtime Enterprise environment.

Requirements:
  • Python:
    • 2.7
    • 3.6 and above
  • Operating system:
    At minimum:
    • RHEL 8
    • SLES 15
    • CentOS 7x
  • Skopeo:
    At minimum:
    • For RHEL or CentOS:
      • Skopeo 0.1.40
    • For SLES:
      • Skopeo 0.1.41
CAUTION

If you will be using an air-gap configuration for Kubernetes objects, then you must configure air-gap settings before adding any Kubernetes hosts.

Apply all air-gap settings with care. These settings do not propagate if updated after Kubernetes hosts have been installed unless one of the following occurs:

  • The Kubernetes host is rebooted.
  • The version of Kubernetes running on a host is upgraded.

Any Kubernetes hosts in a ready state that are not part of a Kubernetes cluster must be restarted for the changes to be applied.

About the Air Gap Utility

HPE Ezmeral Runtime Enterprise provides a utility you can use to query, filter, and download all air gap container images necessary for your environment to a local filesystem or remote registry.

Installing the Air Gap Utility Package

Before downloading files for your air gap environment, you must first install the air gap script package. You can install the package on any non-platform host, even outside the platform installation. Python 2.7 or Python 3.6 and greater is required for install.

To install the air gap utility package:

  1. Download the air gap utility package from the following links:
  2. Install Skopeo. In the CLI, enter the following:
    • If you are using RHEL:
      dnf install -y skopeo
    • If you are using SLES:
      zypper install -y skopeo
  3. Install the hpeairgaputil package:
    • PIP2:
      pip install hpeairgaputil-1.3-py2.py3-none-any.whl
    • PIP3:
      pip3 install hpeairgaputil-1.3-py2.py3-none-any.whl
    NOTE
    To uninstall hpeairgaputil, use:
    • PIP2:
      pip uninstall hpeairgaputil-1.3-py2.py3-none-any.whl
    • PIP3:
      pip3 uninstall hpeairgaputil-1.3-py2.py3-none-any.whl

Using Air Gap Utility Filters

After Installing the Air Gap Utility Package, you can filter the available apps for a given HPE Ezmeral Runtime Enterprise version in a project.

You must provide one of the following mandatory arguments in each of your commands:

  • --list_releases
  • --release
NOTE

To display a list of options available in the hpe-airgap-util, use the following command:

hpe-airgap-util --help

You can use filters to display the following information:

NOTE

The system output in the following examples are for illustration only, and might not represent the software available for your release of HPE Ezmeral Runtime Enterprise.

  • Release: List all releases with the following command:
    hpe-airgap-util --list_releases

    For example:

    hpe-airgap-util --list_releases
    ----------------------------
    ERE          Kubernetes	Release
    ----------------------------
    
    5.4.1    	NA        	GA      
    5.5.0-102	1.0.1      	RC      
    
    ----------------------------
  • Images: List all images for a particular release:
    hpe-airgap-util --release <release-number>
    For example:
    hpe-airgap-util --release 5.4
    INFO: Found 263 repositories to process.
    +---------------------------------------------------------------------------------------+-------------------------------------------------------------------------+----------+-----------------------+-------------+------------+
    |                                      repository                                       |                                 digest                                  | size(mb) |       component       | requirement |  license   |
    +---------------------------------------------------------------------------------------+-------------------------------------------------------------------------+----------+-----------------------+-------------+------------+
    |                                     postgres:9.5                                      | sha256:78bdf72abdd619368cd22fd6372553f4ece87b2ca0f8f9fa4a2ab0e3a3932c36 |  72.78   |        airflow        |  optional   | enterprise |
    |                       pbweb/airflow-prometheus-exporter:latest                        | sha256:8e61f0b7980eb672b1f9a250153ce1f55135b9af5c3f594aa3e4c0a847889766 |   6.39   |        airflow        |  optional   | enterprise |
    |                               k8s.gcr.io/volume-nfs:0.8                               | sha256:3899ca782a272608fb4139eca436e87592eb779ae76adad6f8e0080365d57de0 |  88.99   |        airflow        |  optional   | enterprise |
    |                          k8s.gcr.io/git-sync/git-sync:v3.3.4                          | sha256:866599ca98bcde1404b56152d8601888a5d3dae7fc21665155577d607652aa09 |  56.47   |        airflow        |  optional   | enterprise |
    ...
    ...
    ...
    |                            bluedata/kd-livy:050-5.4.0-1.1                             | sha256:23ddfe633e18ba431d4794097e919760d4c7c6eec89a4032750c436f521fdb0e |  331.72  |         spark         |  optional   | analytics  |
    |                        bluedata/kd-hivemetastore:238-5.4.0-1.1                        | sha256:08d22f25191902f54c37d05bca42e03ceab03dff9cdc45e88f36e080fcc72ec1 |  331.72  |         spark         |  optional   | analytics  |
    +---------------------------------------------------------------------------------------+-------------------------------------------------------------------------+----------+-----------------------+-------------+------------+
    
    • List available images without headers:
      hpe-airgap-util --release <release-number> --noheaders
    • List all required images:
      hpe-airgap-util --release <release-number> --required 
    • List all optional images:
      hpe-airgap-util --release <release-number> --optional
  • List components: List all the components that are available for a particular release:
    hpe-airgap-util --list_components --release <release-number>

    For example:

    hpe-airgap-util --list_components --release 5.4
    INFO: Found 285 repositories to process.
    +-----------------------+
    | components            |
    +-----------------------+
    | airflow               |
    | argocd                |
    | datafabric            |
    | falco                 |
    | hpecp-agent           |
    | hpecp-monitoring      |
    | hpecp-nvidiagpubeat   |
    | hpecp-serviceaccounts |
    | istio                 |
    | kube-state-metrics    |
    | kubedirector          |
    | kubeflow              |
    | kubernetes            |
    | kubernetes-dashboard  |
    | metrics-server        |
    | mlops                 |
    | nvidia-plugin         |
    | opa-gatekeeper        |
    | spark                 |
    | spark-operator        |
    +-----------------------+
    
  • Component: List all images for a particular component:
    hpe-airgap-util --release <release-number> --component <component>

    For example:

    hpe-airgap-util --release 5.4 --component hpecp-agent
    INFO: Found 263 repositories to process.
    +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
    |                  repository                  |                                 digest                                  | size(mb) |  component  | requirement |  license  |
    +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
    |          bluedata/hpecp-dtap:1.8.0           | sha256:3d61f2a5c56da15e4002e720e6ab83103673ef9c4ae39834ca2e692f8fe334a1 |  250.42  | hpecp-agent |  required   | essential |
    |         bluedata/hpecp-fsmount:1.1.1         | sha256:42ba5577fccc7340dd4c58a748ef5ad3465362b77b0e38325e7d720bb421aa6b |  88.04   | hpecp-agent |  required   | essential |
    |          bluedata/hpecp-agent:1.2.1          | sha256:a7d25b4a777f0f2db842a4a6dbef88cf46a73d576d7facc12a7ed83796236f24 |  177.31  | hpecp-agent |  required   | essential |
    |      bluedata/hpecp-bootstrap-tools:0.4      | sha256:501ffa0dfda7a277717158c81441c54fa401ae4aa3ae08c90201bb99e232b998 |  221.02  | hpecp-agent |  required   | essential |
    | bluedata/hpecp-bootstrap-hpecp-agent:1.2.1-3 | sha256:c9fc9084b3904c79f3566c4ea440efc8229bcf0187569144232072d8ce7ed14b |  128.54  | hpecp-agent |  required   | essential |
    +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
    
     
  • Size: Valid values include b, kb, mb, and tb.

    • Display images less than a certain size:
      hpe-airgap-util --release <release-number> --lessthan 1mb 

      For example:

      hpe-airgap-util --release 5.4 --lessthan 1mb
      INFO: Found 263 repositories to process.
      +-----------------------------------+-------------------------------------------------------------------------+----------+------------+-------------+------------+
      |            repository             |                                 digest                                  | size(mb) | component  | requirement |  license   |
      +-----------------------------------+-------------------------------------------------------------------------+----------+------------+-------------+------------+
      | gcr.io/mapr-252711/busybox:latest | sha256:31a54a0cf86d7354788a8265f60ae6acb4b348a67efbcf7c1007dd3cf7af05ab |   0.77   | datafabric |  optional   | enterprise |
      |          busybox:latest           | sha256:b69959407d21e8a062e0416bf13405bb2b71ed7a84dde4158ebafacfa06f5578 |   0.77   |  kubeflow  |  optional   |   mlops    |
      |      k8s.gcr.io/pause:3.4.1       | sha256:9ec1e780f5c0196af7b28f135ffc0533eddcb0a54a0ba8b32943303ce76fe70d |   0.30   | kubernetes |  required   | essential  |
      |       k8s.gcr.io/pause:3.2        | sha256:4a1c4b21597c1b4415bdbecb28a3296c6b5e23ca4f9feeb599860a1dac6a0108 |   0.30   | kubernetes |  required   | essential  |
      +-----------------------------------+-------------------------------------------------------------------------+----------+------------+-------------+------------+
      
    • Display images greater than a certain size:
      hpe-airgap-util --release <release-number> --greaterthan 100mb

      For example:

      hpe-airgap-util --release 5.4 --greaterthan 10gb
      INFO: Found 263 repositories to process.
      +----------------------------------------+-------------------------------------------------------------------------+----------+-----------+-------------+---------+
      |               repository               |                                 digest                                  | size(mb) | component | requirement | license |
      +----------------------------------------+-------------------------------------------------------------------------+----------+-----------+-------------+---------+
      | bluedata/kd-deployment-api-serving:1.0 | sha256:1b5062ca915f0a846d640e58a30b3231761fa4a03a6ed1b1fce5317149fe22c8 | 11885.33 |   mlops   |  optional   |  mlops  |
      |  bluedata/kd-training-api-serving:1.0  | sha256:efd5e86e270d9dc775fd04b17aff5465b48c0c3693feb313a6bd9f17780caab8 | 11876.42 |   mlops   |  optional   |  mlops  |
      |        bluedata/kd-notebook:3.1        | sha256:07234b781ce21518da396ccfec9708edfd2dea463ada0f02098ddbee0a0cc04c | 14777.13 |   mlops   |  optional   |  mlops  |
      +----------------------------------------+-------------------------------------------------------------------------+----------+-----------+-------------+---------+
    • Display images between two sizes:
      hpe-airgap-util --release <release-number> --lessthan 1gb --greaterthan 500mb
  • License: view all images for a particular license.

    Valid values include essential, enterprise, analytics, and mlops.

    For example:

    hpe-airgap-util --release <release-number> --license enterprise
  • You can combine filters to provide a more customized query.

    For example:

    hpe-airgap-util --release <release-number> --component falco
    hpe-airgap-util --release 5.4 --component falco
    repo_name                                                     digest                                                                          size(mb)                 addon      required|optional            license
    ---------------------------------------                       -----------------------------------------------------------------------         ----------               -------    -------------------          ----------
    falcosecurity/falcosidekick-ui:v1.1.0                         sha256:1ad2a5ecf3eae9b760f0f2467a1add7e0f014760f0f105f25ba7465b4981ca2d              9.79                 falco       required                    enterprise
    bluedata/hpecp-bootstrap-tools:0.4:0.5                        sha256:72d5b10e34076f542e1dcc9ae749001c9dc01ae5c6962fd66249704918b63eab              156.99               falco       required                    enterprise
    falcosecurity/falco-no-driver:0.29.1                          sha256:d51f4a74c3c0b10139cf490e32b685d733e878720c5f925f60552058114fd077              6.46                 falco       required                    enterprise
    falcosecurity/falcosidekick:2.23.1                            sha256:e3b7473f25a7e631fe82b15ac6cbb88f7c203a8064db5baca6ff1fa634b392cc              27.02                falco       required                    enterprise
    bluedata/hpecp-bootstrap-falco:0.29.1-2                       sha256:b2de01af4c77dcb90fcc2d9ab5a16b1ae4c15a03a64bd5545790e3807e17cb26              52.05                falco       required                    enterprise
    
  • To filter for a specific name or string, you can use the options --noheaders | grep <String>.

    For example:

    hpe-airgap-util --release <release-number> --noheaders | grep bootstrap
    hpe-airgap-util --release 5.4 --noheaders | grep bootstrap
    bluedata/hpecp-bootstrap-hpecp-agent:1.2.1-0         sha256:d95558301b629c6cc286e045b269324d4197596b1ae0da55782d41b50fb14992   61.63  hpecp-agent       required  enterprise
    bluedata/hpecp-bootstrap-hpecp-monitoring:6.6.5-8.0  sha256:151a57f2c4b3b7ca5319099e2de9cbab06d799ee5595fba31c8d49a49e419bdb   51.35  hpecp-monitoring  required  enterprise
    bluedata/hpecp-bootstrap-velero:1.6.3-2              sha256:ee57acaf232ceb944030f722eeecb70df92572c10cb2a7069b70e8b3cf5ce200   78.55  velero            optional  enterprise
    bluedata/hpecp-bootstrap-argocd:2.1.2-2              sha256:e09637ede315a2e0b0faf3012dfd1ada80fdac71291f998273093ff683435d6e  347.66  argocd            required  enterprise
    bluedata/hpecp-bootstrap-tools:0.4                   sha256:501ffa0dfda7a277717158c81441c54fa401ae4aa3ae08c90201bb99e232b998  221.02  hpecp-agent       required  enterprise
    bluedata/hpecp-bootstrap-tools:0.4:0.5               sha256:72d5b10e34076f542e1dcc9ae749001c9dc01ae5c6962fd66249704918b63eab  156.99  falco             required  enterprise
    bluedata/hpecp-bootstrap-falco:0.29.1-2              sha256:b2de01af4c77dcb90fcc2d9ab5a16b1ae4c15a03a64bd5545790e3807e17cb26   52.05  falco             required  enterprise
    

Downloading Air Gap Files

After Using Air Gap Utility Filters to find the necessary files for your deployment, download the files as follows:

  1. Use a single command to filter and copy air gap files to a local filesystem or remote registry. Include all filters you want to apply to your download.
    Include --dest_compress to compress the files and download in a .tgz file. Otherwise, the files download in a .tar file. For example:
    hpe-airgap-util --release <release-number> --lessthan 1mb --copy --dest_path images/ --dest_compress
    Use --force to delete the .tgz or .tar file of the image if it already exists. For example:
    hpe-airgap-util --release <release-number> --lessthan 1mb --copy --dest_path images/ --force
    hpe-airgap-util --release <release-number> --lessthan 1mb --copy --dest_path images/ --dest_compress --force
    • To copy multiple images to a local filesystem, execute the following command. Provide the destination path where you want to store your files.

      hpe-airgap-util --release <release-number> <add-on_filters> --copy --dest_path <destination-path>
      For example:
      hpe-airgap-util --release 5.4 --component hpecp-agent --copy --dest_path /home/user/image 
      INFO: Found 263 repositories to process.
      +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
      |                  repository                  |                                 digest                                  | size(mb) |  component  | requirement |  license  |
      +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
      |          bluedata/hpecp-dtap:1.8.0           | sha256:3d61f2a5c56da15e4002e720e6ab83103673ef9c4ae39834ca2e692f8fe334a1 |  250.42  | hpecp-agent |  required   | essential |
      |         bluedata/hpecp-fsmount:1.1.1         | sha256:42ba5577fccc7340dd4c58a748ef5ad3465362b77b0e38325e7d720bb421aa6b |  88.04   | hpecp-agent |  required   | essential |
      |          bluedata/hpecp-agent:1.2.1          | sha256:a7d25b4a777f0f2db842a4a6dbef88cf46a73d576d7facc12a7ed83796236f24 |  177.31  | hpecp-agent |  required   | essential |
      |      bluedata/hpecp-bootstrap-tools:0.4      | sha256:501ffa0dfda7a277717158c81441c54fa401ae4aa3ae08c90201bb99e232b998 |  221.02  | hpecp-agent |  required   | essential |
      | bluedata/hpecp-bootstrap-hpecp-agent:1.2.1-3 | sha256:c9fc9084b3904c79f3566c4ea440efc8229bcf0187569144232072d8ce7ed14b |  128.54  | hpecp-agent |  required   | essential |
      +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
      INFO: Processing artifact bluedata/hpecp-dtap:1.8.0 ...
      INFO: Copying artifact to /home/user/image/bluedata_hpecp-dtap_1.8.0.tar
      Getting image source signatures
      Copying blob 8ba884070f61 done
      Copying blob 71d2b71667ea done
      Copying blob 10bb33b8b168 done
      Copying blob 1204286a37b7 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-fsmount:1.1.1 ...
      INFO: Copying artifact to /home/user/image/bluedata_hpecp-fsmount_1.1.1.tar
      Getting image source signatures
      Copying blob 29291e31a76a done
      Copying blob 615a2023df20 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-agent:1.2.1 ...
      INFO: Copying artifact to /home/user/image/bluedata_hpecp-agent_1.2.1.tar
      Getting image source signatures
      Copying blob ed5dc850ecaf done
      Copying blob 79f8ae5118d4 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-bootstrap-tools:0.4 ...
      INFO: Copying artifact to /home/user/image/bluedata_hpecp-bootstrap-tools_0.4.tar
      Getting image source signatures
      Copying blob 3aa8b87b7f88 done
      Copying blob 57584b59d88b done
      Copying blob 135070274eb1 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-bootstrap-hpecp-agent:1.2.1-3 ...
      INFO: Copying artifact to /home/user/image/bluedata_hpecp-bootstrap-hpecp-agent_1.2.1-3.tar
      Getting image source signatures
      Copying blob ed5dc850ecaf done
      Copying blob 79f8ae5118d4 done
      Copying blob e17b1b3fe6d3 done
      ...
      ...
      Copying config 0168331b65 done
      Writing manifest to image destination
      Storing signatures
      
    • To copy a single image to a local filesystem, execute the following command. Provide the destination path where you want to store your files.
      hpe-airgap-util --release <release-number> --image <image-name> --copy --dest_path <destination-path>
    • To copy multiple images to a remote container registry, select one of the following options. Provide the destination URL and credentials for your container registry.

      • Use the --dest_creds <username:password> command line option:
        hpe-airgap-util --release <release-number> <add-on-filters> --copy --dest_url <destination-url> --dest_creds <username:password> 
      • Alternatively, set environment variable AIRGAP_UTIL_CREDS. You can set environmental variables using the export command:
        export AIRGAP_UTIL_CREDS=<username>:<password>

      For example:

      hpe-airgap-util --release 5.4  --component hpecp-agent --copy --dest_url my.local.registry.com/airgap-ecp54
      INFO: Found 263 repositories to process.
      +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
      |                  repository                  |                                 digest                                  | size(mb) |  component  | requirement |  license  |
      +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
      |          bluedata/hpecp-dtap:1.8.0           | sha256:3d61f2a5c56da15e4002e720e6ab83103673ef9c4ae39834ca2e692f8fe334a1 |  250.42  | hpecp-agent |  required   | essential |
      |         bluedata/hpecp-fsmount:1.1.1         | sha256:42ba5577fccc7340dd4c58a748ef5ad3465362b77b0e38325e7d720bb421aa6b |  88.04   | hpecp-agent |  required   | essential |
      |          bluedata/hpecp-agent:1.2.1          | sha256:a7d25b4a777f0f2db842a4a6dbef88cf46a73d576d7facc12a7ed83796236f24 |  177.31  | hpecp-agent |  required   | essential |
      |      bluedata/hpecp-bootstrap-tools:0.4      | sha256:501ffa0dfda7a277717158c81441c54fa401ae4aa3ae08c90201bb99e232b998 |  221.02  | hpecp-agent |  required   | essential |
      | bluedata/hpecp-bootstrap-hpecp-agent:1.2.1-3 | sha256:c9fc9084b3904c79f3566c4ea440efc8229bcf0187569144232072d8ce7ed14b |  128.54  | hpecp-agent |  required   | essential |
      +----------------------------------------------+-------------------------------------------------------------------------+----------+-------------+-------------+-----------+
      INFO: Processing artifact bluedata/hpecp-dtap:1.8.0 ...
      INFO: Copying artifact to my.local.registry.com/airgap-ecp54/bluedata/hpecp-dtap:1.8.0
      Getting image source signatures
      Copying blob 8ba884070f61 done
      Copying blob 71d2b71667ea done
      Copying blob 10bb33b8b168 done
      Copying blob 1204286a37b7 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-fsmount:1.1.1 ...
      INFO: Copying artifact to my.local.registry.com/airgap-ecp54/bluedata/hpecp-fsmount:1.1.1
      Getting image source signatures
      Copying blob 29291e31a76a done
      Copying blob 615a2023df20 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-agent:1.2.1 ...
      INFO: Copying artifact to my.local.registry.com/airgap-ecp54/bluedata/hpecp-agent:1.2.1
      Getting image source signatures
      Copying blob ed5dc850ecaf done
      Copying blob 79f8ae5118d4 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-bootstrap-tools:0.4 ...
      INFO: Copying artifact to my.local.registry.com/airgap-ecp54/bluedata/hpecp-bootstrap-tools:0.4
      Getting image source signatures
      Copying blob 3aa8b87b7f88 done
      Copying blob 57584b59d88b done
      Copying blob 135070274eb1 done
      ...
      ...
      Storing signatures
      INFO: Processing artifact bluedata/hpecp-bootstrap-hpecp-agent:1.2.1-3 ...
      INFO: Copying artifact to my.local.registry.com/airgap-ecp54/bluedata/hpecp-bootstrap-hpecp-agent:1.2.1-3
      Getting image source signatures
      Copying blob ed5dc850ecaf done
      Copying blob 79f8ae5118d4 done
      Copying blob e17b1b3fe6d3 done
      ...
      ...
      Copying config 0168331b65 done
      Writing manifest to image destination
      Storing signatures
    • To copy a single image to a remote container registry, execute the following command. Provide the destination URL and credentials for your container registry.
      hpe-airgap-util --release <release-number> --image <image-name> --copy --dest_url <destination-url> --dest_creds <username:password>

Air Gap Utility Logging

By default, the Air Gap Utility creates a logs/ directory in the present working directory from which you invoked the Air Gap Utility command line.

You can change the log directory location as follows:
  • If you pass the --logdir argument in the Air Gap Utility command line, then the Air Gap Utility creates a logs/ directory in the path provided in the --logdir arguement.
  • If you set the AIRGAP_UTIL_LOGDIR environment variable, but do not pass the --logdir argument in the Air Gap Utility command line, then the Air Gap utility creates a logs/ directory in the path set in the AIRGAP_UTIL_LOGDIR environment variable.
NOTE
The Air Gap Utility does not create log files when commands are run in TTY mode. For example:
hpe-airgap-util --release 5.4 | grep -i argocd