About
This site contains documentation for HPE Ezmeral Runtime Enterprise, including installation, configuration, administration, and reference content, and information about related solutions. Examples of related solutions include HPE Ezmeral ML Ops and HPE Ezmeral Runtime Analytics for Apache Spark.
5.6 Reference
- HPE Ezmeral Runtime Enterprise 5.6
- Software Versions
- Kubernetes Bundles
  Kubernetes Bundles are software packages that can contain software to support newer Kubernetes versions, updated add-ons, and software fixes. Kubernetes Bundles enable you to update your deployment without requiring you to upgrade to a newer version of HPE Ezmeral Runtime Enterprise.
- Quick Links
- What's New in Version 5.6.x
  This topic summarizes the new features and important changes in HPE Ezmeral Runtime Enterprise 5.6.x compared to HPE Ezmeral Runtime Enterprise 5.5.x.
- Prepackaged Applications
- On-Premises, Hybrid, and Multi-Cloud Deployments
- Third-Party Licenses
- Universal Concepts
- Accessing HPE Ezmeral Runtime Enterprise Applications and Services
- Navigating the GUI
  Describes the screen layout of the HPE Ezmeral Runtime Enterprise graphical user interface (GUI).
- HPE Ezmeral ML Ops
  The topics in this section provide information about machine learning operations (ML Ops/MLOps) using HPE Ezmeral ML Ops in HPE Ezmeral Runtime Enterprise. (Not available with HPE Ezmeral Runtime Enterprise Essentials.)
- Spark on Kubernetes
  The topics in this section provide information about Apache Spark on Kubernetes in HPE Ezmeral Runtime Enterprise. (Not available with HPE Ezmeral Runtime Enterprise Essentials.)
- Kubernetes
  - Kubernetes Physical Architecture
  - Hewlett Packard Enterprise Distributions of Kubernetes
    The Hewlett Packard Enterprise distribution of Kubernetes, identified by the -hpe<number> suffix, incorporates the containerd runtime, which is required for all Kubernetes clusters created with HPE Ezmeral Runtime Enterprise version 5.5.0 and later.
  - Kubernetes Cluster Types and Compatibility
    An existing HPE Ezmeral Runtime Enterprise deployment that is upgraded from a previous release might contain Kubernetes clusters that use the containerd runtime and Kubernetes clusters that use the Docker runtime. All nodes in a Kubernetes cluster must use the same type of runtime.
  - Migrating Kubernetes Clusters from Docker to containerd
    This topic describes migrating legacy Kubernetes clusters from Docker container runtime to the the new Hewlett Packard Enterprise distribution of Kubernetes, which implements containerd runtime.
  - About HPE Ezmeral Data Fabric on Kubernetes
  - Kubernetes Tenant RBAC
  - Disabling or Enabling the Kubernetes Web Terminal
    As a Platform Administrator, you can enable or disable user access to the Kubernetes Web terminal. The Kubernetes Web Terminal is not available in HPE Ezmeral Runtime Enterprise Essentials.
  - Kubernetes Metadata
  - Centralized Policy Management
    Defines centralized policy management and describes the features and benefits of applying policies to Kubernetes clusters managed by HPE Ezmeral Runtime Enterprise. Not available in HPE Ezmeral Runtime Enterprise Essentials.
  - Kubernetes Troubleshooting Overview
  - Using Kubernetes
    The topics in this section describe information and tasks for non-administrator users of Kubernetes in HPE Ezmeral Runtime Enterprise.
  - Tenant/Project Administration
    The topics in this section describe information and tasks that Kubernetes Tenant/Project Administrators can perform in HPE Ezmeral Runtime Enterprise. .
  - Kubernetes Cluster Administrator Tasks
    The topics in this section describe information and tasks that Kubernetes Cluster Administrators can perform in HPE Ezmeral Runtime Enterprise.
  - Kubernetes Administrator Tasks
    The topics in this section describe information and tasks that Kubernetes Administrators can perform in HPE Ezmeral Runtime Enterprise.
    - Dashboard - Kubernetes Administrator
    - Toolbar and Main Menu - Kubernetes Administrator
    - Kubernetes Tenant Administration
      The topics in this section describe information and tasks related to Kubernetes tenant administration on HPE Ezmeral Runtime Enterprise.
    - Clusters
      The topics in this section describe information and tasks related cluster administration tasks performed by Kubernetes Administrators in HPE Ezmeral Runtime Enterprise.
    - Istio Service Mesh
      This topic describes Istio Service Mesh and its implementation and versions in HPE Ezmeral Runtime Enterprise.
    - Falco Container Runtime Security
      The Falco Container Runtime Security feature of HPE Ezmeral Runtime Enterprise improves container security and threat detection.
    - NVIDIA GPU Monitoring
    - Kubeflow
      Kubeflow is a machine learning (ML) toolkit for Kubernetes that makes deployments of ML workflows and pipelines on Kubernetes simple, portable and scalable.
    - Airflow
      Describes Airflow, an open-source workflow automation and scheduling system that can be used to author and manage data pipelines.
      - Airflow Requirements
        Describes the system, computation, and storage requirements for Airflow clusters.
      - Installing Airflow
        Describes how to install Airflow on a Kubernetes cluster in HPE Ezmeral Runtime Enterprise.
      - Creating an Airflow Cluster Automatically
        Describes how to create an Airflow Kubernetes cluster from a Git repository through the HPE Ezmeral Runtime Enterprise UI. This is the recommended method of Airflow cluster creation.
      - Creating an Airflow Cluster Manually
        This procedure describes an alternative method of creating an Airflow Kubernetes cluster. Use this method to perform extra tuning of the Airflow cluster through the command line. However, if no extra tuning is required, use the recommended method described in reference/kubernetes/kubernetes-administrator/airflow/creating_an_airflow_cluster_automatically.html.
      - Accessing Data From Outside Airflow DAGs with DataTap
        Describes how to access data from tenant storage in Airflow DAGs using DataTap. This can be done with either Airflow BashOperator or Airflow PythonOperator.
      - Notes about using Airflow
      - List of Airflow Images
      - List of Airflow Providers
    - Kubernetes Hosts
      The topics in this section describe information and tasks related to Kubernetes Hosts on HPE Ezmeral Runtime Enterprise.
    - Downloading Kubernetes Usage Details
      Platform Administrators can download scripts to view Kubernetes usage details in HPE Ezmeral Runtime Enterprise.
  - Kubernetes Application Administration
    The topics in this section describe information and tasks related to Kubernetes application administration on HPE Ezmeral Runtime Enterprise.
- Platform Administration
  Tasks and reference information for Platform Adminstrators (Site Administrators) managing the HPE Ezmeral Runtime Enterprise deployment.
- HPE Ezmeral Data Fabric Introduction
  HPE Ezmeral Data Fabric is a platform for data-driven analytics, ML, and AI workloads. The patented file-system architecture was designed and built for performance, reliability, and scalability. HPE Ezmeral Runtime Enterprise supports multiple implementations of HPE Ezmeral Data Fabric.
- HPE Ezmeral Data Fabric on Kubernetes Administration
  You administer HPE Ezmeral Data Fabric on Kubernetes and Embedded Data Fabric as part of your HPE Ezmeral Runtime Enterprise environment. The external "bare metal" implementation of HPE Ezmeral Data Fabric is administered through its own tools and has its own documentation. (Not available in HPE Ezmeral Runtime Enterprise Essentials.)
- GPU and MIG Support
  This topic provides information about support for NVIDIA GPU and MIG devices on HPE Ezmeral Runtime Enterprise.
- Licensing
- Global Settings
- Planning the Deployment
  A high-level overview of the items to consider when planning an HPE Ezmeral Runtime Enterprise deployment.
- System Requirements
- Deploying the Platform
  The topics in this section describe deploying HPE Ezmeral Runtime Enterprise. Deployment is divided into phases.
- Upgrading to HPE Ezmeral Runtime Enterprise 5.6.x
  This article describes the process to upgrade to the latest 5.6.x version of HPE Ezmeral Runtime Enterprise.
- Upgrading from HPE Ezmeral Runtime Enterprise Essentials
  Upgrade from HPE Ezmeral Runtime Enterprise Essentials to the full-featured HPE Ezmeral Runtime Enterprise or to HPE Ezmeral ML Ops by uploading a license. No additional steps are required.
- Manually Restarting HPE Ezmeral Runtime Enterprise Services
  This topic describes restarting HPE Ezmeral Runtime Enterprise services in non-Kubernetes hosts.
- Uninstalling and Reinstalling HPE Ezmeral Runtime Enterprise
- Support and Troubleshooting
App Workbench 5.1

Airflow Requirements

Describes the system, computation, and storage requirements for Airflow clusters.

System Requirements

Pod DNS Domain is cluster.local.
Cluster authentication is configured as AD or LDAP.

Computation Requirements

You can use DAGs to create many workers that can run simultaneously. In general, for an Airflow cluster with workers, approximately 5 CPUs and 5000Mi memory are needed.

Specifically, the following resources must be available:

To install the Airflow Operator and Base during cluster creation or editing:
- CPU: 1800m
- Memory: 4100Mi
To install Airflow cluster per tenant:
- CPU: 3000m
- Memory: 4000Mi
To launch one worker:
- CPU: 1500m
- Memory: 712Mi

Storage Requirements

Airflow requires dynamic-volume provisioning. Persistent storage must be configured and registered in the HPE Ezmeral Runtime Enterprise deployment. For information, see Storage.