Release Notes (1.5.6)

Settings, including certificates and custom configurations, do not persist to the new version of HPE Ezmeral Unified Analytics Software. You must back up all custom application configurations and certificates prior to performing an upgrade. After you upgrade HPE Ezmeral Unified Analytics Software, you must reconfigure applications that had custom configurations and apply certificates.

You cannot automatically download the workload cluster upgrade bundle. You must use the manual procedure described in Download the Workload Cluster Upgrade Bundle.

Workaround

Restart the KubeRay pods.

Cause

CSV downloads are served through the internal file download service hpecp-webhdfs. Version 2.3.0 of hpecp-webhdfs has a hard-coded timeout that is not long enough to finish streaming very large files, so the request is cut off and fails before completion. This is unrelated to the query itself; the query runs and completes successfully in Presto. Only the download step fails.

Workaround

To fix the issue, a new version of the webhdfs image (2.8.0) has been built with the timeout issue fixed and validated (successfully tested with a 2GB+ file download).

Workaround

1. After upgrading your HPE Ezmeral Unified Analytics Software nodes to RHEL 8.10, see: EZUA 1.5.6 - Required Actions After RHEL 8.10 Node Upgrade.
2. To update the GPU driver version:

   1. For air-gapped environments, download the driver image from the marketplace and place it inside the customer registry:

      marketplace.us1.greenlake-hpe.com/ecp-airgap/nvcr.io/nvidia/driver:550.127.05-rhel8.10

   2. Patch the driver from version 550.54.15 to version 550.127.05 in custerpolicy. For example:

      • Old version:

        kubectl get clusterpolicies.nvidia.com cluster-policy -o jsonpath='{.spec.driver.version}'
        550.54.15

      • Patch:

        kubectl patch clusterpolicy cluster-policy --type='merge' -p '{"spec":{"driver":{"version":"550.127.05"}}}'

      • New version:

        kubectl get clusterpolicies.nvidia.com cluster-policy -o jsonpath='{.spec.driver.version}'
        550.127.05

   3. Validate driver pod status:

       kubectl get po -n hpecp-gpu-operator|grep nvidia-driver
      nvidia-driver-daemonset-ml74w                      1/1     Running   0             31h

When you execute %manage_spark in any of the Spark kernels within a Notebook and then add an endpoint through the wizard, the contents of the Create Session tab are not completely visible unless you reduce the page resolution. Also, the wizard is not scrollable horizontally which hides the Create Session button. This issue occurs because the version of Jupyter Notebook in HPE Ezmeral Unified Analytics Software version 1.5.5 does not support overflow scrolling.

Workaround

Enlarge the window for full content visibility.

When you use the %createKernel magic command to create custom Jupyter kernels, packages installed via conda may not be accessible within the kernel. This occurs because kernel registration does not properly link the conda environment's PATH. This issue affects:

• Python packages installed during kernel creation, such as numpy and pandas
• CLI tools installed via conda, such as pigz

Workaround

Note that this resolution has the following known limitations:

• Applies to pip-installable packages only
• Conda-specific packages or CLI tools may not be available using this method
• You must repeat these steps for each new notebook using a custom kernel

1. In python, run the %createKernel magic command without specifying any packages, or pass a minimal dummy package:

   %createKernel
   

2. Log out of the notebook environment and then log back in to ensure that the new kernel is properly registered.
3. Open a new notebook and select the newly created kernel (myenv) from the kernel selection menu.
4. Inside the new notebook, install your required packages using pip:

   !pip install numpy pandas scikit-learn

   If packages fail to download, verify that all HTTP proxy settings are correctly configured.

Error creating: Internal error occurred:
failed calling webhook "admission-webhook-deployment.kubeflow.org": 
failed to call webhook: Post "https://admission-webhook-service.kubeflow.svc:443/apply-poddefault?timeout=10s": 
tls: failed to verify certificate: x509: certificate signed by unknown authority

Workaround

Restart the pods in the cert-manager namespace. After the pods start, restart the admission-webhook-deployment pod in the kubeflow namespace.

kubectl delete po -n cert-manager --all --force --grace-period=0
kubectl delete po -n kubeflow admission-webhook-deployment-<deployment_id>
kubectl delete po -n cert-manager --all --force --grace-period=0

Workaround

Perform the following steps BEFORE upgrading HPE Ezmeral Unified Analytics Software:

1. Get the current Airflow Postgres Database image:
   1. To get the airflow-base YAML, run the following command:

      kubectl get airflowbase af-base -n airflow-base -o yaml

   2. In the YAML, under spec, get the airgapRegistry field, and under postgres get the image and version fields.
2. Find out which Airgap Registry will be used for the platform after upgrade. This is the value of the airgap container registry that will be used for the new platform. The team that performs the upgrade should be able to provide this information.
3. Compare the values from steps 1 and 2 and verify the following:
   1. Verify that the airgapRegistry prior to upgrade is not the same as the airgap registry that will be used after the upgrade. If the airgapRegistry is not the same, you must update it (step 4).
   2. Verify that the image prior to upgrade is not the same as <airgap-registry>gcr.io/mapr-252711/postgres. Where <airgap-registry> is the airgap registry which will be used after upgrade (step 2). If the image is not the same, you must update it (step 4).
   3. Verify that the current version (before upgrade) is not 14.12. If the version is not 14.12, you must update it (step 4).
4. Edit the airflow-base YAML by running the following command:

   kubectl edit airflowbase af-base -n airflow-base

5. Update the airgapRegistry, image, and version fields with the expected values and then save the changes. Wait for database readiness.
6. Upgrade HPE Ezmeral Unified Analytics Software.

   Example

   If an airflow-base YAML has the following fields and data:

   • airgapRegistry: ""
   • image: gcr.io/mapr-252711/postgres
   • version: 14.12

   And you consult with the upgrade team to find out what the fields will be post upgrade, for example:

   • airgapRegistry: marketplace.us1.greenlake-hpe.com/ezua/
   • image: marketplace.us1.greenlake-hpe.com/ezua/gcr.io/mapr-252711/postgres
   • version: 14.12(the same)

   IMPORTANT

   You must update the fields in the YAML to match what the upgrade team has provided before performing the HPE Ezmeral Unified Analytics Software upgrade.

Workaround

1. Download the following file and put it in the /mnt/user directory:

   kale-katib.patch

2. Open a notebook terminal and run the following command:

   cd /opt/conda/lib/python3.11/site-packages

3. From the notebook terminal, run the following command:

   git apply /mnt/user/kale-katib.patch

4. Close all the open notebook tabs and shut down all the kernels running in notebooks.
5. In the top menu bar, select File > Log Out.
6. Log in again.

In HPE Ezmeral Unified Analytics Software setups, etcd runs as a system managed service (not a static pod) with default TLS settings for broad compatibility. Supports weak 3DES ciphers (64-bit block size) alongside strong ones like AES-GCM and ChaCha20-Poly1305 (TLS 1.2/1.3). Security scanners flag Sweet32 (CVE-2016-2183) because any weak cipher triggers the alert. This is due to cipher negotiation defaults, etcd remains secure and not externally exposed.

Remedy