Release Notes (1.5.4)

This document provides a comprehensive overview of the latest updates and security fixes in HPE Ezmeral Unified Analytics Software (version 1.5.4).

HPE Ezmeral Unified Analytics Software provides software foundations for enterprises to develop and deploy end-to-end data and advanced analytics solutions from data engineering to data science and machine learning across hybrid cloud infrastructures – delivered as a software-as-a-service model.

Version 1.5.4 Updates

Version 1.5.4 of HPE Ezmeral Unified Analytics Software includes the following updates:
NOTE
HPE Ezmeral Unified Analytics Software version 1.5.4 does not support RHEL 8.10. For supported OS versions, see Operating System.

Installation

Before you install or upgrade, HPE recommends that you back up your data.  If you encounter any issues during or after the installation or upgrade process, please contact HPE Support. We appreciate your feedback and strive to continually enhance your product experience. 

Upgrade Notes

Back-up custom configurations

Settings, including certificates and custom configurations, do not persist to the new version of HPE Ezmeral Unified Analytics Software. You must back up all custom application configurations and certificates prior to performing an upgrade. After you upgrade HPE Ezmeral Unified Analytics Software, you must reconfigure applications that had custom configurations and apply certificates.

Manually download the workload cluster upgrade bundle

You cannot automatically download the workload cluster upgrade bundle. You must use the manual procedure described in Download the Workload Cluster Upgrade Bundle.

Configure the Air Gap registry URL
If HPE Ezmeral Unified Analytics Software 1.5.3 was installed without enabling the Air Gap environment or the Air Gap registry URL is not configured for your existing 1.5.3 setup (URL is an empty string), you must manually set the Air Gap registry URL to marketplace.us1.greenlake-hpe.com/ezua/. To set the Air Gap registry URL to marketplace.us1.greenlake-hpe.com/ezua/, run the following commands on the Kubernetes master node:
kubectl patch cm ezua-cluster-config -n ezua-system --type merge -p '{"data":{"airgap.registryUrl":"marketplace.us1.greenlake-hpe.com/ezua/"}}'

kubectl patch ezac in-cluster -n ezaddon-system --type merge -p '{"spec":{"registryUrl":"marketplace.us1.greenlake-hpe.com/ezua/"}}'

Known Issues

Katib jobs fail when launched through Kale
If you launch a Katib job through Kale from a notebook, the Katib job fails because resource limits are not provided. Pods get stuck in a pending state and the system returns a warning message stating that resource limits must be defined.

Workaround

To work around this issue:
IMPORTANT
In air-gapped environments or environments that cannot access docker.io, push the akravacyber/katib-kfp-trial:v0.7.0 image into the air gap registry that is configured for the cluster. Update line 8 in the kale-katib.patch file to add the air gap registry address as the prefix to akravacyber/katib-kfp-trial:v0.7.0.
  1. Download the following file and put it in the /mnt/user directory:
    kale-katib.patch
  2. Open a notebook terminal and run the following command:
    cd /opt/conda/lib/python3.11/site-packages
  3. From the notebook terminal, run the following command:
    git apply /mnt/user/kale-katib.patch
  4. Close all the open notebook tabs and shut down all the kernels running in notebooks.
  5. In the top menu bar, select File > Log Out.
  6. Log in again.

Cannot set Git-Sync container limits through the UI
Airflow DAGs use a sidecar container for Git-Sync. The sidecar container consumes resources for every DAG run. You cannot limit the resources allocated to the container through the HPE Ezmeral Unified Analytics Software UI.

Workaround

If you have kubectl access to the master node, you can change the git-sync setting in the AirflowCluster CR. If you complete the following steps to change the CR, the Airflow cluster operator will not update the statefulset anymore. For example, if the git repo changes, you will need to update the env in the statefulset manually unless you delete the statefulset.
  1. Stop the Airflow operator statefulset in the airflow-hpe namespace (scale to 0).
    kubectl scale  sts -n airflowop-system  airflowop-controller-manager --replicas=0
  2. Edit the UI statefulset and remove all labels and ownerReferences. This blocks the operator so it can be udpated.
    NOTE
    Back up the sts before making any changes.
    kubectl edit sts -n airflow-hpe af-cluster-airflowui

 
//Remove labels:
labels:
   custom-resource: v1alpha1.AirflowCluster

//Remove ownerReferences:
ownerReferences:
 - apiVersion: airflow.hpe.com/v1alpha1
  blockOwnerDeletion: true
  controller: true
  kind: '*v1alpha1.AirflowCluster'
  name: af-cluster
  uid: 5e13d8db-6d69-4460-9e42-346c230a3c9e
  3. Edit the git-sync resource settings.
    apiVersion: apps/v1
kind: StatefulSet
~~~
   name: airflow-ui
        resources:
          limits:
            cpu: 1160m
            memory: 3080Mi
          requests:
            cpu: 320m
            memory: 1640Mi
~~~
       name: git-sync
        ports:
        - containerPort: 2020
          name: gitsync
          protocol: TCP
        resources:
          limits:
            cpu: 400m
            memory: 800Mi
          requests:
            cpu: 80m
            memory: 410Mi
  4. Verify the pod status in the airflow-hpe namespace.
    kubectl get pod af-cluster-airflowui-0 -n airflow-hpe -o jsonpath="{range .spec.containers[*]}Name: {.name}, Requests: CPU={.resources.requests.cpu}, Memory={.resources.requests.memory}, Limits: CPU={.resources.limits.cpu}, Memory={.resources.limits.memory}{'\n'}"
  5. Scale up the operator statefulset.
    kubectl scale  sts -n airflowop-system  airflowop-controller-manager --replicas=1
NOTE
If you want the operator to automatically update the statefulset in the future, delete the statefulset. Once deleted, the operator recreates it automatically.
kubectl delete sts -n airflow-hpe af-cluster-airflowui

//Run the following commands to verify that sts is running and verify resource allocations:
kubectl get sts,po -n airflow-hpe |grep airflowui
kubectl get pod af-cluster-airflowui-0 -n airflow-hpe -o jsonpath="{range .spec.containers[*]}Name: {.name}, Requests: CPU={.resources.requests.cpu}, Memory={.resources.requests.memory}, Limits: CPU={.resources.limits.cpu}, Memory={.resources.limits.memory}{'\n'}"

Additional Resources

Thank you for choosing HPE Ezmeral Unified Analytics Software. Enjoy the new features and improvements introduced in this release.