User Interface

Run the following command to see the keycloak-postgres pod logs:

kubectl -n keycloak logs keycloak-postgresql-0

Check the keycloak-postgres pod logs for the following Postgres error:

PANIC: replication checkpoint has wrong magic …

If you see this Postgres error, complete the following workaround steps to recover Postgres from Keycloak and then restart the pods.

1. Delete the checkpoint file:
   1. To get the volume name, run:

      VOLUMENAME=$(kubectl -n keycloak get pvc data-keycloak-postgresql-0 -o jsonpath='{.spec.volumeName}')

   2. To get the volume path, run:

      VOLUMEPATH=$(kubectl get pv $VOLUMENAME -o jsonpath='{.spec.csi.volumeAttributes.volumePath}')

   3. To remove the checkpoint file, run:

      kubectl -n dataplatform exec admincli-0 -- hadoop fs -rm -f "$VOLUMEPATH/data/pg_logical/replorigin_checkpoint"

2. Restart the keycloak and oauth2-proxy pods:
   1. To restart the keycloak-postgresql-0 pod, run the following command and then wait for the pod to restart:

      kubectl -n keycloak delete pod keycloak-postgresql-0

   2. To restart the keycloak-0 pod, run the following command and then wait for the pod to restart:

      kubectl -n keycloak delete pod keycloak-0

   3. To restart the oauth2-proxy pod, run the following command and then wait for the pod to restart:

      kubectl -n oauth2-proxy delete pod -l app=oauth2-proxy

To resolve this issue, complete the following steps:

1. SSH into the HPE Ezmeral Unified Analytics Software master node or have access to the kubeconfig of the HPE Ezmeral Unified Analytics Software cluster.
2. To identify the nodes running the OpenShift ingress pods, run:

   kubectl get pod -n   openshift-ingress -o wide

   Example

   [core@master0 ~]$ kubectl get pod -n   openshift-ingress -o wide
   NAME                             READY   STATUS    RESTARTS   AGE   IP               NODE                        NOMINATED NODE   READINESS GATES
   router-default-b6c47bcf6-5nlcm   1/1     Running   0          37h   10.227.209.134   worker2.pooja.ezfab.local   <none>           <none>
   router-default-b6c47bcf6-w7vcv   1/1     Running   0          38h   10.227.209.135   worker3.pooja.ezfab.local   <none>           <none>
   [core@master0 ~]$ 
   

3. To identify the nodes with the "ezkf.hpe.com/ingress-gateway=true" label, run:

   kubectl get node --show-labels | grep ingress-gateway

   NOTE

   The conflict occurs on the node(s) running OpenShift ingress pods that also have the "ezkf.hpe.com/ingress-gateway=true" label.
   Example

   [core@master0 ~]$ kubectl get node --show-labels | grep ingress-gateway
   worker2.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ezkf.hpe.com/ingress-gateway=true,hpe.com/compute=true,hpe.com/dataplatform=true,hpe.com/exclusivecluster=none,hpe.com/pin-dataplatform-cldb=true,hpe.com/pin-dataplatform-zk=true,hpe.com/status=available,hpe.com/usenode=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=worker1.pooja.ezfab.local,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,nvidia.com/gpu.deploy.operands=true
   worker4.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ezkf.hpe.com/ingress-gateway=true,hpe.com/compute=true,hpe.com/dataplatform=true,hpe.com/exclusivecluster=none,hpe.com/status=available,hpe.com/usenode=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=worker4.pooja.ezfab.local,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,nvidia.com/gpu.deploy.operands=true
   

   In this example output, the conflicting node is worker2.pooja.ezfab.local because this node has the OpenShift ingress pod running and also has the label. The "ezkf.hpe.com/ingress-gateway=true" label on this node must be deleted and then added to a different worker node.
4. On each node with the conflict, complete the following steps:
   1. To remove the label, run:

      kubectl label node <nodename> ezkf.hpe.com/ingress-gateway-

      NOTE

      The - at the end of ingress-gateway- is included to indicate deletion of the label.
   2. To list all worker nodes, run:

      kubectl get node

      Example

      [core@master0 ~]$ kubectl get node
      NAME                        STATUS   ROLES                  AGE   VERSION
      master0.pooja.ezfab.local   Ready    control-plane,master   46h   v1.25.16+5c97f5b
      master1.pooja.ezfab.local   Ready    control-plane,master   46h   v1.25.16+5c97f5b
      master2.pooja.ezfab.local   Ready    control-plane,master   46h   v1.25.16+5c97f5b
      worker0.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b
      worker1.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b
      worker2.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b
      worker3.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b
      worker4.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b
      worker5.pooja.ezfab.local   Ready    worker                 46h   v1.25.16+5c97f5b
      

   3. Identify one worker node that does not have the ingress gateway or the "ezkf.hpe.com/ingress-gateway=true" label and then run the following command to add the label to that node:

      kubectl label --overwrite node <nodename>  ezkf.hpe.com/ingress-gateway=true

      In the example, the following nodes qualify as worker nodes that could have the "ezkf.hpe.com/ingress-gateway=true" label added:

      worker0.pooja.ezfab.local
      worker1.pooja.ezfab.local
      worker5.pooja.ezfab.local

5. To verify that there are two running istio-ingressgateway pods, run:

   kubectl get pod -n istio-system

   NOTE

   An OpenShift cluster with three worker nodes has only one ​istio-ingressgateway pod in the Running state.
6. To get the ingressgateway IPs, run:

   kubectl -n istio-system get pod -l app=istio-ingressgateway -o jsonpath='{.items[*].status.hostIP}'

   Use the IP address to configure the DNS A records.

Failed to upload file.

If you are signed in to HPE Ezmeral Unified Analytics Software and the license has either expired or has not been uploaded, the URL redirects you to Keycloak instead of redirecting you to the page where you can upload a license, and Keycloak displays the following message:

Realm not enabled