HPE Kubernetes Cluster Troubleshooting
Troubleshooting Kubernetes clusters that are running the Hewlett Packard Enterprise distribution of Kubernetes can involve examining
the .service
files, environment variables, and using journald to examine
logs.
This article contains troubleshooting information that is specific to Kubernetes clusters that are running the Hewlett Packard Enterprise distribution of Kubernetes, which uses the containerd runtime.
Service Files
Each Kubernetes service has its own .service
file.
/etc/systemd/system/kubelet.service
/etc/systemd/system/kube-proxy.service
/etc/systemd/system/kube-apiserver.service
Environment Variables
ezctl
configures
are saved at the following location:
/opt/ezkube/bootstrap/systemd
The environment variables are editable, but changes might be overwritten when updating Kubernetes.
Logs
Journald manages the logs for all the services.
- By default, all log messages are logged to:
/var/log/messages
- To get the log for a service, enter the appropriate
journalctl
command.For example, the following command jumps to the end of the log of thekube-apiserver
unit:journalctl -e -u kube-apiserver
Kubernetes Audit Information
/opt/ezkube/k8s-audit-policy.yaml
/var/log/ezkube/audit/k8s-audit.log
Kubernetes Node Bring-Up
/var/log/bluedata/install/ezctl-config-<date>.yaml
The following logs are related to Kubernetes node bring-up. The contents of the logs
are also logged to the standard
/var/log/bluedata/install/k8s_cluster_*
log files:
/var/log/bluedata/install/ezctl-<date>.log
/var/log/bluedata/install/ezctl_pod_deletes.log