Troubleshooting Kubernetes Host Installation

Kubernetes Host Installation Fails: Prechecks Errors

To determine which precheck error occurred, do the following:

  1. Use SSH to access the host that has the prechecks error.
  2. Run the following command: 
    ls -laht /tmp | grep prechecks
  3. Find the most recent prechecks log. The name of the file is in the following format: 
    bd_prechecks.{number}.log
  4. Read through the file to find the error. Prechecks errors have the following format:
    Checking {description_of_test}: FAILED

If the error is not one of the following errors, contact Hewlett Packard Enterprise Technical Support.

Error: Falco kernel module schema_version must be in the 2.x.y range

Symptom

The Kubernetes host installation fails and the following prechecks error is returned:

Falco kernel module schema_version must be in the 2.x.y range

Cause

The version of the Falco Kernel Module is out of date and does not contain the correct version of the schema.

Action

Update the Falco Kernel Module on this host to version 0.32 or later. Hewlett Packard Enterprise recommends using the latest released version. You must use the Linux modprobe tool the Linux modprobe tool to install the module.

Error: Falco kernel module must be loaded with modprobe to be allowed in ERE

Symptom

The Kubernetes host installation fails and the following precheck error is returned:

Falco kernel module must be loaded with modprobe to be allowed in ERE

Cause

The Falco Kernel Module was installed using a method other than using the Linux modprobe tool.

Action

Delete the Falco Kernel Module, then use the Linux modprobe tool to reinstall the module. For information about modprobe, see the modprobe(8) manpage.

Kubernetes Host Installation Fails: Security Error

Symptom
The Kubernetes host installation fails and a security error is returned.
Cause
The local times on the Controller host and the Kubernetes hosts differ significantly.
Action

Set the local time on the Kubernetes host to match the local time on the Controller host. Then begin the installation process again.

Kubernetes Host Installation Fails: Storage Error

Symptom

This environment is an air-gapped environment using a secure container registry, and when you attempt to add a Kubernetes host, the following occurs:

  • A storage error is returned.
  • An error message similar to the following is added to the bds-mgmt.log:
    dictionary update sequence element #3 has length 4; 2 is required
Cause

The container client certificate uses an RSA key length other than 4086 bits.

Action

Ensure that the certificate key uses RSA 4086. Upload a certificate that uses the correct RSA key length, and then begin the installation again. For more information about the client certificate, see Air Gap Tab.

Logs for Troubleshooting Kubernetes Hosts

If you experience other issues when installing a Kubernetes host, then access the following logs:

  • Controller host:
    • Host Installer log:
      /var/log/bluedata/install/addworker.out_.log
    • Xtrace file: This file is a verbose, line-by-line description of the exact commands used by the script to both get data and determine the outcome of each test. 
      /var/log/bluedata/addworker/install.out_.log.xtrace
  • Kubernetes host:
    • Host setup log: 
      /var/log/bluedata/install/worker_setup_<timestamp>
    • Host Xtrace file: This file is a verbose, line-by-line description of the exact commands used by the script to both get data and determine the outcome of each test. 
      /var/log/bluedata/install/worker_setup_<timestamp>.xtrace

Using Logs to Troubleshoot Issues

General steps:

  1. Begin reading the logs from top to bottom.

  2. Stop at the first ERROR that you find. This first error can often cause additional problems downstream. Taking a start-to-finish approach (instead of working your way back from the tail end of the log file) can help you solve one error that in turn resolves a series of cascading errors. If the problem is obvious, then correct the problem and retry the installation.

  3. If you are unable to resolve the problems on your own, then contact Hewlett Packard Enterprise for support. You might be asked to provide these installer logs and xtrace files.