Getting Started with AI and ML in Kubernetes

Kubernetes Administrator

1. Verify that the Platform Administrator has done the following:
   • Verified that HPE Ezmeral Runtime Enterprise is licensed for at least the number of CPU cores that will be used for the new Kubernetes cluster.
   • Configured LDAP/AD authentication.

     LDAP must be configured in order to run HPE Ezmeral ML Ops in a Kubernetes cluster. All AI/ML project users (Project Members and Project Administrators) must be LDAP/AD users. They cannot be authenticated using local authentication.

   • Configured and registered tenant storage on the HPE Ezmeral Runtime Enterprise deployment.
2. Log into the web interface as a Kubernetes Administrator, as described in Launching and Signing In.

3. Create a Kubernetes cluster, as described in Creating a New Kubernetes Cluster.

   IMPORTANT

   Be sure to provide LDAP server information in the Step 3: Authentication screen; LDAP must be configured in order to run HPE Ezmeral ML Ops in a Kubernetes cluster.

4. Assign at least one user to be a Kubernetes Administrator for the Kubernetes cluster you just created. See Managing Kubernetes Admin Users (to assign a user role using local authentication) or Updating External Kubernetes Cluster Admin Groups (to assign a user role using LDAP/AD groups).
5. Note the hostname or IP address of the Kubernetes control plane hosts. Control plane hosts have the role master in the Host(s) Info tab of the Kubernetes Cluster Details screen (see The Kubernetes Cluster Details Screen).
6. Create a new Kubernetes AI/ML project, as described in Creating a New Kubernetes Tenant or Project. Ensure that you do the following:
   • Check the AI/ML Project check box.
   • Enter the external LDAP/AD user group in the External Authentication tab (see Kubernetes Tenant/Project External Authentication).
7. Assign at least one user to be a Kubernetes Project Administrator for the project you just created. See Viewing and Assigning Kubernetes Cluster Users.

LDAP/AD Administrator (For Jupyter Notebook KDapp Use)

If the environment will include the ability to use the Jupyter Notebook KubeDirector application (kdapp), LDAP server group settings must be changed for all members of the group.

The LDAP/AD Administrator must add member user IDs to user groups manually:

1. Connect to the LDAP server.
2. Access the Groups.
3. For each group that has members that will log in to a Jupyter notebook, do the following:
   1. For each member, create a memberUid attribute that has a value of the member's user ID.

   The following example shows the entry for the Eng group after members have been added.

   
   
   
4. You can verify which groups a member belongs to by selecting the entry for the member in People. For example:
   
   

Kubernetes Project Administrator

1. Confirm that the Kubernetes Administrator has completed all of the steps described in Kubernetes Administrator, above.
2. Create a project repository, as described in Creating/Updating a Kubernetes Project Repository.
3. Browse the project repository to ensure that you can create/remove directories and subdirectories, and upload/download files. See Browsing the Kubernetes Project Repository.

4. If needed, create a new LDAP/AD user who will be assigned a role in the new project. If you create new users, The LDAP/AD Administrator might need to perform additional tasks (see LDAP/AD Administrator (For Jupyter Notebook KDapp Use)).

   NOTE

   All AI/ML project users (Project Members and Project Administrators) must be LDAP/AD users. They cannot be authenticated using local authentication.
5. Assign at least one user to the new project as described in Assigning/Revoking User Roles (LDAP/AD/SAML).

6. Configure one or more global source control configurations, as described in The Kubernetes Source Controls Screen and Adding a New Global Kubernetes Source Control Configuration.

Kubernetes Project Member (Data Scientist)

To use MLflow for model management, see MLflow for Model Management.

1. Confirm that the Kubernetes Project Administrator has completed all of the steps described in Kubernetes Project Administrator, above.
2. Log in as the project user that was created or assigned in Step 5 of the Kubernetes Project Administrator workflow.
3. Configure at least one individual source control repository, as described in Adding a New Individual Kubernetes Source Control.
4. Access the Kubernetes Training screen (see The Kubernetes Training Screen), and then onboard the necessary training applications, as described in Launching Training. You can then access the application endpoints, as described in Training Endpoints Tab.
5. Access the Kubernetes Notebooks screen (see The The Kubernetes Notebooks Screen), and then launch the notebook application, as described in Launching a Notebook.
6. After the notebook status appears as Configured, you can then view your notebooks applications, as described in Notebook Applications Tab, and can also access the application endpoints.
7. Select the Jupyterlab endpoint from the Notebook Endpoints tab, as described in Notebook Endpoints Tab.
8. Log in to Jupyterlab using your AD/LDAP credentials, and then launch a Python 3 notebook.

9. Edit the following in the Python cell. Provide your AD/LDAP password:

   # use to set kubeconfig context
   from ezmllib.kubeconfig.ezkubeconfig import set_kubeconfig
   
   # Please uncomment and set the password to run this example.
   # PASSWORD = ''
   set_kubeconfig(PASSWORD)

   The password is not stored in the notebook, so unauthorized users cannot use the notebook to access the server.

   You may now begin entering kubectl commands within the notebook.

10. Open the Model Management screen, and then click the Register New Model button to open the Register Model screen. See Registering/Updating a Model in a Kubernetes Project.
11. Register the serialized model, being sure to include the Model Version and Path to Model Repo. See Registering/Updating a Model in a Kubernetes Project.

12. Open the Kubernetes Model Serving screen (see The Kubernetes Model Serving Screen).

13. Onboard the necessary training applications, as described in Launching Model Serving.

    You may then access the application endpoints, as described in Ezmeral Serving Endpoints Tab and Launching Model Serving.

You may now use the deployed model to make predictions, as described in Making Predictions.

Making Predictions

To make predictions:

1. Create a Postman API call that is formulated as follows:
   • Prefix: Either http:// or https://, depending on how the environment is configured. Typically, production environments use HTTPS.
   • Body: The access point from the haproxy service of the LoadBalancer role in the Deployment Endpoints tab. See Ezmeral Serving Endpoints Tab.

   • Suffix: Registered model name and version number, in the format /model_name_registered/version_number. For example:

     /mymodel/1

   • Ending:/predict

2. Verify that the finished API call looks similar to this example:

   http://hostname.enterprise.com:10022/mymodel/1/predict

3. In the Deployment Endpoints tab, click the Copy Auth Token link for the LoadBalancer role.
4. Launch Postman, and then enter the following information:
   • Request URL: The URL you created in Step 2.
   • Content-Type: application/json
   • X-AUTH-TOKEN: Auth token copied in Step 3, above.
5. Click Send to get a prediction.