AI and ML Project Workflow

This topic describes getting started with the AI and ML workflows in HPE Ezmeral Runtime Enterprise deployments that implement HPE Ezmeral ML Ops.

The AI/ML workflow enables you to build, train, and deploy a model, and then send API requests to that model in order to make predictions. This workflow consists of the following high-level steps, which users must perform in the following order in accordance with their roles:

  • Kubernetes Administrator
  • LDAP/AD Administrator (For Jupyter Notebook KDapp Use)
  • Project Administrator
  • Project Member (Data Scientist)

You can then make predictions, as described in Making Prediction Calls With Deployed Models.

Kubernetes Administrator

  1. Verify that the Platform Administrator has done the following:
    • Verified that HPE Ezmeral Runtime Enterprise is licensed for at least the number of CPU cores that will be used for the new Kubernetes cluster.
    • Configured LDAP/AD authentication.

      LDAP must be configured in order to run HPE Ezmeral ML Ops in a Kubernetes cluster. All AI/ML project users (Project Members and Project Administrators) must be LDAP/AD users. They cannot be authenticated using local authentication.

    • Configured and registered tenant storage on the HPE Ezmeral Runtime Enterprise deployment.
  2. Log into the web interface as a Kubernetes Administrator, as described in Launching and Signing In.

  3. Create a Kubernetes cluster, as described in Creating a New Kubernetes Cluster.

    IMPORTANT

    Be sure to provide LDAP server information in the Step 3: Authentication screen; LDAP must be configured in order to run HPE Ezmeral ML Ops in a Kubernetes cluster.

  4. Assign at least one user to be a Kubernetes Administrator for the Kubernetes cluster you just created. See Managing Kubernetes Admin Users (to assign a user role using local authentication) or Updating External Kubernetes Cluster Admin Groups (to assign a user role using LDAP/AD groups).
  5. Note the hostname or IP address of the Kubernetes control plane hosts. Control plane hosts have the role master in the Host(s) Info tab of the Kubernetes Cluster Details screen (see The Kubernetes Cluster Details Screen).
  6. Create a new Kubernetes AI/ML project, as described in Creating a New Kubernetes Tenant or Project. Ensure that you do the following:
  7. Assign at least one user to be a Kubernetes Project Administrator for the project you just created. See Viewing and Assigning Kubernetes Cluster Users.

LDAP/AD Administrator (For Jupyter Notebook KDapp Use)

If the environment will include the ability to use the Jupyter Notebook KubeDirector application (kdapp), LDAP server group settings must be changed for all members of the group.

The LDAP/AD Administrator must add member user IDs to user groups manually:

  1. Connect to the LDAP server.
  2. Access the Groups.
  3. For each group that has members that will log in to a Jupyter notebook, do the following:
    1. For each member, create a memberUid attribute that has a value of the member's user ID.

    The following example shows the entry for the Eng group after members have been added.


    group with memberUid attributes added
  4. You can verify which groups a member belongs to by selecting the entry for the member in People. For example:
    LDAP one member entry with groups highlighted

Kubernetes Project Administrator

  1. Confirm that the Kubernetes Administrator has completed all of the steps described in Kubernetes Administrator, above.
  2. Create a data source. See Adding Data Sources.

  3. If needed, create a new LDAP/AD user who will be assigned a role in the new project. If you create new users, The LDAP/AD Administrator might need to perform additional tasks (see LDAP/AD Administrator (For Jupyter Notebook KDapp Use)).

    NOTE
    All AI/ML project users (Project Members and Project Administrators) must be LDAP/AD users. They cannot be authenticated using local authentication.
  4. Assign at least one user to the new project as described in Assigning/Revoking User Roles (LDAP/AD/SAML).

  5. Configure one or more source control configuration templates, as described in Creating Source Control Configurations.

Kubernetes Project Member

Follow the instructions described in ML Ops Tasks to perform the following:
  • Create a source control configuration template.
  • Create a notebook server.
  • After running experiments in your notebook, view experiment results.
  • Register and deploy models.
  • Make prediciton calls with deployed models.