Using KServe to Deploy a Model on S3 Object Storage

Add YAML configurations that perform the following actions and then run the code from a Kubeflow notebook:

• Create a service account
• Create a secret
  IMPORTANT

  • When you create the secret, note that the {os.environ['AUTH_TOKEN']} option assigns a value to the AWS_ACCESS_KEY_ID. The value assigned is the value on the JWT for the current notebook user.
  • When accessing object store data via the S3 proxy in Unified Analytics, enter "s3" as the AWS_SECRET_ACCESS_KEY value. For additional information about the S3 proxy, see Configuring a Spark Application to Access External S3 Object Storage.
• Deploy a model (InferenceService)
• Apply the YAML (!kubectl apply -f {yaml_name})

The following example shows a YAML configuration:

best_model_uri =    '<path_to_the_model>'  # for example 's3://mlflow/2/0e4508d276a0427cb67da7630acb2e14/artifacts/model'
secret_name = 's3-proxy-kserve-secret'
sa_name = 's3-proxy-kserve-sa'
inference_service_name = "service-name"
yaml_name = './s3-proxy-kserve.yaml'

############################################

with open(yaml_name, 'w') as file:
    text = f"""---
apiVersion: v1
kind: Secret
metadata:
  name: "{secret_name}"
  annotations:
    serving.kserve.io/s3-cabundle: ""
    serving.kserve.io/s3-endpoint: "local-s3-service.ezdata-system.svc.cluster.local:30000/"
    serving.kserve.io/s3-useanoncredential: "false"
    serving.kserve.io/s3-usehttps: "0"
    serving.kserve.io/s3-verifyssl: "0"
stringData:
  AWS_ACCESS_KEY_ID: "{os.environ['AUTH_TOKEN']}"
  AWS_SECRET_ACCESS_KEY: "s3"
type: Opaque

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: "{sa_name}"
secrets:
  - name: "{secret_name}"

---
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "{inference_service_name}"
spec:
  predictor:
    serviceAccountName: "{sa_name}"
    sklearn:
      protocolVersion: "v2"
      storageUri: "{best_model_uri}"
"""
    file.write(text)
   
############################################
!kubectl apply -f {yaml_name}