Accessing Data in External S3 Object Stores

Describes how to access data in external object stores from clients such as Spark and Kubeflow notebooks.

After an administrator connects HPE Ezmeral Unified Analytics Software to an external object store in AWS, MinIO, or HPE Ezmeral Data Fabric Object Store, you can access data in those data sources through clients such as Spark or Kubeflow notebooks, without providing an access key or secret key. Your HPE Ezmeral Unified Analytics Software administrator provides the access credentials when creating the data source connection. Your access to the data source is authorized through HPE Ezmeral Unified Analytics Software.

To connect a client to an object store, you provide the client with the following information:
  • Data source name
  • Endpoint URL
  • Bucket that you want the client to access
You can find the data source name and endpoint URL on the data source tile in the HPE Ezmeral Unified Analytics Software UI.

Once connected, clients can:

  • Read and download files in a bucket
  • Upload files from a bucket
  • Create buckets

Getting the Data Source Name and S3 Proxy Endpoint URL

To get the data source name and S3 proxy endpoint URL:

  1. Sign in to HPE Ezmeral Unified Analytics Software.
  2. In the left navigation bar, select Data Engineering > Data Sources.
  3. On the Data Sources page, find the tile for the object store that you want to connect to.

    The following image shows an example of a tile for an AWS S3 data source with the name aws-s3 and the enpoint URL:

    NOTE
    By default, a local-s3 Ezmeral Data Fabric tile also displays. This Ezmeral Data Fabric version of S3 is a local S3 version used internally by HPE Ezmeral Unified Analytics Software. Do not connect to this data source.
  4. Note the data source name and endpoint URL and then use them to configure the client.