HPE Ezmeral Data Fabric Object Store

The HPE Ezmeral Data Fabric Object Store is a native object storage solution that efficiently stores objects and metadata for optimized access.

You can deploy Object Store on-premises, on edge, or in the cloud and access data through multiple protocols, including the native S3 API, S3 Select, and mc, s3cmd, and aws CLI commands.

No limit on the number or size of buckets and objects used with Object Store exists. Data is secured through the following mechanisms.
Authentication
Users and applications authenticate to Object Store through S3 keys (accessKey/secretKey).
Authorization
Object Store authorizes access to buckets and objects through AWS S3-compliant resource and user policies.
Wire-level encryption
Object Store uses HTTPS.

Object Store leverages the patented HPE Ezmeral Data Fabric filesystem capabilities that protect data and make object storage reliable and scalable. This capability includes snapshots, mirroring, replication, global namespace, data tiering, and erasure coding. In addition to the filesystem features, Object Store provides:

Object Store UI
Administrators can manage accounts, buckets, objects, access policies, and users through a simple, intuitive interface.
Multi-part uploads
The ability to upload a large object as a set of contiguous parts. Each part is uploaded independently, in any order. Multi-part uploads are useful for objects greater than 100 MB.
WORM (Write Once Read Many)
When the Object Lock feature is enabled (using the CLI or UI), write operations that would normally overwrite existing objects result in the creation of new versions of that object in the same bucket.
Load balancing
Object Store supports:
  • DNS and VIP-based load balancing
  • IP-based load balancing
Multiple S3 gateways can serve one bucket, and any S3 gateway can serve any bucket.
Accounts
Like AWS accounts, accounts in Object Store are the administrative units that own buckets, policies, and users.

Use Cases

Some potential Object Store use cases include:
  • Archive data and build on-premises applications, or migrate to cloud-native applications.
  • Store media for operational use; reduce costs of storing globally distributed media, such as music, video, and images.
  • Run analytics on data with tools like Apache Spark, Apache Drill, Presto, and S3 Select to gain valuable insights into customers, operations, or markets.
  • Maintain Spark Delta Lake time travel information. You can time travel to see different versions of the data when Object Store is configured as a data lake for Spark Delta Lake.
  • Store ML model data and share the ML models in real-time with downstream applications.
  • Publish S3 events to HPE Ezmeral Data Fabric Streams.

Object Store vs S3 Gateway

The following table compares the HPE Ezmeral Data Fabric Object Store to the S3 Gateway that was previously supported (EEP 6.0 – EEP 8.0):

HPE Ezmeral Data Fabric Object Store S3 Gateway Ecosystem Component
Bucket namespaces (list of objects in a bucket) and object metadata are stored in database tables. Bucket namespaces and metadata are stored as files in a directory.
Object size determines how objects are stored for efficiency and optimized performance. Objects are all stored as files regardless of size.
Provides simultaneous bucket access from multiple cluster nodes. Storage restricts bucket access to one cluster node at a time.
Introduces the AWS account concept for bucket ownership, access control, and usage show back. No concept of account.
Object Store UI is included in the HPE Ezmeral Data Fabric Management Control System UI. Object storage UI is within the MinIO UI.