Object Lifecycle
Explains the object lifecycle for objects stored in the HPE Data Fabric Object Store.
The object lifecycle or the information lifecycle, is a set of S3-compliant rules that govern the expiration of objects stored within the HPE Data Fabric Object Store. The object lifecycle or the information lifecycle is managed by using the Information Lifecycle Management(ILM) commands or the Data Fabric UI.
- MinIO CLI via mc cli commands (mc ilm - add, edit, export, import, ls, rm)
- AWS CLI via s3 APIs (put-bucket-lifecycle-configuration, get-bucket-lifecycle-configuration, delete-bucket-lifecycle-configuration)
- MinIO SDK via APIs (PutBucketLifecycleConfiguration, GetBucketLifecycleConfiguration, DeleteBucketLifecycleConfiguration)
- AWS SDK via APIs (setBucketLifecycle, getBucketLifecycle, deleteBucketLifecycle)
The following use cases fall under the scope of the object life cycle:
-
Data expiration: Set time-based rules to automatically delete objects that are no longer needed after a certain period, after ensuring compliance with data retention policies.
-
Object versioning management: Manage multiple versions of an object and allow automatic deletion of older versions to save space while maintaining data integrity.
-
Compliance management: Implement data retention policies that are based on regulatory requirements. Set expiration dates for sensitive data.
-
Data lifecycle governance: Define rules for data management throughout its lifecycle, including data movement and data deletion. Per-bucket configuration supports multiple rules for deletion.
An object life cycle rule contains one action and one or more conditions. Object-related rules can have filters based on object name prefix, tags, or object size.
For example, you can define rules that would perform the following operations automatically:
- Delete objects with a prefix or suffix 90 days after creation or last update to another bucket.
- Delete uncommitted or failed multipart uploads after 3 days.
- Delete any previous object versions 60 days after the object version transitions from the latest version to a previous version.
To use object lifecycle management, you must define a lifecycle configuration that can be used for one or more S3 buckets. The configuration is a set of rules that apply to current and future objects in the bucket.
ILM Scanner
The Information Lifecycle Management(ILM) scanner is an S3 server background process that runs once every 24 hours for each bucket. Use the commands start, stop, monitor, and control scan throttling.
The following table lists the options related to the ilm commands on MinIO CLI and AWS CLI, and the support for ILM options in AWS SDK and MinIO SDK.
| ILM option | MinIO CLI option | AWS CLI option | supported on AWS SDK? | supported on MinIO SDK? |
|---|---|---|---|---|
| current object expiration | expire-days, expire-date | Expiration | yes | yes |
| noncurrent object version expiration | noncurrent-expire-days, noncurrent-expire-newer | NonCurrentVersionExpiration | yes | yes |
| Expired delete marker cleanup | expire-delete-marker | ExpiredObjectDeleteMarker | yes | yes |
| Filters | prefix, tags, size-lt, size-gt | Filter | yes | yes |
| Abort multi-part | abort-incomplete-multipart-upload-days | abort-incomplete-multipart-upload-days | yes | yes |
See topics related to object lifecycle rules under Administering Buckets for Data Fabric UI operations related to object lifecycle management.