Replication Rules JSON Format

Describes how replication rules work for object copying from AWS S3 bucket or Data Fabric S3 bucket to Data Fabric S3 bucket.

Object replication rules

The object copier hub evaluates objects to be copied from an AWS S3 bucket or a Data Fabric S3 bucket to a Data Fabric S3 bucket against a set of replication rules. The replication rule configuration contains the replication rules. The rule configuration is a JSON file.

The object copier hub evaluates the replication configuration file against each incoming S3 event and copies matching objects to the configured destination bucket.

Note the following with respect to the replication configuration rules:
  • You can have multiple such rules in a single JSON file.
  • The source bucket and the destination bucket combination must be unique in each rule.
  • The rule must be enabled for object replication to take place from the source bucket to the destination bucket.
  • Rules are evaluated in priority order; the highest-priority rule is the rule where all filters pass.
Following is the JSON schema for the rule configuration file for a single rule. 
{
  "replicationConfigs": {
    "<source-bucket-name>": {
      "rules": [
        {
          "id": "<rule-id>",
          "priority": <priority between 1 and 10, 1 being the highest and 10 being the lowest>,
          "status": "<Enabled | Disabled>",
          "filter": {
            "prefix": "optional/prefix/",
            "objectSizeGreaterThan": <object size in MB>,
            "objectSizeLessThan": <object size in MB>,
            "tag": { "key": "tier", "value": "hot" },
            "and": {
              "prefix": "prefix/",
              "objectSizeGreaterThan": <object size in MB>,
              "objectSizeLessThan": <object size in MB>,
              "tags": [ { "key": "<key-name>", "value": "<key-value>" } ]
            }
          },
          "destination": { "bucket": "<destination-bucket-name>" },
          "deleteMarkerReplication": { "status": "Enabled | Disabled" },
          "existingObjectReplication": { "status": "Enabled | Disabled" }
        }
      ]
    }
  }
}

You can have multiple such rules configured in the JSON rules configuration file.

Features supported for replication

The following features are supported for replication:

  • Prefix filter
  • Rule status (Enabled/Disabled)
  • Priority resolution
  • Multiple rules / multi-destination
  • Multi-source isolation
  • Tag filter (single + AND block)
  • Object size filter
  • Delete Marker Replication
  • Delete event classifier
  • Unversioned delete replication
  • Bidirectional loop prevention
  • ExistingObjectReplication
  • Metadata preservation
  • VersionId in copy
  • Multipart copy
  • AWS STS session token

Features not supported for replication

The following features are not supported for replication:
  • SSE-KMS Encrypted Objects
  • Replica KMS Key
  • IAM Role ARN (rule field)
  • Storage class override
  • ReplicationTime Control (RTC)
  • ACL / Owner Translation
  • Object Lock Replication