Moving Objects Across Fabrics and from AWS S3 to Fabric

Describes how objects are moved or transferred across buckets from one fabric to another or from Amazon S3 to Data Fabric.

The Object Copier Hub (OCH) is a service that facilitates the data movement across buckets between fabrics in a global namespace, and from Amazon S3 to Data Fabric.

The OCH service supports the copy, delete, and remote delete operations across fabrics and between Amazon S3 and Data Fabric.

The OCH service listens to bucket events and consumes notifications from AWS SQS for cloud S3 buckets or Data FabricStreams/Kafka for Data Fabric buckets.

The mapr-och package is required to run the Object Copier Hub service. The Object Copier Hub service can be run on all cluster/fabric nodes.

The following sections explain the the Object Copier Hub in detail.