Replica Autosetup for HPE Ezmeral Data Fabric Database Tables
The option to automatically set up table replication, also known as replica autosetup, performs the steps to set up and start the replication of HPE Ezmeral Data Fabric Database binary table and HPE Ezmeral Data Fabric Database JSON tables. The replica autosetup option is available through the Control System and maprcli commands.
In general, replica autosetup performs the following steps to set up replication:
- Creates a new table with metadata from the source table in the destination cluster. The primary table attributes are copied initially if an auto setup is used, because auto setup creates the destination table and any subsequent changes to the primary table's metadata are not propagated to the destination table. In the manual table replication setup, no metadata is propagated, even during the setup.
- Declares the new table to be a replica of the source table and sets a paused replication state to ensure that replication does not begin immediately after the next step.
- Declares the source table as an upstream source for the replica.
- Loads a copy of the source data into the replica.
- For multi-master replication, replica autosetup declares the source table to be a replica of the new table and then declares the new table to be an upstream source for the source table.
- Clears the paused replication state to start the replication stream.
Replica Autosetup with Directcopy (default)
The directcopy option uses gateways to perform all setup operations
including the initial population of data into the replica table. Directcopy is the default
option when you setup table replication using the Control System or with the maprcli
table replica autosetup
command.
When a client submits a request to automatically setup table replication to the cluster, the source cluster acknowledges the request and begins to track the replica autosetup request from start to finish.
If a failure occurs when replica autosetup operations are in progress, the source cluster resumes operations from the point of failure.
maprcli table replica
list
command. To stop the automatic setup of table replication, run
maprcli table replica
remove
, or delete the source or replica table. - Replica autosetup operations do not block the client from submitting additional requests. When setting up table replication, the process to copy source data to the replica can be time consuming. The client does not need to wait for the replica autosetup request to complete before submitting another request.
- Source cluster retries replica autosetup operations in case of failure. The source cluster keeps track of the replica autosetup progress. This allows the source cluster to resume autosetup operations in the event of an intermittent failure. If you choose to not use directcopy, user intervention is required if a failure occurs.
- Throttling of copy table operations is done by default. Throttling prevents the initial copy of data from the source to the replica table from consuming all cluster resources.
Replica Autosetup without Directcopy (not default)
Without the directcopy option, replica autosetup submits a majority of the replication setup
requests through the client and then runs a copy table utility to populate the initial table
data. To use replica autosetup without the directcopy option, run maprcli table
replica autosetup
command with the -directcopy
parameter set to
false
.
Without the directcopy option, once a client submits a replica autosetup request to the cluster,
it must wait until the source cluster sends a notification that the autosetup request is
complete before it can submit another request to the cluster. In this case, replica
autosetup uses the client connection to submit autosetup operation requests such as create
replica, add replica, and add upstream source. To populate the initial table data, the
client runs mapr copytable
for JSON tables and the
CopyTable
utility for binary tables.
If a failure occurs when replica autosetup operations are in progress, the client hangs and any replica tables that were created during the failed autosetup operations must be manually deleted before trying to setup replication again.