Reading Your Own Writes in Java OJAI

Description

You should use this feature if it is important for your query results to reflect synchronized data between a JSON table and its secondary indexes. Because HPE Data Fabric Database updates secondary indexes asynchronously, it is possible for a JSON table and its secondary indexes to become out-of-sync while the index update is in progress.

For example, consider the following scenario:

• Your application updates a JSON table.
• The JSON table includes an address field that is a nested document with a zipCode subfield.
• You have a secondary index on zipCode.
• Later in your application, you query the JSON table filtering on zipCode.

You want your query result to reflect the updates from earlier in your application. To achieve this, use the DocumentStore and Query APIs that enable you to retrieve up-to-date information from the index. The APIs synchronize write operations on the JSON table with read operations on a secondary index.

See Asynchronous Secondary Index Updates for more information about index updates.

API Details

The OJAI DocumentStore and Query interfaces provide the following methods to support this functionality.

DocumentStore.beginTrackingWrites

Begins tracking the write operations performed through this instance of DocumentStore. The method takes an optional previousWritesContext parameter. If you specify this parameter, the tracking uses that context as the base state.

DocumentStore.endTrackingWrites

Flushes any buffered writes operations for this DocumentStore and returns a writesContext. Use this context to ensure that writes are visible to later queries. You can use the context across DocumentStore objects in the same, as well as different, client processes, when the stores refer to the same JSON table. For example, you can pass the writesContext returned by one DocumentStore to a second DocumentStore, to begin write tracking on the second store.

DocumentStore.clearTrackedWrites

Stops the write tracking and clears any state on this DocumentStore instance.

Query.waitForTrackedWrites

Sets the writesContext parameter for this query. A writesContext allows this query to "see" all the writes that happened inside the writesContext of a DocumentStore.

For the complete API, see Java OJAI Client API.

Read Your Own Writes Example

A complete code example is available on github at OJAI_013_ReadYourOwnWrite.java. The following are code snippets from that example. Each step contains links to corresponding lines of code in the github example:

1. Call beginWriteTracking to set the starting point for the commit context on the JSON table /demo_table:

   // Create an OJAI connection to MapR cluster
   final Connection connectionNode1 = DriverManager.getConnection("ojai:mapr:");
   
   // Get an instance of OJAI DocumentStore
   final DocumentStore storeNode1 = connectionNode1.getStore("/demo_table");
   
   // initiate tracking of commit-context
   storeNode1.beginTrackingWrites();

2. Update the zipCode of an existing user and insert a new user in /demo_table:

   // issue a set of mutations/insert/delete/etc
   storeNode1.update("user0000", connectionNode1.newMutation().set("address.zipCode", 95110L));
   storeNode1.insertOrReplace(connectionNode1.newDocument(
       "{\"_id\": \"user0004\", \"firstName\": \"Joel\", \"lastName\": \"Smith\", \"age\": 56, \"address\": {\"zipCode\":{\"$numberLong\":95110}}}"));

3. Call endWriteTracking to flush the write operations after step 1, including updates to the secondary index:

   final String commitContext = storeNode1.endTrackingWrites();

   The call also returns a commitContext.

4. Issue a query that calls waitForTrackedWrites with the commitContext from step 3:

   /*
    * Next section of the code can run on the same or on a different node,
    * the `commitContext` obtained earlier needs to be propagated to that node.
    */
   
   // Create an OJAI connection to MapR cluster
   final Connection connectionNode2 = DriverManager.getConnection("ojai:mapr:");
   
   // Get an instance of OJAI DocumentStore
   final DocumentStore storeNode2 = connectionNode2.getStore("/demo_table");
   
   // Build an OJAI query and set its commit context with timeout of 2 seconds
   final Query query = connectionNode2.newQuery()
       .select("_id", "firstName", "lastName", "address.zipCode")
       .where("{\"$gt\": {\"address.zipCode\": 95110}}")
       .waitForTrackedWrites(commitContext)
       .build();

   The query filters on the indexed subfield address.zipCode. The commitContext ensures that the query result includes the changes made in step 2.