Data Types and Secondary Index Fields

Prior to data-fabric 6.1, the indexed fields in a secondary index had to contain scalar data. For each scalar data value, HPE Ezmeral Data Fabric Database stored a row in the index. See the table in the Scalar Data section of JSON Document Data Types for a list of scalar types.

Beginning with data-fabric 6.1, indexed fields can also be nested documents or arrays, but not array elements. As with scalar data values, HPE Ezmeral Data Fabric Database stores a row in the index for each nested document and array. The index improves equality filters on the entire nested document or array.

data-fabric 6.1 also supports using container field paths as indexed fields.

The following table summarizes what HPE Ezmeral Data Fabric Database supports, depending on the characteristics of the indexed field:

To understand what HPE Ezmeral Data Fabric Database stores for an indexed field defined on different data types, consider an example in which you have the following documents:

{"_id":"0", "field":0}
{"_id":"1", "field":[0,1,2]}
{"_id":"2", "field":{"subField":1}}
{"_id":"3", "field":{"subField":[1,2,3]}}
{"_id":"4", "field":[{"subField":1},{"subField":2}]}
{"_id":"5", "field":[{"subField":[1,2,3]},{"subField":[4, 5]}]}

The following table shows what an index defined on field stores and an OJAI query condition that matches the value stored in the index:

0

{"$lt":{"field":1}}

[0,1,2]

{"$eq":{"field":[0,1,2]}}

{"subField":1}

{"$eq":{"field":{"subField":1}}}

{"subField":[1,2,3]}

{"$eq":{"field":{"subField":[1,2,3]}}}

[{"subField":1},{"subField":2}]

{"$eq":{"field":[{"subField":1},{"subField":2}]}}

[{"subField":[1,2,3]},{"subField":[4, 5]}]

{"$eq":{"field":[{"subField":[1,2,3]},{"subField":[4,5]}]}}

The following table shows what an index defined on field.subField stores and an OJAI query condition that matches the value stored in the index:

1

{"$lt":{"field.subField":5}}

[1,2,3]

{"$eq":{"field.subField":[1,2,3]}}

These indexes enable HPE Ezmeral Data Fabric Database to quickly look up values stored in the index. As shown in the table, these values can be scalars, arrays, or nested documents. In the case of the latter two types, HPE Ezmeral Data Fabric Database can only use the index for equality conditions.

As shown in these examples, defining an index on a container field path is different from defining an index on an entire array field. For example, an index on field[] can filter on individual array elements, whereas the index on field can filter only the entire value. Similarly, defining an index on field[].subField[] provides the most generality. It allows you to filter on any elements in subField, regardless of the data types in both field and subField. However, you also incur the overhead of storing more data in your index and the performance impact of updating the index.

With a container field path, you may need to add included fields in your index to make the index covering. See Covering Indexes and Container Field Paths for details.

There are also limitations in the composite indexes you can define. See Composite Indexes and Container Field Paths for details.