HPE Data Fabric Streams Java API Library

Use the HPE Data Fabric Streams Admin Java API library as an alternative to maprcli commands and the REST APIs for performing administrative tasks on streams and topics. This library can also be used for analysis of the contents of streams.

Javadoc

The following Apache Kafka Java API versions are supported:
Table 1. Supported Apache Kafka APIs
Core version Apache Kafka API
7.8 and later 3.6.1
7.0 through 7.7 2.6.1
6.2 and later 2.1.1
6.1 and later 1.1
6.0.1 and later 1.0
6.0.0 and earlier 0.90

See the following APIs for detailed information:

HPE Data Fabric Streams Java APIs (as of 7.8)

The following HPE Data Fabric Streams Java APIs are available as of HPE Data Fabric 7.8:

Table 2. HPE Data Fabric Streams Java APIs applicable for HPE Data Fabric 7.8
Interface Method Description
StreamDescriptor void setCompact(boolean compact) Sets log compaction on a stream.
StreamDescriptor boolean getCompact() Gets the log compaction on a stream. Returns true if the stream has log compaction on the stream.
StreamDescriptor void setMinCompactionLagMS(long ts) Sets the time in (milliseconds) that a message should remain uncompacted in the topic-partition. Applies only if log compaction is enabled on the stream.
StreamDescriptor long getMinCompactionLagMS() Returns the minimum time (in milliseconds) a message will remain uncompacted in the topic-partition. Applies only if log compaction is enabled on the stream.
StreamDescriptor void setDeleteRetentionMS(long ts) Sets the time (in milliseconds) for which deleted records are retained. Applies only if log compaction is enabled on the stream.
StreamDescriptor long getDeleteRetentionMS() Returns the time (in milliseconds) for which deleted records are retained. Applies only if log compaction is enabled on the stream.
Producer ProducerConfig class The idempotence producer option is set by setting the enable.idempotence value of true passed through the ProducerConfig class.
NOTE
For Apache Kafka 3.6.1, some methods are not suppported.
Admin methods:
  • Using Topic IDs:
    • DescribeTopicsResult describeTopics(TopicCollection topics,  DescribeTopicsOptions options)
    • DeleteTopicsResult deleteTopics(TopicCollection topics, DeleteTopicsOptions options)
  • With more than one group:
    ListConsumerGroupOffsetsResult listConsumerGroupOffsets(Map<String,ListConsumerGroupOffsetsSpec> groupSpecs, ListConsumerGroupOffsetsOptions options)
  • And the following additional methods:
    public DescribeUserScramCredentialsResult describeUserScramCredentials(List<String> users, DescribeUserScramCredentialsOptions options) {
public AlterUserScramCredentialsResult alterUserScramCredentials(List<UserScramCredentialAlteration> alterations, AlterUserScramCredentialsOptions options) {
public DescribeFeaturesResult describeFeatures(DescribeFeaturesOptions options) {
public UpdateFeaturesResult updateFeatures(Map<String, FeatureUpdate> featureUpdates, UpdateFeaturesOptions options) {
public DescribeMetadataQuorumResult describeMetadataQuorum(DescribeMetadataQuorumOptions options) {
public UnregisterBrokerResult unregisterBroker(int brokerId, UnregisterBrokerOptions options) {
public DescribeProducersResult describeProducers(Collection<TopicPartition> partitions, DescribeProducersOptions options) {
public DescribeTransactionsResult describeTransactions(Collection<String> transactionalIds, DescribeTransactionsOptions options) {
public AbortTransactionResult abortTransaction(AbortTransactionSpec spec, AbortTransactionOptions options) {
Consumer methods:
  • OptionalLong currentLag(TopicPartition topicPartition)
  • void enforceRebalance(String reason)

HPE Data Fabric Streams Java APIs (as of 6.0.1)

The following table lists the new Interfaces and APIs for HPE Data Fabric 6.0.1. They are the delta between HPE Data Fabric 6.0.1 and 6.0.0, meaning, they are applicable to HPE Data Fabric6.0.1 but not HPE Data Fabric 6.0.0.

Table 3. HPE Data Fabric Streams APIs (as of 6.0.1)
Interface and Methods Description
Admin.close Long duration for TimeUnit.
Admin.createTopic TopicDescriptor array for topic attributes.
Admin.editTopic TopicDescriptor array for topic attributes.
Admin.getTopicDescriptor Method for retrieving topic attributes.
Admin.listTopic Method for listing all the topics in a stream.
Admin.streamExists Method for determining whether a stream exists.
StreamDescriptor.getDefaultTimestampType Method for retrieving the timestamp type.
StreamDescriptor.setDefaultTimestampType Method for setting the timestamp type.
TopicDescriptor New HPE Data Fabric interface.
TopicDescriptor.getPartitions Method associated with the new interface.
TopicDescriptor.setPartitions Method associated with the new interface.
TopicDescriptor.getTimestampType Method associated with the new interface.
TopicDescriptor.setTimestampType Method associated with the new interface.
Enum TimestampType New Enum class and associated methods.

Backward Compatibility

As of HPE Data Fabric 6.0.1, Apache Kafka 1.0 is supported. The following pause, resume, seekToBeginning, and seekToEnd APIs support the Collection Interface. The deprecated APIs will continue to run unchanged, however, they may be removed in a future release.
Table 4. Deprecated HPE Data Fabric APIs (as of 6.0.1)
Replacement Collection APIs Deprecated APIs
void pause(Collection<TopicPartition> partitions); void pause(TopicPartition... partitions);
void resume(Collection<TopicPartition>partitions); void resume(TopicPartition... partitions);
void seekToBeginning(Collection<TopicPartition>); void seekToBeginning(TopicPartition... partitions);
void seekToEnd(Collection<TopicPartition>); void seekToEnd(TopicPartition... partitions);
The following subscribe and assign APIs support the Collection Interface (which is more generalized) as well as the List Interface. Support for the List Interface has been retained for backward binary compatibility.
Table 5. Retained HPE Data Fabric APIs (as of 6.0.1)
Replacement Collection APIs Retained APIs
void subscribe(Collection<String> topics); void subscribe(java.util.List<java.lang.String> topics);
void subscribe(Collection<String> topics, ConsumerRebalanceListener); void subscribe(java.util.List<java.lang.String> topics, ConsumerRebalanceListener listener);
void assign(Collection<TopicPartition> partitions); void assign(java.util.List<TopicPartition> partitions);