Migrating Apache Kafka Java Applications to HPE Data Fabric Streams

There are only two steps that you need to follow to migrate applications written with the Apache Kafka Java API to HPE Data Fabric Streams.

About this task

The following steps assume that migration is from either:
  • Apache Kafka 2.6.1 to HPE Data Fabric Streams 6.2 or higher
  • Apache Kafka 2.1.1 to HPE Data Fabric Streams 6.2 or higher
  • Apache Kafka 1.1 to HPE Data Fabric Streams 6.1 or higher
  • Apache Kafka 1.0 to HPE Data Fabric Streams 6.0.1 or higher
  • Apache Kafka 0.9.0 to HPE Data Fabric Streams 6.0.0 or earlier
IMPORTANT
For information on backward compatibility, see HPE Data Fabric Streams Java API Library

Procedure

  1. Change the names of topics to include the path and name of the MapR Stream stream in which the topic is located.
    Here is the syntax to use:
    /<path and name of stream>:<name of topic>
    For example, you might have a stream in a MapR cluster that is named stream_A, and the stream might be in a volume named IoT and in a directory named automobile_sensors. You want to redirect a producer application to a topic in that stream. The syntax of the path to the topic might look like this:
    /mapr/IoT/automobile_sensors/stream_A:<name of topic>
  2. If a producer application uses the Kafka interface Partitioner to compute which partitions to publish messages to, revise the application so that it uses the Kafka StreamsPartitioner interface instead.