Getting Started with Apache Kafka Wire Protocol Service

Describes the steps required to install, configure, and use Apache Kafka Wire Protocol Service.

The following sections describe the prerequisites and steps required to use Apache Kafka Wire Protocol Service.

Prerequisites

Apache Kafka Wire Protocol Service has the following component version requirements:
  • HPE Ezmeral Data Fabric 7.3.0 or later
    WARNING

    Although, Apache Kafka Wire Protocol Service is available in Data Fabric versions 7.1 and 7.2, it is not forward compatible. To ensure continued support and compatibility for this service on these versions, upgrade to Data Fabric version 7.3 or later.

  • Data Access Gateway (DAG)

    To determine the DAG versions for supported combinations of core and EEPs, see Data Access Gateway Support Matrix. For information about EEP and core version support, see EEP Support and Lifecycle Status.

  • Apache Kafka clients 2.6.x

1 - Install Data Access Gateway

Apache Kafka Wire Protocol Service is included with the Data Access Gateway (DAG) package (mapr-data-access-gateway) and is installed when you install DAG.

You can install DAG on multiple Data Fabric data nodes alongside the mfs (file server) service through the Data Fabric Installer or manually using package managers. For more information about installing DAG, see Installing Data Access Gateway and Data Access Gateway Support Matrix.

High Availability
Apache Kafka Wire Protocol Service runs on all Data Fabric nodes that run DAG.
Port
The default DAG port for Apache Kafka Wire Protocol Service is 9092; however, you can change the port as described in Configuring Apache Kafka Wire Protocol Service.

2 - Configure Apache Kafka Wire Protocol Service

Set cluster and node-specific settings, as described in Configuring Apache Kafka Wire Protocol Service.

3 - Administer Kafka Topic

Use maprcli commands to administer Kafka topics as described in kafkatopic.

4 - Secure Apache Kafka Wire Protocol Service

See Securing Apache Kafka Wire Protocol Service.

5 - Use Apache Kafka Client Libraries to Connect to HPE Ezmeral Data Fabric Streams

For a list of available clients, see https://cwiki.apache.org/confluence/display/KAFKA/Clients.

6 - (Optional) - View Sample Applications

See Sample Kafka Python Producer and Consumer.