Apache Kafka Wire Protocol Service
HPE Ezmeral Data Fabric Streams supports Apache Kafka Wire Protocol Service. Apache Kafka Wire Protocol Service is a TCP/IP service that emulates a Kafka cluster backed by HPE Ezmeral Data Fabric Streams. The service makes it possible for Apache Kafka clients written in any programming language to access topics in HPE Ezmeral Data Fabric Streams.
Before Apache Kafka Wire Protocol Service was introduced, you had to use a modified version of the open-source
libraries (kafka-clients.jar
and librdkafka.so
) for user
applications to connect and publish/subscribe messages to topics in HPE Ezmeral Data Fabric Streams. With Apache Kafka Wire Protocol Service, you can
simply use the Apache Kafka client libraries.
- User applications can connect, publish, and subscribe to HPE Ezmeral Data Fabric Streams topics using standard Apache Kafka client libraries. Starting with data access gateway 6.0, only topics created through Apache Kafka Wire Protocol Service are supported.
- User applications developed using Apache Kafka clients do not require any modification to work with this new system, including recompilation, reconfiguration, or dependency management.
- Supports available Apache Kafka clients in multiple programming languages, both from the Apache Software Foundation and the community.
For more information, see Getting Started with Apache Kafka Wire Protocol Service.
Architecture
The Apache Kafka Wire Protocol Service works within DAG (data access gateway) to act as a Kafka Broker (server in an Apache Kafka cluster). For all application purposes, the HPE Ezmeral Data Fabric cluster (with DAG and Apache Kafka Wire Protocol Service) runs like an Apache Kafka cluster.
Monitoring
DAG is not a Kafka Broker; therefore, Broker metrics are not available. However, you can monitor Data Fabric core services, as described in Using HPE Ezmeral Data Fabric Monitoring (Spyglass Initiative).
Limitations
- Quotas
- Transactions
- Idempotent Producer
- Kafka ACL-based security
- Topic-level security using ACEs.
- Group ACEs for Consumer Groups
- Delegation tokens
- Log directories