Prerequisites for Using Structured Streaming in Spark

To deploy a structured streaming application in Spark, you must create a Data Fabric Streams topic and install a Kafka client on all nodes in your cluster.

Creating a Data Fabric Streams Topic

Procedure

Create a Data Fabric Streams topic consisting of the stream path and topic name separated by a colon (:); for example, /test_stream:topic1.

Installing a Kafka Client

Procedure

Install a kafka-client on all nodes of your cluster or copy the kafka-clients.jar file from /opt/mapr/lib/kafka-clients-<version>mapr<release>.jar to /opt/mapr/spark/spark-<version>/jars/.