Prerequisites for Using Structured Streaming in Spark
To deploy a structured streaming application in Spark, you must create a Data Fabric Streams topic and install a Kafka client on all nodes in your cluster.
Creating a Data Fabric Streams Topic
Procedure
Create a Data Fabric Streams topic consisting of the
stream path and topic name separated by a colon (:); for example,
/test_stream:topic1
.
Installing a Kafka Client
Procedure
Install a kafka-client on all nodes of your cluster or copy the
kafka-clients.jar
file from
/opt/mapr/lib/kafka-clients-<version>mapr<release>.jar
to
/opt/mapr/spark/spark-<version>/jars/
.