HDFS Examples
These examples provides sample code for streaming data to and from the file system.
Streaming Data from HPE Ezmeral Data Fabric Streams to the File System
This example provides sample code for streaming data from HPE Ezmeral Data Fabric Streams to the file system.
POST /connectors HTTP/1.1
Host: connect.example.com
Content-Type: application/json
Accept: application/json
{
"name": "maprfs-sink-connector",
"config": {
"connector.class": "io.confluent.connect.hdfs.HdfsSinkConnector",
"tasks.max": "1",
"topics": "/kafka-connect:topic1",
"hdfs.url": "maprfs:///",
"flush.size": "5",
"rotate.interval.ms": "1000"
}
}
Streaming Data from HPE Ezmeral Data Fabric Streams to the File System in Parquet
This example provides sample code for streaming data from HPE Ezmeral Data Fabric Streams to the file system in Parquet.
POST /connectors HTTP/1.1
Host: connect.example.com
Content-Type: application/json
Accept: application/json
{
"name":"hdfs-connector-parquet",
"config":{
"connector.class":"io.confluent.connect.hdfs.HdfsSinkConnector",
"tasks.max":"10",
"topics":"/kafka-connect:topic2",
"hdfs.url":"maprfs:///",
"format.class":"io.confluent.connect.hdfs.parquet.ParquetFormat",
"flush.size":"3"
}
}