HDFS Examples

These examples provides sample code for streaming data to and from the file system.

Streaming Data from HPE Ezmeral Data Fabric Streams to the File System

This example provides sample code for streaming data from HPE Ezmeral Data Fabric Streams to the file system.

POST /connectors HTTP/1.1
        Host: connect.example.com
        Content-Type: application/json
        Accept: application/json
              
        {
        "name": "maprfs-sink-connector",
        "config": {
        "connector.class": "io.confluent.connect.hdfs.HdfsSinkConnector",
        "tasks.max": "1",
        "topics": "/kafka-connect:topic1",
        "hdfs.url": "maprfs:///",
        "flush.size": "5",
        "rotate.interval.ms": "1000"
        }
        }

Streaming Data from HPE Ezmeral Data Fabric Streams to the File System in Parquet

This example provides sample code for streaming data from HPE Ezmeral Data Fabric Streams to the file system in Parquet.

POST /connectors HTTP/1.1
        Host: connect.example.com
        Content-Type: application/json
        Accept: application/json
        
        {  
        "name":"hdfs-connector-parquet",
        "config":{  
        "connector.class":"io.confluent.connect.hdfs.HdfsSinkConnector",
        "tasks.max":"10",
        "topics":"/kafka-connect:topic2",
        "hdfs.url":"maprfs:///",
        "format.class":"io.confluent.connect.hdfs.parquet.ParquetFormat",
        "flush.size":"3"
        }
        }