Configure Storm Topology to Work with MapR Streams

About this task

For Storm to work with MapR Streams, you must configure the Storm serializer and deserializer properties for each topology.

Procedure

  1. Verify that the Streams Client (mapr-kafka) is installed on one node in the Storm cluster. See Installing Storm
  2. On the node where the Streams Client is installed, create a link between the Storm library and the Streams Client JAR file. 
    ln -s /opt/mapr/lib/kafka-clients-0.9.0.0* ${STORM_HOME}/lib/
  3. On any Nimbus node or Supervisor nodes where the Streams client is not installed, copy the Streams Client JAR file into the Storm library.
  4. Create a src/main/resources/kafka.properties file.
  5. Add the following serializer and deserializer properties to the kafka.properties file: 
    #Producer
key.serializer=org.apache.kafka.common.serialization.StringSerializer
value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
#Consumer
key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
  6. When defining the topology in the Storm program, refer to this file for setting the properties of the topology.