Configuration Parameters for MapR Streams Java Producers

You can control some of the aspects of how producers publish messages by setting various configuration parameters every time you start a producer.

Configuration parameters supported from Apache Kafka

buffer.memory
The total bytes of memory the producer can use to buffer records waiting to be sent to the server. If records are generated faster than they can be delivered to the server the producer will block.
Default: 33554432
client.id
Producers can tag records with a client ID that identifies the producer. Consumers can then be aware of which producer sent a message or set of messages. Apache Drill or other analytic tools querying messages can include this ID in the filters for their queries.
Default: No client ID.
key.serializer
The name of the appropriate serialization class in the org.apache.kafka.common.serialization package or a class that implements the Serializer interface for serializing keys.
metadata.max.age.ms
The producer generally refreshes the topic metadata from the server when there is a failure. It will also poll for this data regularly.
Default: 600 * 1000 msec
value.serializer
The class that implements the Serializer interface for serializing values.

Configuration parameters specific to MapR Streams

streams.buffer.max.time.ms
Messages are buffered in the producer for at most the specified time. A thread will flush all the messages that have been buffered for more than the time specified.
Default: 3 * 1000 msec
streams.parallel.flushers.per.partition
If enabled, producer may have multiple parallel send requests to the server for each topic partition. If this setting is set to true, it is possible for messages to be sent out of order.
Default: true
streams.partitioner.class
The class that implements the StreamsPartitioner interface. This interface lets you write custom algorithms for determining which topic and partition to use for messages that match specific criteria. Use this configuration parameter only for producers that are written in Java.
streams.producer.default.stream
Specifies the stream that the producer will use by default if the producer does not provide the name of a stream when specifying a topic to write to.
The syntax is
/mapr/<cluster name>/<volume name>/<stream name>
fs.mapr.hardmount
Specifies whether to use a hard mount or a soft mount for connections to the MapR Streams server.

The default is to use a hard mount and the value is true.

If a value for this parameter is set in the core-site.xml file, the value in that file is ignored.

fs.mapr.rpc.timeout
Specifies the length of time in seconds to wait for a response from the MapR Streams server if the configuration parameter fs.mapr.hardmount is set to false.

The default value is 300. The minimum value is 30.

If a soft mount is used, the time expires while a producer waits for a response from the MapR Streams server, and the producer used the KafkaProducer.send(ProducerRecord<K,V> record, Callback callback) method, the callback is invoked with the error EAGAIN, which means "Resource temporarily unavailable."

If a value for this parameter is set in the core-site.xml file, the value in that file is ignored.