MapR 5.1 is at End of Life (EOL) and no longer supported. Please see the latest documentation. This documentation is not being updated.

Home
5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
MapR Streams
MapR Streams brings integrated publish/subscribe messaging to the MapR Converged Data Platform.
Utilities for MapR Streams
MapR Streams provides the following utilities.

MapR 5.1 Documentation

5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
- MapR-DB
- MapR Streams
  MapR Streams brings integrated publish/subscribe messaging to the MapR Converged Data Platform.
  - MapR Streams Concepts
  - Life of a Message
    To show how the MapR Streams concepts fit together, here is an example of the flow of one message from a producer to a consumer.
  - Getting Started with MapR Streams
    If you have a basic understanding of MapR Streams's components and the typical flow of messages from producers to consumers, you can give MapR Streams a try.
  - MapR Streams in Detail
    If you understand the basics about the components of MapR Streams, the flow of messages from producers to consumers, and how producers and consumers are coded, it's time to cover them in more detail.
  - Monitoring the Flow of Messages
    The speed at which messages flow from producers to partitions and from partitions to consumers depends on the performance of your producers, cluster nodes hosting partitions, and your consumers.
  - Replicating MapR Streams
    You can replicate streams to other MapR clusters worldwide or to other streams within a MapR cluster.
  - Querying Messages in MapR Streams
    You can develop Java applications or modify existing Java applications to query messages that are in MapR streams by using the methods in the Streams class.
  - Migrating Apache Kafka 0.9.0 Applications to MapR Streams
    There are only two steps that you need to follow to migrate applications written with the Apache Kafka 0.9.0 Java API to MapR Streams.
  - Mirroring Topics with Apache Kafka's MirrorMaker
    Use Apache Kafka's MirrorMaker utility either to mirror topics that are in Apache Kafka clusters to streams that are in MapR clusters or to Mirror topics that are in MapR clusters to Apache Kafka clusters.
  - Utilities for MapR Streams
    MapR Streams provides the following utilities.
- MapR Gateways
  A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. MapR-DB binary tables and MapR Streams streams can be replicated.

Utilities for MapR Streams

MapR Streams provides the following utilities.

mapr copystream: This utility copies data from one MapR stream to another MapR stream. You can use it, for example, if you want to set up replication manually from one stream to another.
mapr diffstreams: This utility compares the message IDs, metadata, and data in two MapR streams. Then, generates two directories that contain sequence files that you can use to merge the rows from the two MapR streams.
mapr diffstreamswithcrc: This utility uses a cyclic redundancy check to detect differences between sets of messages in the specified MapR streams. Then, for each set of non-identical messages, it performs a detailed comparison. Finally, it generates one or more directories of sequence files. You can use these files either to make a MapR stream identical to its master or merge the messages from two MapR streams.
mapr exportstream and mapr importstream: Use these utilities together to export data from MapR streams into binary sequence files, and then import the data from the binary sequence files into other MapR streams. You can also use the mapr importstream utility to import changes that are specified in sequence files output by the mapr diffstreams utility.
mapr perfconsumer: This utility runs a consumer reading messages from topics in a MapR stream. Use this utility to run consumers when you want to estimate the performance of consumers for your MapR Streams applications, given your network configuration.
mapr perfproducer: This utility runs a producer, generating messages and publishing them to a MapR stream. Use this utility to run producers when you want to estimate the performance of producers for your MapR Streams applications, given your network configuration.
mapr streamanalyzer: This light-weight utility, which is a sample application for the Streams Java class for analytics on MapR streams, lets you count the messages in a stream or a subset of the topics in a stream. The utility also lets you print either whole retrieved messages or a subset of the fields in each message.