MapR 5.1 is at End of Life (EOL) and no longer supported. Please see the latest documentation. This documentation is not being updated.

Home
5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
MapR Gateways
A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. MapR-DB binary tables and MapR Streams streams can be replicated.
Gateways for Indexing MapR-DB Data in Elasticsearch
When you index MapR-DB tables in Elasticsearch, MapR-DB replicates table updates to corresponding Elasticsearch types. The MapR-DB tables are in MapR clusters, and the types are in Elasticsearch clusters. Gateways receive table updates and pass them to nodes in Elasticsearch clusters.
Replicating to a MapR Cluster that is also Indexing Data in Elasticsearch
You can also replicate from a MapR cluster to a cluster that is indexing data in Elasticsearch. Use the maprcli cluster gateway set command if you want to use one subset of your gateways for table replication and the other subset for indexing.

MapR 5.1 Documentation

5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
- MapR-DB
- MapR Streams
  MapR Streams brings integrated publish/subscribe messaging to the MapR Converged Data Platform.
- MapR Gateways
  A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. MapR-DB binary tables and MapR Streams streams can be replicated.
  - Gateways for Replicating MapR-DB Binary Tables
    In table replication, MapR-DB replicates updates to binary tables on source MapR clusters to replicas of those tables on destination MapR clusters. Gateways are services that receive these updates and apply them to the replicas.
  - Gateways for Replicating MapR Streams Streams
    When replicating streams, MapR Streams replicates messages that are published to a source stream. Gateways are services that receive messages from source streams and publish them in replica streams.
  - Gateways for Indexing MapR-DB Data in Elasticsearch
    When you index MapR-DB tables in Elasticsearch, MapR-DB replicates table updates to corresponding Elasticsearch types. The MapR-DB tables are in MapR clusters, and the types are in Elasticsearch clusters. Gateways receive table updates and pass them to nodes in Elasticsearch clusters.
    - Configuring a MapR Gateway on an Independent Node for Elasticsearch Indexing
      This topic provides instructions for setting up MapR Gateway on a node for the purpose of replicating MapR-DB binary tables to Elasticsearch indexes. This configuration procedure is applicable for independent gateway nodes and gateway nodes on a Elasticsearch cluster.
    - Gateway Placement for Elasticsearch Indexing
      This topic describes several gateway topologies used for replicating MapR data for Elasticsearch indexing.
    - Replicating and Indexing from a Single MapR Source Cluster
      You can replicate to one or more MapR clusters and index table data in one or more Elasticsearch clusters all from a single MapR cluster.
    - Replicating to a MapR Cluster that is also Indexing Data in Elasticsearch
      You can also replicate from a MapR cluster to a cluster that is indexing data in Elasticsearch. Use the maprcli cluster gateway set command if you want to use one subset of your gateways for table replication and the other subset for indexing.
    - Types of Elasticsearch Clients for Gateways to Use
  - Configuring a MapR Gateway Master-Slave Topology
    Configuring gateways involves installing the mapr-gateway package on nodes in a MapR destination cluster and then configuring the MapR source cluster to communicate with the destination cluster. The Mapr source cluster is configured by specifying the destination cluster's CLDB node(s) and gateway nodes.
  - Managing Gateways
    This topic describes various methods for listing gateways, checking status of gateways, managing gateways if they fail, and troubleshooting gateways.

Replicating to a MapR Cluster that is also Indexing Data in Elasticsearch

You can also replicate from a MapR cluster to a cluster that is indexing data in Elasticsearch. Use the maprcli cluster gateway set command if you want to use one subset of your gateways for table replication and the other subset for indexing.

For example, suppose that you want to replicate from the source MapR cluster sanfrancisco to the destination MapR cluster newyork. You also want to index table data in Elasticsearch cluster newyork_es. You envision a configuration like this one:

As in the next diagram, you configure four gateways in the cluster newyork, planning to use two for table replication and two for indexing, as depicted in this diagram of what you want your configuration to look like. For indexing, you plan to place the two gateways physically in the Elasticsearch cluster, though logically they will part of the cluster newyork.

However, as the next diagram shows, what will happen is that table replication will use all of the gateways, and so will indexing.

This configuration will not necessarily slow down the performance of the Elasticsearch nodes where the gateways are located. However, if you do notice any slow down as you test your configuration, you could try partitioning the gateways by using the maprcli cluster gateway set command. When you run this command on the sanfrancisco cluster, specify only gateways A and B. When you run this command on the newyork cluster, specify only gateways C and D. By using this method of specifying only a subset of the gateways on the newyork cluster both times, you partition the gateways so that they are either handling traffic for table replication or handling traffic for indexing, but not handling both.