MapR 5.1 is at End of Life (EOL) and no longer supported. Please see the latest documentation. This documentation is not being updated.

Home
5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
MapR Gateways
A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. MapR-DB binary tables and MapR Streams streams can be replicated.
Gateways for Indexing MapR-DB Data in Elasticsearch
When you index MapR-DB tables in Elasticsearch, MapR-DB replicates table updates to corresponding Elasticsearch types. The MapR-DB tables are in MapR clusters, and the types are in Elasticsearch clusters. Gateways receive table updates and pass them to nodes in Elasticsearch clusters.

MapR 5.1 Documentation

5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
- MapR-DB
- MapR Streams
  MapR Streams brings integrated publish/subscribe messaging to the MapR Converged Data Platform.
- MapR Gateways
  A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. MapR-DB binary tables and MapR Streams streams can be replicated.
  - Gateways for Replicating MapR-DB Binary Tables
    In table replication, MapR-DB replicates updates to binary tables on source MapR clusters to replicas of those tables on destination MapR clusters. Gateways are services that receive these updates and apply them to the replicas.
  - Gateways for Replicating MapR Streams Streams
    When replicating streams, MapR Streams replicates messages that are published to a source stream. Gateways are services that receive messages from source streams and publish them in replica streams.
  - Gateways for Indexing MapR-DB Data in Elasticsearch
    When you index MapR-DB tables in Elasticsearch, MapR-DB replicates table updates to corresponding Elasticsearch types. The MapR-DB tables are in MapR clusters, and the types are in Elasticsearch clusters. Gateways receive table updates and pass them to nodes in Elasticsearch clusters.
    - Configuring a MapR Gateway on an Independent Node for Elasticsearch Indexing
      This topic provides instructions for setting up MapR Gateway on a node for the purpose of replicating MapR-DB binary tables to Elasticsearch indexes. This configuration procedure is applicable for independent gateway nodes and gateway nodes on a Elasticsearch cluster.
    - Gateway Placement for Elasticsearch Indexing
      This topic describes several gateway topologies used for replicating MapR data for Elasticsearch indexing.
    - Replicating and Indexing from a Single MapR Source Cluster
      You can replicate to one or more MapR clusters and index table data in one or more Elasticsearch clusters all from a single MapR cluster.
    - Replicating to a MapR Cluster that is also Indexing Data in Elasticsearch
      You can also replicate from a MapR cluster to a cluster that is indexing data in Elasticsearch. Use the maprcli cluster gateway set command if you want to use one subset of your gateways for table replication and the other subset for indexing.
    - Types of Elasticsearch Clients for Gateways to Use
  - Configuring a MapR Gateway Master-Slave Topology
    Configuring gateways involves installing the mapr-gateway package on nodes in a MapR destination cluster and then configuring the MapR source cluster to communicate with the destination cluster. The Mapr source cluster is configured by specifying the destination cluster's CLDB node(s) and gateway nodes.
  - Managing Gateways
    This topic describes various methods for listing gateways, checking status of gateways, managing gateways if they fail, and troubleshooting gateways.

Gateways for Indexing MapR-DB Data in Elasticsearch

When you index MapR-DB tables in Elasticsearch, MapR-DB replicates table updates to corresponding Elasticsearch types. The MapR-DB tables are in MapR clusters, and the types are in Elasticsearch clusters. Gateways receive table updates and pass them to nodes in Elasticsearch clusters.

You can place gateways on existing nodes in your source MapR cluster, on existing nodes in your Elasticsearch cluster, or on nodes that are not part of either cluster -- wherever you find that the network performance from source MapR cluster to gateway and from gateway to Elasticsearch cluster is best.

NOTE: When gateways are on nodes that are part of an Elasticsearch cluster, the gateways are invisible to Elasticsearch. All management of gateways is done from the source MapR cluster. If you use the maprcli cluster gateway set command, then the -dstcluster parameter is set to the MapR source cluster. MapR-DB would then understand that indexing to the Elasticsearch cluster goes through that gateway.

Wherever you place MapR gateways that you use for indexing, they become part of the source MapR-DB cluster for the following two reasons:

After you install the mapr-gateway package on a node, you run the configure.sh script, supplying the name of the MapR cluster that the gateway belongs to as the value for the -N parameter.
When you tell your source MapR cluster where the gateways are, you list the gateway nodes together with the name of the MapR cluster that they belong to.

As a result, you can manage gateways from the maprcli when you are logged into the source MapR cluster. Also, gateways are able to access information on the source MapR cluster about source tables and their corresponding Elasticsearch types.