MapR 5.1 is at End of Life (EOL) and no longer supported. Please see the latest documentation. This documentation is not being updated.

Home
5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
MapR Gateways
A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. MapR-DB binary tables and MapR Streams streams can be replicated.
Gateways for Indexing MapR-DB Data in Elasticsearch
When you index MapR-DB tables in Elasticsearch, MapR-DB replicates table updates to corresponding Elasticsearch types. The MapR-DB tables are in MapR clusters, and the types are in Elasticsearch clusters. Gateways receive table updates and pass them to nodes in Elasticsearch clusters.
Types of Elasticsearch Clients for Gateways to Use

MapR 5.1 Documentation

5.1 DB & Streams
This section covers the MapR-DB, MapR-Streams, and MapR-Gateways features. Each feature may contain both administrator and developer topics.
- MapR-DB
- MapR Streams
  MapR Streams brings integrated publish/subscribe messaging to the MapR Converged Data Platform.
- MapR Gateways
  A MapR gateway mediates one-way communication between a source MapR cluster and a destination cluster. MapR-DB binary tables and MapR Streams streams can be replicated.
  - Gateways for Replicating MapR-DB Binary Tables
    In table replication, MapR-DB replicates updates to binary tables on source MapR clusters to replicas of those tables on destination MapR clusters. Gateways are services that receive these updates and apply them to the replicas.
  - Gateways for Replicating MapR Streams Streams
    When replicating streams, MapR Streams replicates messages that are published to a source stream. Gateways are services that receive messages from source streams and publish them in replica streams.
  - Gateways for Indexing MapR-DB Data in Elasticsearch
    When you index MapR-DB tables in Elasticsearch, MapR-DB replicates table updates to corresponding Elasticsearch types. The MapR-DB tables are in MapR clusters, and the types are in Elasticsearch clusters. Gateways receive table updates and pass them to nodes in Elasticsearch clusters.
    - Configuring a MapR Gateway on an Independent Node for Elasticsearch Indexing
      This topic provides instructions for setting up MapR Gateway on a node for the purpose of replicating MapR-DB binary tables to Elasticsearch indexes. This configuration procedure is applicable for independent gateway nodes and gateway nodes on a Elasticsearch cluster.
    - Gateway Placement for Elasticsearch Indexing
      This topic describes several gateway topologies used for replicating MapR data for Elasticsearch indexing.
    - Replicating and Indexing from a Single MapR Source Cluster
      You can replicate to one or more MapR clusters and index table data in one or more Elasticsearch clusters all from a single MapR cluster.
    - Replicating to a MapR Cluster that is also Indexing Data in Elasticsearch
      You can also replicate from a MapR cluster to a cluster that is indexing data in Elasticsearch. Use the maprcli cluster gateway set command if you want to use one subset of your gateways for table replication and the other subset for indexing.
    - Types of Elasticsearch Clients for Gateways to Use
  - Configuring a MapR Gateway Master-Slave Topology
    Configuring gateways involves installing the mapr-gateway package on nodes in a MapR destination cluster and then configuring the MapR source cluster to communicate with the destination cluster. The Mapr source cluster is configured by specifying the destination cluster's CLDB node(s) and gateway nodes.
  - Managing Gateways
    This topic describes various methods for listing gateways, checking status of gateways, managing gateways if they fail, and troubleshooting gateways.

Types of Elasticsearch Clients for Gateways to Use

After deciding where to place your gateways, decide which type of Elasticsearch client for the gateways to use to connect to the Elasticsearch cluster. There are two types of Elasticsearch client for this purpose: transport clients and node clients.

NOTE: All of the gateways that you use to index data that is in a source MapR cluster must use the same type of client. For example, if you are using four gateways, you cannot use two with transport ciients and two with node clients. All four must use either transport clients or node clients.

Transport clients

When gateways use transport clients to connect to an Elasticsearch cluster, the gateways connect to one or more transport nodes in that cluster. The gateways pass replicated updates from the source MapR cluster to the transport nodes. These nodes are responsible for distributing the updates to the correct nodes in the Elasticsearch cluster.

Node clients

When gateways use node clients to connect to an Elasticsearch cluster, the gateways themselves distribute updates to the correct nodes in the Elasticsearch cluster.

For more information about these clients and for instructions about how to use them, see the Elasticsearch documentation at https://www.elastic.co/guide/index.html.