Configuring a MapR Gateway Master-Slave Topology

Configuring gateways involves installing the mapr-gateway package on nodes in a MapR destination cluster and then configuring the MapR source cluster to communicate with the destination cluster. The Mapr source cluster is configured by specifying the destination cluster's CLDB node(s) and gateway nodes.

About this task

Although it is possible to use a single gateway, the recommended practice is to configure at least two (2), so that replication can continue if a gateway fails. MapR source clusters distribute requests among the gateways in a round-robin fashion.

NOTE: If you are configuring an independent node for replicating MapR-DB binary tables to Elasticsearch indexes, see Configuring a MapR Gateway on an Independent Node for Elasticsearch Indexing. These instructions assume that the gateway node is on a MapR cluster.

Procedure

  1. On the destination cluster, install the mapr-gateway package on each node where you want to run a gateway on.See Installing MapR Software.
    NOTE: On the gateway nodes in the destination cluster, when you run (or re-run) the configure.sh script (in addition to your regular parameters) be sure to specify the -N parameter with the name of the cluster that the gateway belongs to. See configure.sh for more information about configure.sh usage, options, and examples.
  2. If you want to change the port that a gateway is using (by default, gateways use port 7660), follow these steps:
    1. On the node where the gateway is running, edit the /opt/mapr/conf/gateway.conf file, uncommenting the line #gateway.port=7660 and changing the port number.
    2. After saving the file, restart the gateway by running this command: maprcli node services -name gateway -action restart
  3. On every node in the source cluster, specify the destination cluster's name and gateway nodes by using one of the following methods.:
    • Using the maprcli cluster gateway set command.
    • Adding a DNS record to your DNS server's zone file for your domain.
    Using the maprcli cluster gateway set command
    The syntax of this command is: 
    maprcli cluster gateway set -dstcluster <cluster name> -gateways "<space-delimited list of gateways>"

    There is another maprcli command that you can use to generate a list of the gateways in a MapR cluster: maprcli cluster gateway list. You can then copy this list and paste it into the maprcli cluster gateway set command. On the cluster where the gateways are located, run the command maprcli cluster gateway local -format text. If you want to run the command from a different cluster and point to the cluster where the gateways are located, use the -cluster parameter to provide the name of that cluster.

    For example, suppose that you are configuring table replication from the cluster sanfrancisco to the cluster newyork and want to use two gateways. The nodes on which these gateways are located are named gw1 and gw2.

    The command that you run will look like this:

    maprcli cluster gateway set -dstcluster newyork -gateways "gw1.bigcompany.com gw2.bigcompany.com"
    NOTE: If you are indexing MapR-DB data to Elasticsearch, the -dstcluster parameter is set to the MapR source cluster.
    Adding a DNS record to your DNS server's zone file for your domain

    In your DNS server’s zone file for your domain, add a record for the cluster where gateways are located, listing the nodes to use as gateways. You can use the MapR Control System (MCS) to create a record that you can copy into a DNS configuration file, run a maprcli command to generate the record, or create a record manually.

    To create a record with the MCS, follow these steps:

    1. Log into MCS on the cluster where the gateways are located.
    2. In the Navigation pane, select MapR-DB Tables.
    3. In the MapR-DB Tables section, click the button Generate Gateway DNS. A window opens with the generated DNS entry.
    4. Copy and paste the record into your zone file.

    To generate a record by using the maprcli, follow these steps:

    1. On the cluster where the gateways are located, run the command maprcli cluster gateway local -format dns. If you want to run the command from a different cluster and point to the cluster that hosts the gateways, use the -cluster parameter to provide the name of the latter cluster.
    2. Copy and paste the output of this command into your zone file.
    If you want to create a record manually, use this format:
    gateway.<clustername> IN TXT “<space-delimited list of hostnames>”
    You can also specify IP addresses, though using hostnames is recommended so that it is easier to locate gateways if their IP addresses change. Combinations of hostnames and IP addresses are also supported. The default port is 7660. If a gateway is using a different port, append a colon to the address and then specify the port number. Here is an example entry:
    gateway.newyork.bigcompany.com gw1ny.bigcompany.com gw2ny.bigcompany.com
    Multi-homing is also supported. Simply separate the entries for a single node with semicolons, as in this example that uses IP addresses:
    gateway.newyork.bigcompany.com 10.10.34.20 10.10.34.22 10.10.34.24;173.194.79.121