MapR 5.1 is at End of Life (EOL) and no longer supported. Please see the latest documentation. This documentation is not being updated.

Home
5.1 Ecosystem
This guide contains a section for each open source project that MapR supports. You can learn how to install, configure, use, and integrate each project within the context of a MapR cluster.
Spark
Spark on YARN
Configuring Spark on YARN
Configure MapR Client Node to Run Spark Applications
When Spark runs on YARN, MapR client nodes require the hadoop-yarn-server-web-proxy JAR file to run Spark applications. A MapR client node (a node with the mapr-client package, but without mapr-core packages) is also known as an edge node.

MapR 5.1 Documentation

5.1 Ecosystem
This guide contains a section for each open source project that MapR supports. You can learn how to install, configure, use, and integrate each project within the context of a MapR cluster.
- AsyncHBase
- Cascading
- Drill
- Flume
- HBase
- HCatalog
- Hive
- HttpFS
- Hue
- Impala
- Mahout
- Myriad
- Oozie
- Pig
- OpenStack Sahara (Mitaka)
- Sentry
- Spark
  - Spark Feature Support
    MapR supports most Spark features. However, there a few exceptions.
  - Spark Standalone
  - Spark on YARN
    - Installing Spark on YARN
    - Upgrading Spark on YARN
    - Configuring Spark on YARN
      - Configure MapR Client Node to Run Spark Applications
        When Spark runs on YARN, MapR client nodes require the hadoop-yarn-server-web-proxy JAR file to run Spark applications. A MapR client node (a node with the mapr-client package, but without mapr-core packages) is also known as an edge node.
      - Configure Spark JAR Location
      - Configure Spark with the NodeManager Local Directory Set to MapR-FS
      - Security for Spark on YARN
    - Using Spark on YARN
      This section includes information about using Spark on YARN in a MapR cluster.
  - Integrating Spark
  - Spark 2.0.0 Developer Preview
    Apache Spark is an open-source processing engine that you can use to process Hadoop data. Although MapR does not yet ship a Spark 2.0.0 package, you can install and use Spark 2.0.0 on a non-secure MapR 5.1 cluster or on a secure MapR 5.1 cluster that uses MapR-SASL authentication.
- Sqoop
- Storm
- Third Party Solutions

Configure MapR Client Node to Run Spark Applications

When Spark runs on YARN, MapR client nodes require the hadoop-yarn-server-web-proxy JAR file to run Spark applications. A MapR client node (a node with the mapr-client package, but without mapr-core packages) is also known as an edge node.

The mapr-client package does not include the JAR file required to run Spark applications. Therefore, copy the/opt/mapr/hadoop/hadoop-2.x.x/share/hadoop/yarn/hadoop-yarn-server-web-proxy-<version>.jar from a MapR cluster node to the same location on the MapR client node from which you want to run the Spark application.