MapR 5.1 is at End of Life (EOL) and no longer supported. Please see the latest documentation. This documentation is not being updated.

Home
5.1 Ecosystem
This guide contains a section for each open source project that MapR supports. You can learn how to install, configure, use, and integrate each project within the context of a MapR cluster.
Spark
Spark on YARN
Using Spark on YARN
This section includes information about using Spark on YARN in a MapR cluster.
Run Spark from the Spark Shell

MapR 5.1 Documentation

5.1 Ecosystem
This guide contains a section for each open source project that MapR supports. You can learn how to install, configure, use, and integrate each project within the context of a MapR cluster.
- AsyncHBase
- Cascading
- Drill
- Flume
- HBase
- HCatalog
- Hive
- HttpFS
- Hue
- Impala
- Mahout
- Myriad
- Oozie
- Pig
- OpenStack Sahara (Mitaka)
- Sentry
- Spark
  - Spark Feature Support
    MapR supports most Spark features. However, there a few exceptions.
  - Spark Standalone
  - Spark on YARN
    - Installing Spark on YARN
    - Upgrading Spark on YARN
    - Configuring Spark on YARN
    - Using Spark on YARN
      This section includes information about using Spark on YARN in a MapR cluster.
      - Deployment Modes
      - Run Spark from the Spark Shell
  - Integrating Spark
  - Spark 2.0.0 Developer Preview
    Apache Spark is an open-source processing engine that you can use to process Hadoop data. Although MapR does not yet ship a Spark 2.0.0 package, you can install and use Spark 2.0.0 on a non-secure MapR 5.1 cluster or on a secure MapR 5.1 cluster that uses MapR-SASL authentication.
- Sqoop
- Storm
- Third Party Solutions

Run Spark from the Spark Shell

About this task

In yarn-client mode, complete the following steps to run spark from the Spark shell:

Procedure

Navigate to the Spark on YARN installation directory: Substitute your Spark version in the command. For example: 1.3.1
```
cd /opt/mapr/spark/spark-<version>/
```
Issue the following command to run Spark from the Spark shell:
```
MASTER=yarn-client ./bin/spark-shell
```
NOTE: You must use yarn-client mode to run Spark from the Spark shell. The yarn-cluster mode is not supported.