Configuring the Classpath for External Applications

When an external applications submits a MapReduce v1 job, it requires the hadoop 1 classpath. Likewise, MapReduce v2 applications require the hadoop 2.x or the yarn classpath, and other applications that can run on YARN require the yarn classpath.

The method to specify the classpath differs based on how the job or application is submitted:

Method used to Submit the Job Method to Specify Classpath
The external application uses the hadoop jar or the yarn jar command.

MapReduce programs submitted using the hadoop jar command use the classpath associated with the mode set by the MAPR_MAPREDUCE_MODE environment variable. If the MAPR_MAPREDUCE_MODE is not set, the classpath is set based on the default_mode specified in the hadoop_version file on the current node.

YARN applications (MapReduce or custom applications) that are submitted using the yarn jar command will automatically use the yarn classpath.

If the external application has a service that submits the job, you can set the CLASSPATH environment variable to point to a different classpath prior to starting the service. In this case, the hadoop classpath that you set in the CLASSPATH environment variable takes priority over the hadoop classpath for the hadoop jar command.

The external application does not use the hadoop jar or the yarn jar command.

Set the classpath using one of the following options:

  • If the external application has a service that submits the job or application, you can set the CLASSPATH environment variable to point to the hadoop or yarn classpath prior to starting the service.

  • Set the classpath within the application.

Use one of the following methods to get the classpath:

  • hadoop classpath: Gets the classpath based on default_mode set in the hadoop_verison file on the current node.

  • hadoop1 classpath or hadoop -classic classpath: Gets the classpath for MapReduce v1 jobs.

  • hadoop2 classpath or hadoop -yarn classpath: Gets the classpath for MapReduce v2 applications.

  • yarn classpath: Gets the classpath for YARN applications (MapReduce v2 or other applications that can run on YARN).