External Applications and Classpath

Describes how to configure the class path for external applications.

MapReduce version 2 applications require the hadoop 2.x or the yarn classpath, and other applications that can run on YARN require the yarn classpath.

The method to specify the classpath differs based on how the job or application is submitted:

Method used to Submit the Job Method to Specify Classpath
The external application uses the hadoop jar or the yarn jar command. YARN applications (MapReduce or custom applications) that are submitted using the yarn jar command will automatically use the yarn classpath.

If the external application has a service that submits the job, you can set the CLASSPATH environment variable to point to a different classpath prior to starting the service. In this case, the hadoop classpath that you set in the CLASSPATH environment variable takes priority over the hadoop classpath for the hadoop jar command.
The external application does not use the hadoop jar or the yarn jar command.

Set the classpath using one of the following options:

  • If the external application has a service that submits the job or application, you can set the CLASSPATH environment variable to point to the hadoop or yarn classpath prior to starting the service.

  • Set the classpath within the application.

Use one of the following methods to get the classpath:

  • hadoop2 classpath or hadoop -yarn classpath: Gets the classpath for MRv2 applications.

  • yarn classpath: Gets the classpath for YARN applications (MRv2 or other applications that can run on YARN).
IMPORTANT
When you launch a spring boot application, ensure that you prefix the classpath with /opt/mapr/conf:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop. Alternatively, copy the core-site.xml file to the /src/main/resources/ folder.