Supported Commands for Hadoop

HPE Data Fabric supports the following hadoop commands:

Command

Description

archive -archiveName NAME <src>* <dest>

Creates a Hadoop archive, a file that contains other files. A Hadoop archive always has a .har extension.

CLASSNAME

The hadoop script can be used to invoke any class.

hadoop CLASSNAME runs the class named CLASSNAME.

classpath

Prints the class path needed to access the Hadoop JAR and the required libraries.

conf

The hadoop conf command prints the configuration information for the current node.

daemonlog

The hadoop daemonlog command may be used to get or set the log level of Hadoop daemons.

distcp <source> <destination>

The hadoop distcp command is a tool for large inter- and intra-cluster copying. It uses MapReduce to effect its distribution, error handling and recovery, and reporting. It expands a list of files and directories into input to map tasks, each of which will copy a partition of the files specified in the source list.

fs

The hadoop fs command runs a generic filesystem user client that interacts with the Data Fabric file system.

jar <jar>

The hadoop jar command runs a JAR file. Users can bundle their MapReduce code in a JAR file and execute it using this command.

mfs

The hadoop mfs command performs operations on directories in the cluster. The main purposes of hadoop mfs are to display directory information and contents, to create symbolic links, and to set compression and chunk size on a directory.

version

The hadoop version command prints the Hadoop software version.

For example, if you run the hadoop job command, you see this message:

# hadoop job
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead, use the mapred command for it.

The syntax for the mapred command is:

mapred [--config confdir] [--loglevel loglevel] COMMAND

Commands used with mapred include:

Command Description
historyserver Runs job history servers as a standalone daemon
hsadmin The job history server admin interface
job Manipulates MapReduce applications
pipes Runs a pipes job
queue Gets information regarding JobQueues