hadoop pipes
The hadoop pipes
command runs a pipes job.
Hadoop Pipes is the C++ interface to Hadoop Reduce. Hadoop Pipes uses sockets to enable tasktrackers to communicate processes running the C++ map or reduce functions.
Syntax
hadoop [GENERIC OPTIONS ] pipes
[-output <path>]
[-jar <jar file>]
[-inputformat <class>]
[-map <class>]
[-partitioner <class>]
[-reduce <class>]
[-writer <class>]
[-program <executable>]
[-reduces <num>]
Parameters
Command Options
The following command parameters are supported for hadoop
pipes
:
Parameter |
Description |
---|---|
|
Specify the output directory. |
|
Specify the jar filename. |
|
InputFormat class. |
|
Specify the Java Map class. |
|
Specify the Java Partitioner. |
|
Specify the Java Reduce class. |
|
Specify the Java RecordWriter. |
|
Specify the URI of the executable. |
|
Specify the number of reduces. |
Generic Options
The following generic options are supported for the hadoop
pipes
command: -conf <configuration
file>
, -D <property=value>
, -fs
<local|filesystem URI>
, -jt
<local|jobtracker:port>
, -files
<file1,file2,file3,...>
, -libjars
<libjar1,libjar2,libjar3,...>
, and -archives
<archive1,archive2,archive3,...>
. For more information
on generic options, see Generic
Options.