mapred
Class Producer
java.lang.Object
org.apache.hadoop.conf.Configured
mapred.Producer
- All Implemented Interfaces:
- org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
public class Producer
- extends org.apache.hadoop.conf.Configured
- implements org.apache.hadoop.util.Tool
This is a MapReduce job used for generating a large data of set and storing in the HDFS.
The generated data in the HDFS will be read later by a counting job. This class was created
and used for experiments.
Usage:
hadoop jar <JARFILE> %mapred.Producer -Dquantity=<quantity> -Dparamfile=<paramfile> <genoutput>
- JARFILE
- The JAR file containing the application.
- quantity
- Number of records to generate.
- paramfile
- Name of parameters file. Path to the parameters file must be a relative path to this class in the JAR file.
See
Parameters
for details on parameters file.
- genoutput
- A HDFS path to the directory containing the generated data.
For details on what data is generated and how records are formatted, see Producer
.
- Author:
- vohuudtr
Method Summary |
static void |
main(java.lang.String[] args)
|
int |
run(java.lang.String[] args)
main function |
Methods inherited from class org.apache.hadoop.conf.Configured |
getConf, setConf |
Methods inherited from class java.lang.Object |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface org.apache.hadoop.conf.Configurable |
getConf, setConf |
Producer
public Producer()
run
public int run(java.lang.String[] args)
throws java.lang.Exception
- main function
- Specified by:
run
in interface org.apache.hadoop.util.Tool
- Throws:
java.lang.Exception
main
public static void main(java.lang.String[] args)
throws java.lang.Exception
- Throws:
java.lang.Exception