Tuesday, August 11, 2009

Hadoop Example: MyMaxTemperatureWithCombiner


  1. Set up Hadoop in Eclipse

    Hadoop on Windows with Eclipse

  2. Hadoop Example

    MyMaxTemperatureWithCombiner.java,
    MaxTemperatureMapper.java,
    MaxTemperatureReducer.java


    How to run the example codes:


    1. You need to set up Hadoop as shown above (Set up Hadoop in Eclipse)

    2. make a directory named "tempIn" at your hadoop:

      bin/hadoop fs -mkdir tempIn

    3. copy input files 1901 and
      1902 to your HDF:

      bin/hadoop fs -cp 1901 tempIn/

      bin/hadoop fs -cp 1902 tempIn/

    4. In the eclipse IDE, imports three java files above under package named "edu.calstatela.hipic.hadoop.util"

    5. Start Hadoop cluster as shown above (Set up Hadoop in Eclipse)

    6. In the eclipse IDE, Right click on
      MyMaxTemperatureWithCombiner.java, choose "Run as" > "Run Hadoop
      Application"

    7. You will see the map/reduce results at the HDF folder "tempOut"
      in DFS Location of eclipse IDE

2 comments:

Followers

Profile