Bigdata and HadoopGuide to Import, Export, Run A MapReduce Program

Guide to Import, Export, Run A MapReduce Program

Guide to Import, Export, Run A MapReduce ProgramA MapReduce program is written in Java. And mostly Eclipse IDE is used for programming by the developers. In our last guide we saw how to run MapReduce program in detail, in this guide, we will see how to export a MapReduce program into a jar file and run it.

We need to create a Java project for MapReduce, here, I am providing you a wordcount java project. Firstly we will import this project in Eclipse IDE.

Step 1
Download the wordcount project using below link. And extract the zip file.
Wordcount source code

Open your Eclipse IDE. Select File and click on Import
1
Step 2
Select General and then Existing Projects into Workspace. Click on Next.
2
Step 3
Select root directory option and click on Browse.
3
Step 4
Now browse to the folder you extracted earlier, select it and click OK.
4
Step 5
You will see the project folder got inserted on Projects: option, click on Finish.
5
You can find the project in your Eclipse IDE now.

But you will get errors, because the jars used for this project would be missing.
6
Step 6
To resolve the above issue, right click on the project –> Buid Path –> Configure Build Path…
7
Step 7
Select Libraries option, and remove only the two jars present in it by selecting them and clicking on Remove.
8
Step 8
Now click on Add External Jars.
9
Step 9
Go to the directory where you extracted the project, then inside it go to lib directory and select both the jar files present in it.
10
Step 10
Go to Order and Export option,  select both the jars and JRE and click OK .
11
Now all the errors from the code will be eliminated.
12
Step 11
Now we will export the project to run it. Select File option in eclipse ide and click on Export.
13
Step 12
In Java option, select Jar file and click on Next.
14
Step 13
Select the Wordcount project and give the path and name for the jar file, I am keeping it wordcount.jar, Click on Next.
15
Step 14
Click on Next.
16
Step 15
Click on Browse.
17
Step 16
Click on WordCount and then click on OK.
18
Step 17
You can see WordCount as Main Class now, click on Finish.
19
Step 18
In case you get any warning as below, just click OK.
20
Note: In case you are using Eclipse IDE in Windows, transfer this wordcount.jar file to your VM using tools like Filezilla or WinSCP.

Step 19
We have our input file onto HDFS for wordcount program. If not use below command.
Command: hadoop dfs -put input hdfs:/
21
Step 20
Check whether your cluster is up and working or not.
Command: jps
Now run the wordcount.jar file using below command. My jar file is in Downloads directory.
Note: Since we selected the main class while exporting wordcount.jar , so no need to mention the main class in the command.
Command: hadoop jar Downloads/wordount.jar /input /new_output
22
Step 21
After the program runs successfully, go to HDFS and check the part file inside new_output directory
Below is the output of wordcount program.
23

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Exclusive content

- Advertisement -

Latest article

21,501FansLike
4,106FollowersFollow
106,000SubscribersSubscribe

More article

- Advertisement -