Scribd
Upload
184 views5 pages

Hadoop Exercise Mapreduce

1. The document outlines the steps to execute a word count program using MapReduce on Hadoop, including formatting HDFS, starting relevant Hadoop services, adding input files, and running MapReduce jobs to perform word counting and grep searches. 2. Key steps are formatting HDFS, starting HDFS and YARN, adding files to HDFS, and using Hadoop jar files to run MapReduce jobs for word counting and grep searches on the input files. 3. The results of the MapReduce jobs are viewed through the HDFS browser or by downloading and viewing output files.

Uploaded by

SureshAnand CSE
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
184 views5 pages

Hadoop Exercise Mapreduce

1. The document outlines the steps to execute a word count program using MapReduce on Hadoop, including formatting HDFS, starting relevant Hadoop services, adding input files, and running MapReduce jobs to perform word counting and grep searches. 2. Key steps are formatting HDFS, starting HDFS and YARN, adding files to HDFS, and using Hadoop jar files to run MapReduce jobs for word counting and grep searches on the input files. 3. The results of the MapReduce jobs are viewed through the HDFS browser or by downloading and viewing output files.

Uploaded by

SureshAnand CSE
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

Ex. No.

: 9 WORD COUNT PROGRAM – USING MAP AND


REDUCE TASK

AIM:
To execute wordcount program using map and reduce task

PROCEDURE:
1. Format the path.
2. Start the dfs and check the no. of nodes running.
3. Start the yarn and check the no. of nodes running.
4. Open the browser and check whether the hadoop is installed correctly.
5. Add a file and check whether we can view the file.
6. Implement the grep command for the file added and see the result.
7. Implement the wordcount command for the file added and see the result.
8. After completing the process stop dfs and yarn properly.

COMMANDS:
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ bin/hadoop namenode -format
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ sbin/start-dfs.sh
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ jps
5632 DataNode
5428 NameNode
5979 Jps
5851 SecondaryNameNode

cloud@ubuntu:~/Downloads/hadoop-2.7.0$ sbin/start-yarn.sh
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ jps
5632 DataNode
6209 NodeManager
6050 ResourceManager
5428 NameNode
6522 Jps
5851 SecondaryNameNode
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ sbin/stop-dfs.sh
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ sbin/stop-yarn.sh
Open a browser and type as: http:\\localhost:50070.
Choose DataNode. U can see a single node created in it

U can see the folder created inside utilities tab.


cloud@ubuntu:~/Downloads/hadoop-2.7.0$ bin/hdfs dfs -mkdir /user1
Open a new terminal
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ cd ..
cloud@ubuntu:~/Downloads$ tar zxvf mrsampledata.tar.gz
file2.txt
file5.txt
file1.txt
file4.txt
file3.txt

cloud@ubuntu:~/Downloads/hadoop-2.7.0$ bin/hdfs dfs -put ../file1.txt /user1

Inside the browser. Click the folder name user1 and you can see the file1.txt
file inside the user1 folder

cloud@ubuntu:~/Downloads/hadoop-2.7.0$ bin/hadoop jar


share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar grep /user1/*
/output '(CSE)'
cloud@ubuntu:~/Downloads/hadoop-2.7.0$ bin/hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount
/user1/file1.txt /output1

cloud@ubuntu:~/Downloads/hadoop-2.7.0$ bin/hdfs dfs -cat /output/*


9894 CSE

cloud@ubuntu:~/Downloads/hadoop-2.7.0$ bin/hdfs dfs -cat /output1/*


B.ARCH 9864
B.TECH(BIO) 9964
B.TECH(IT)10000
BE(AME) 9853
BE(CIVIL) 10043
BE(CSE) 9894
BE(ECE) 10048
BE(EEE) 9937
BE(ICE) 9872
BE(MECH) 9873

You can also see the ouput by


1) output
Inside Browser Directory
output --> part-r-00000 --> Click Download

2) output1
Inside Browser Directory
output --> part-r-00000 --> Click Download

RESULT:
Thus the wordcount program using map and reduce task is executed
successfully.

You might also like