Scribd
Upload
79 views

Hdfs Lab Work

This document provides instructions for completing lab exercises on HDFS. The exercises include: creating a file in HDFS containing student data; copying files between local file system and HDFS; moving files within HDFS; deleting files and directories from HDFS; and running a MapReduce job to analyze sample input data stored in HDFS.

Uploaded by

Roni Vincent
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
79 views

Hdfs Lab Work

This document provides instructions for completing lab exercises on HDFS. The exercises include: creating a file in HDFS containing student data; copying files between local file system and HDFS; moving files within HDFS; deleting files and directories from HDFS; and running a MapReduce job to analyze sample input data stored in HDFS.

Uploaded by

Roni Vincent
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
You are on page 1/ 2

Introduction to HDFS

Lab Work

1 Lab Exercises:

All these exercises are run from the working directory.

1.1 Exercise 1:
a. Create a file called students.txt with the following data:
Emp #,Name,Score

1000,John Doe,12
1001,Jane Doe,14
1002,Jill Doe,17
1003,John Smith,16
1004,Jill Smith,18
1005,Tim Ford,17
1006,Tom Ford,15
1007,Bill Smith,14
1008,John Ford,16
1009,Chuck Jones,14
1010,Chad Smith,17

b. Type the below command in the working directory to view the HDFS path:
hadoop fs –ls / (HDFS root directory)
hadoop fs –ls /user/xxxx/ (current user directory)

c. Create a directory in HDFS


hadoop fs -mkdir /user/xxxx/lab/xxxx

d. Copy the file to HDFS using the following command


hadoop fs -put /source_file_path /user/xxxx/lab/xxxx/
(OR)
hadoop fs -copyFromLocal /source_file_path /user/xxxx/lab/xxxx/

e. View the contents of the HDFS file


hadoop fs -cat /user/xxxx/lab/xxxx/students.txt

or
hadoop fs -cat lab/xxxx/students.txt

Page No. 1
1.2 Exercise 2:

a. To get a file from HDFS to local directory

hadoop fs -get /user/xxxx/lab/xxxx/students.txt /local_destination_directory


(OR)
hadoop fs -copyToLocal /user/xxxx/lab/xxxx/students.txt /local_destination_directory

b. To copy file from one path to another in HDFS


hadoop fs -cp /source_hdfs_path /destination_hdfs_path

c. To delete the contents of a directory in HDFS(for recursive delete)

hadoop fs -rmr /user/xxxx/lab/xxxx/students.txt

here instead of -rmr, we can use -rmdir to remove the entire directory

or

hadoop fs -rm /user/xxxx/lab/xxxx/students.txt

- rm is used to remove a particular file

Running MapReduce job:

-Copy the input file to HDFS using the following command


hadoop fs -put /home/trainee/mapreduce/wordcount_input.txt /user/xxxx/lab/xxxx/
-Type the below command in the working directory to view the HDFS path:
hadoop fs -ls lab/xxxx/
-View the contents of the HDFS file
hadoop fs -cat lab/xxxx/wordcount_input.txt

-Execute command.
hadoop jar wordcount.jar WordCount E0_lab/<emp_id>/wordcount_input.txt lab/xxxx/wordcount_output
or
yarn jar wordcount.jar WordCount lab/xxxx/wordcount_input.txt lab/xxxx/wordcount_output
-View the contents of the wordcount output file
hadoop fs -cat lab/xxxx/wordcount_output/part-*

Page No. 2

You might also like