Scribd
Upload
132 views

HDFC Commands

The document provides examples of Hadoop HDFS commands for listing directories, formatting listings, making directories, copying files between local and HDFS, counting directories, copying files within HDFS, displaying file sizes, copying files from HDFS to local, viewing file contents, moving and deleting files and directories, displaying the last kilobytes of a file, and changing file permissions. It also describes the usage of the hadoop fs command and how to append contents from a local file to a file in HDFS.

Uploaded by

Nagraj Goud
Copyright
© © All Rights Reserved
Available Formats
Download as ODT, PDF, TXT or read online on Scribd
132 views

HDFC Commands

The document provides examples of Hadoop HDFS commands for listing directories, formatting listings, making directories, copying files between local and HDFS, counting directories, copying files within HDFS, displaying file sizes, copying files from HDFS to local, viewing file contents, moving and deleting files and directories, displaying the last kilobytes of a file, and changing file permissions. It also describes the usage of the hadoop fs command and how to append contents from a local file to a file in HDFS.

Uploaded by

Nagraj Goud
Copyright
© © All Rights Reserved
Available Formats
Download as ODT, PDF, TXT or read online on Scribd
You are on page 1/ 4

HADOOP HDFS COMMANDS:

1. Listing the all directory?


hduser@ubuntu:~$ hadoop fs -ls /
Found 4 items
drwxr-xr-x - hduser supergroup 0 2017-02-25 03:56 /city
drwxr-xr-x - hduser supergroup 0 2017-02-25 03:28 /student
drwxr-xr-x - hduser supergroup 0 2017-02-25 03:32 /studentcity
drwxrwx--- - hduser supergroup 0 2017-02-25 03:06 /tmp
hduser@ubuntu:~$

2.formating listing with hidden files:


hduser@ubuntu:~$ hadoop fs -ls al /
ls: `al': No such file or directory
Found 4 items
drwxr-xr-x - hduser supergroup 0 2017-02-25 03:56 /city
drwxr-xr-x - hduser supergroup 0 2017-02-25 03:28 /student
drwxr-xr-x - hduser supergroup 0 2017-02-25 03:32 /studentcity
drwxrwx--- - hduser supergroup 0 2017-02-25 03:06 /tmp

4. make a directory ?

hduser@ubuntu:~$ hadoop fs -mkdir /niit1

5. copy file from local ?


Ans:--use copyFromLocal or copyTo
hduser@ubuntu:~$ hadoop fs -copyFromLocal file1 /niit1/
hduser@ubuntu:~$ hadoop fs -ls /niit1/
Found 1 items
-rw-r--r-- 1 hduser supergroup 0 2017-02-26 21:59 /niit1/file1

5. count the number of directories?


Ans:--- count -h,count -q (q-count quata,-h:count the)
hduser@ubuntu:~$ hadoop fs -count -h /
17 14 358.1 K /
hduser@ubuntu:~$ hadoop fs -count -q /
9223372036854775807 9223372036854775776 none inf 17 14
366666 /

6.copy the file in the hadoop ?


Ans:----
hduser@ubuntu:~$ hadoop fs -cp /niit1/file1 /niit
17/02/26 22:07:04 WARN hdfs.DFSClient: DFSInputStream has been closed already
hduser@ubuntu:~$

7.display the size of files or directories ?


hduser@ubuntu:~$ hadoop fs -du /
53 /city
0 /niit
0 /niit1
247 /student
0 /studentcity
366366 /tmp

8. display the summary of files lengths ?

Ans:--hduser@ubuntu:~$ hadoop fs -du -s /


366666 /

9. copy file to the local files system ?


hduser@ubuntu:~$ hadoop fs -get /city/part-r-00000
17/02/26 22:16:20 WARN hdfs.DFSClient: DFSInputStream has been closed already
hduser@ubuntu:~$ ls
city.jar Documents examples.desktop hadoop-2.7.1 installation steps.txt Music Pictures
student1 Templates
Desktop Downloads file1 hadoop-2.7.1.tar.gz jar_files part-r-00000 Public
student2 Videos
hduser@ubuntu:~$

10. to see the content of file


hduser@ubuntu:~$ hadoop fs -cat /city/part-r-00000
Bangalore
Chennai
Delhi
Hyderabad
Jaipur
Mumbai
Pune
hduser@ubuntu:~$

11.move the file ?


Ans:---
hduser@ubuntu:~$ hadoop fs -mv /city/part-r-00000 /niit1/
hduser@ubuntu:~$

12.move the directory ?


Ans:---hduser@ubuntu:~$ hadoop fs -mv /niit1/ /niit/
mv: `/niit': File exists

13.Delete the file ?


Ans:-----
hduser@ubuntu:~$ hadoop fs -rm /niit1/file1
17/02/26 22:23:21 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0
minutes, Emptier interval = 0 minutes.
Deleted /niit1/file1

14.delete the directory ?


Ans:--------if directory is empty...
hduser@ubuntu:~$ hadoop fs -rmdir /niit1
rmdir: `/niit1': Directory is not empty

14.Display the last kilobytes of file ?


Ans:----
hduser@ubuntu:~$ hadoop fs -tail /student/student1
201,Arun,85,Delhi
202,Brajesh,98,Mumbai
203,Chetan,76,Pune
204,Priti,82,Bangalore
205,Ravi,70,Pune
206,Kiran,74,Mumbai
hduser@ubuntu:~$

hadoop fs -tail -f will give appended data as file grow ….

15.hadoop commands usage ? Provode the information of each commands.


Ans:---

hduser@ubuntu:~$ hadoop fs -usage


Usage: hadoop fs [generic options]
[-appendToFile <localsrc> ... <dst>]
[-cat [-ignoreCrc] <src> ...]
[-checksum <src> ...]
[-chgrp [-R] GROUP PATH...]
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-copyFromLocal [-f] [-p] [-l] <localsrc> ... <dst>]
[-copyToLocal [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
[-count [-q] [-h] <path> ...]
[-cp [-f] [-p | -p[topax]] <src> ... <dst>]
[-createSnapshot <snapshotDir> [<snapshotName>]]
[-deleteSnapshot <snapshotDir> <snapshotName>]
[-df [-h] [<path> …]]

16.To change the permission of the file or directory?


Ans:----
hduser@ubuntu:~$ hadoop fs -chmod 777 /student/student1

17.append the contents of file from local to the files in hdfs ?


Ans:----
hduser@ubuntu:~$ hadoop fs -appendToFile file2 /niit1/file1
hduser@ubuntu:~$

You might also like