Scribd
Upload
102 views4 pages

Hdfs Commands

The document describes various commands used to perform operations on HDFS like listing contents of directories, checking disk usage, copying/moving files between local file system and HDFS, setting permissions on files. Some key commands are hadoop fs for interacting with HDFS, -ls to list directory contents, -du to check disk usage, -copyFromLocal/-copyToLocal to copy files between local and HDFS, -chmod to set permissions and -mv to move files/directories.

Uploaded by

uday kiran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
102 views4 pages

Hdfs Commands

The document describes various commands used to perform operations on HDFS like listing contents of directories, checking disk usage, copying/moving files between local file system and HDFS, setting permissions on files. Some key commands are hadoop fs for interacting with HDFS, -ls to list directory contents, -du to check disk usage, -copyFromLocal/-copyToLocal to copy files between local and HDFS, -chmod to set permissions and -mv to move files/directories.

Uploaded by

uday kiran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

Hdfs commands

1. Print the Hadoop version


hadoop version

2. List the contents of the root directory in HDFS


hadoop fs -ls /

3. Report the amount of space used and available on currently mounted


filesystem
hadoop fs -df hdfs:/

4. Count the number of directories,files and bytes under the paths that
match the specified file pattern
hadoop fs -count hdfs:/

5. Create a new directory in HDFS.


hadoop fs -mkdir /training
6. Add a sample text file from the local directory named “data” to the new
directory you created in HDFS during the previous step.
hadoop fs -put /data/sample.txt /training

7. List the contents of this new directory in HDFS.


hadoop fs -ls /training/

08. Add the entire local directory called “retail” to the /training directory
in HDFS.
hadoop fs -put /data/retail /training/

09. See how much space this directory occupies in HDFS.


hadoop fs -du -s -h /training

10. Delete a file ‘retail’ from the “training” directory.


hadoop fs -rmr /training/retail

11. Delete all files from the “retail” directory using a wildcard.
hadoop fs -rm /retail/*

12.To empty the trash


hadoop fs –expunge
13. Finally, remove the entire test directory and all of its contents in HDFS.
hadoop fs -rm -r /test/demo/one

14. Add the purchases.txt file from the local directory named
“/home/training/” to the hadoop directory you created in HDFS
hadoop fs -copyFromLocal /data/sample.txt /training/demo

15. To view the contents of your text file sample.txt which is present in
your hadoop directory.
hadoop fs -cat /data/sample.txt

16. Add the purchases.txt file from “hadoop” directory which is present in
HDFS directory to the directory “data” which is present in your local
directory
hadoop fs -copyToLocal /training/sample.txt /home/training/data

17. cp is used to copy files between directories present in HDFS


hadoop fs -cp /training/sample.txt /user/training/hadoop

18. ‘-get’ command can be used alternaively to ‘-copyToLocal’ command


hadoop fs -get /training/hadoop/sample.txt /home/training/

19. Display last kilobyte of the file “purchases.txt” to stdout.


hadoop fs -tail /hadoop/purchases.txt

20. Default file permissions are 666 in HDFS Use ‘-chmod’ command to
change permissions of a file
hadoop fs -chmod 777 /hadoop/purchases.txt
21. Move a directory from one location to other
hadoop fs -mv /hadoop /apache_hadoop

22. Copy a directory from one node in the cluster to another Use ‘-distcp’
command to copy,
-overwrite option to overwrite in an existing files
-update command to synchronize both directories
hadoop fs -distcp hdfs://namenodeA/apache_hadoop
hdfs://namenodeB/hadoop
23.List all the hadoop file system shell commands
hadoop fs
24.The hadoop touchz command creates a zero byte file. This is similar to
the touch command in unix. The syntax is shown below:
hadoop fs -touchz /path
25.Hadoop tail command prints the last kilobytes of the file. The -f option
can be used same as in unix.
hadoop fs -tail /path

30. Run a DFS filesystem checking utility


hadoop fsck – /
31. Run a cluster balancing utility
hadoop balancer

scp source_file_name username@destination_host:destination_folder

scp demo.txt [email protected]:APRB2/demo.txt

You might also like