Scribd
Upload
22 views

DA Lab Program-1

Uploaded by

Diksha Padiyar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
22 views

DA Lab Program-1

Uploaded by

Diksha Padiyar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

DATA ANALYTICS LABORATORY (21CSL66)

1. FILE MANAGEMENT IN HADOOP


Implement the following file management tasks in Hadoop,
a) Adding Files and directories.
b) Retrieving files.
c) Deleting files.
d) Create a new file in HDFS environment.
e) List files in HDFS.
f) Upload and download files in HDFS as well other properties copy file,
move files and remove file operations in HDFS.

File management in Hadoop is primarily done through the Hadoop Distributed File
System (HDFS), which is designed to store large data sets reliably, and to stream
those data sets at high bandwidth to user applications. To manage files in HDFS,
you use the Hadoop File System shell (hadoop fs) or the HDFS shell (hdfs dfs),
which provides a variety of commands for managing files and directories.

Below are the steps and commands for performing the listed file management tasks
in Hadoop:

a) Adding Files and directories.

• Creating a Directory: To create a directory in HDFS, use the mkdir


command.
hdfs dfs -mkdir /user/hadoop/mydirectory

• Adding Files to HDFS: To add a file from your local file system to HDFS,
use the put command.

hdfs dfs -put localfile.txt /user/hadoop/mydirectory

• Upload a directory to HDFS:

hdfs dfs -put localdirectory /path/in/hdfs


1
b) Retrieving Files.

• To retrieve files from HDFS to your local file system, use the get command:

hdfs dfs -get /path/in/hdfs/localfile.txt /path/in/local

c) Deleting files.

• Deleting a File: To delete a file from HDFS, use the rm command.

hdfs dfs -rm /user/hadoop/mydirectory/localfile.txt

• Deleting a Directory: To delete a directory and its contents, use the rm -r


command.

hdfs dfs -rm -r /user/hadoop/mydirectory

d) Create a new file in HDFS environment.

• Creating a new empty file in HDFS is not directly supported by a command.


However, you can create an empty file locally and then upload it to HDFS.

touch local_empty_file.txt

hdfs dfs -put local_empty_file.txt /path/in/hdfs/emptyfile.txt

e) List files in HDFS.

• To list files and directories in HDFS, use the ls command:

hdfs dfs -ls /path/in/hdfs

2
f) Upload and download files in HDFS as well other properties copy file,
move files and remove file operations in HDFS.

• Upload Files: hdfs dfs -put localfile.txt /path/in/hdfs

• Download Files: hdfs dfs -get /path/in/hdfs/localfile.txt /path/in/local

• Copy Files: hdfs dfs -cp /path/in/hdfs/sourcefile.txt


/path/in/hdfs/destination/

• Move Files: hdfs dfs -mv /path/in/hdfs/sourcefile.txt


/path/in/hdfs/destination/

• Remove Files: hdfs dfs -rm /path/in/hdfs/localfile.txt

You might also like