Hadoop - copyFromLocal Command
Last Updated :
27 Dec, 2021
Hadoop copyFromLocal command is used to copy the file from your local file system to the HDFS(Hadoop Distributed File System). copyFromLocal command has an optional switch –f which is used to replace the already existing file in the system, means it can be used to update that file. -f switch is similar to first delete a file and then copying it. If the file is already present in the folder then copy it into the same folder will automatically throw an error.
Syntax to copy a file from your local file system to HDFS is given below:
hdfs dfs -copyFromLocal /path 1 /path 2 .... /path n /destination
The copyFromLocal local command is similar to the -put command used in HDFS. we can also use hadoop fs as a synonym for hdfs dfs. The command can take multiple arguments where all the paths provided are of the source from where we want to copy the file except the last one which is the destination, where the file is copied. Make sure that the destination should be a directory.
Our objective is to copy the file from our local file system to HDFS. In my case, I want to copy the file name Salaries.csv which is present at /home/dikshant/Documents/hadoop_file directory.
Steps to execute copyFromLocal Command
Let's see the current view of my Root directory in HDFS.
Step 1: Make a directory in HDFS where you want to copy this file with the below command.
hdfs dfs -mkdir /Hadoop_File
Step 2: Use copyFromLocal command as shown below to copy it to HDFS /Hadoop_File directory.
hdfs dfs -copyFromLocal /home/dikshant/Documents/hadoop_file/Salaries.csv /Hadoop_File
Step 3: Check whether the file is copied successfully or not by moving to its directory location with below command.
hdfs dfs -ls /Hadoop_File
Overwriting or Updating the File In HDFS with -f switch
From below Image, you can observe that copyFromLocal command itself does not copy the same name file at the same location. it says that the file already exists.
To update the content of the file or to Overwrite it, you should use -f switch as shown below.
hdfs dfs -copyFromLocal -f /home/dikshant/Documents/hadoop_file/Salaries.csv /Hadoop_File
Now you can easily observe that using copyFromLocal with -f switch does not produce any error or it will easily update or modify your file in HDFS.
Similar Reads
rcp Command in Linux with examples When working in a Linux environment, there often comes a time when you need to transfer files from one computer to another. While more secure options like scp or rsync exist, the rcp (Remote Copy Protocol) command offers a simple and efficient way to copy files between systems, especially for beginn
5 min read
How to Use the Xcopy Command in Windows? The xcopy command in Windows is a powerful tool for copying files and directories. It extends the functionality of the standard copy command by allowing the copy of entire directories along with their subdirectories and files all at once. This makes xcopy highly useful for tasks such as backing up f
4 min read
PHP copy( ) Function The copy() function in PHP is an inbuilt function which is used to make a copy of a specified file. It makes a copy of the source file to the destination file and if the destination file already exists, it gets overwritten. The copy() function returns true on success and false on failure. Syntax: bo
2 min read
How to Clone all Remote Branches in Git? Cloning a repository is a common task when working with Git, but what if you need all the branches from a remote repository? By default, Git clones only the default branch (usually 'master' or 'main'). However, you may need access to all remote branches in many scenarios. This article will guide you
2 min read
How to Copy Files and Directories in Linux | cp Command The cp (copy) command is your go-to tool in Linux for duplicating files and folders quickly. Whether youâre backing up data, organizing files, or sharing content, cp lets you copy items between locations while keeping the original intact. The cp command requires at least two filenames in its argumen
8 min read
Copy Files Recursively using AWS S3 cp Command Amazon S3(Simple Storage Service) is AWS's powerful and scalable solution for storing data in the cloud. It offers a reliable and cost-effective way to store and retrieve data of all kinds. Among its many features, the aws s3 cp command is particularly useful for transferring files. Whether you need
8 min read