Remote File Synchronization in Linux

In this article we are going to understand about the rsync command in Linux that deals with remote file synchronization.This article will provide practical examples on how to use rsync with the most commonly used options.

Remote Synchronization or rsync is a powerful command line utility tool that takes care of synchronizing the files and directories between two machines remotely as well locally.While synchronizing the files one machine will act as the host and the next one as the destination.

Linux system administrators are the most that make use of this command. You can use rsync command, to copy and sync, files and directories locally as well as remotely, maintain backup and mirroring data.

Features of Rsync command

The rsync is the most commonly used command when it comes to copying, synching and taking data backups. Using this we can locate files faster based on the file size changes and last modified time. It makes use of a quick-check algorithm for the same.

The file transfer using rsync is said to be faster as it uses the delta transfer algorithm. In this it takes care of comparing the host and destination file and only the difference seen is sent across. For the very first time the entire file will be transferred and for the next transfers only the difference is send, this makes the file transfer light and faster.

Rsync takes care of saving you your bandwidth by zip-compressing the files and transferring them across.

Syntax

Following is the syntax for rsync command:

Rsync - Local to Local

rsync [OPTION]... SRC [SRC]... DEST

Rsync - Local to Remote

rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST

Rsync - Remote to Local

rsync [OPTION]... [USER@]HOST:SRC [DEST]

As we discussed the rsync command can be used to copy a file/directory from one machine to another. So the file transfer can happen between local machines, local to remote machines and from remote to local machines.

The rsync command will differ when the remote machine is involved , let us check the syntax for all three cases.

Parameters

Following are the parameters of this command ?

• OPTION ? There are many options that you can use with the rsync command.Let us discuss the same in detail in the section below.

• SRC ? The source file or directory

• DEST ? The destination file or directory.

• USER ? The remote machine username.

• HOST ? The remote machine hostname or IP address.

Options in Rsync command

Here is the list of the most basic options used with the rsync command.

• -a ? This option takes care of copying the files recursively as well making sure that the file permission for (user, owner, others), symbolic link permissions are maintained.

• -r ? This option also copies files recursively but file permissions , symbolic link permissions are not taken care of.

• -z ? Used to compress data while transferring.

• -p ? This helps to preserve the file permissions.

• -q ? This takes care of preserving the group permissions.

• -t ? This makes sure the modification file is maintained.

• -P ? helps to see the progress of the file being transferred.

• -v ? will also show progress but a little detail of each file.

Example

File transfer from one folder to another on my local machine. In the example below we have a very big file in my current folder called test_backup.gz . I want to transfer the same in another folder called test. The command to transfer the file to the folder test/ is as follows ?

rsync -v test_backup.gz test/

The options -v or -p will take care of showing the progress of file transfer to the user.The same file name test_backup.gz will get copied inside test/ directory. You can also specify a file name of your own incase you don't want to retain the original filename being copied.

Output

The output of the command is as shown below ?

ubuntu@ip-xx-xx-xx-xx:~$ mkdir test
ubuntu@ip-xx-xx-xx-xx:~$ rsync -v test_backup.gz test/
sending incremental file list
test_backup.gz

sent 828,331,571 bytes  received 35 bytes  66,266,528.48 bytes/sec
total size is 828,129,280  speedup is 1.00
ubuntu@ip-xx-xx-xx-xx:~$ cd test
ubuntu@ip-xx-xx-xx-xx:~/test$ ls
test_backup.gz
ubuntu@ip-xx-xx-xx-xx:~/test$

You can skip the -v option if you don't want to see the progress of rsync command. The rsync command takes care of creating the file or directory if it does not exists.

Example

Let us first see the command to transfer a directory from one location to another on my machine locally.

rsync -av test/  demo/

The -a option will take off recursively copying all the files from test/ folder to demo/ folder. Make sure you have read permission on test/ folder and write permission to write in it.

Please note the rsync command takes care of creating the destination directory if it does not exist.

Output

The output of the command is as shown below ?

ubuntu@ip-xx-xx-xx-xx:~$ rsync -av test/ demo/
sending incremental file list
./
abc.xhtml
hello.xhtml
login.xhtml
login-test.xhtml
success.xhtml
activation.xhtml
password.xhtml
reset.xhtml
return.xhtml
setter.xhtml

sent 73,972 bytes  received 247 bytes  148,438.00 bytes/sec
total size is 73,012  speedup is 0.98
ubuntu@ip-xx-xx-xx-xx:~$ 

Example

On my machine I have a backup file which I want to copy to another remote machine. To do that we need the remote machine username, host or ip address. Right now I have the username and ip address. Let us try copying the backup file to the remote machine.

The command to do so will be as follows:

rsync -v  my_backupfile.zip  [email protected]:/tmp/

The file is my_backupfile.zip that we want to send to the machine with the machine ip address : 12.343.11.34. The username of that machine is ubuntu. We would like to copy the file my_backupfile.zip inside tmp/ folder. If the server is password protected you will have to enter the password when the file transfer takes place.

Output

The output of the command is as shown below:

ubuntu@ip-xx-xx-xx-xx:~$ rsync -v  my_backupfile.zip  [email protected]:/tmp/
my_backupfile.zip

sent 828,331,547 bytes  received 35 bytes  61,357,894.96 bytes/sec
total size is 828,129,280  speedup is 1.00
ubuntu@ip-10-97-24-72:~$

Let us now try the directory and copy the same to the tmp/ folder. The command is ?

rsync -av  pages  [email protected]:/tmp/

In the above command you can copy pages directory to tmp/ folder. We have used -a option so that all the subdirectories also get copied.

ubuntu@ip-xx-xx-xx-xx:~$ rsync -av  pages  [email protected]:/tmp/
sending incremental file list
test/
test/abc/
test/abc/one.xml
test/abc/cases/
test/abc/cases/my.properties.xhtml
test/abc/incl/
test/abc/incl/tpl/
test/abc/incl/tpl/login.xhtml
test/src/
test/src/password.xhtml
test/src/otp.xhtml
test/src/login.xhtml
test/src/reset-password.xhtml

sent 99,589 bytes  received 371 bytes  199,920.00 bytes/sec
total size is 98,078  speedup is 0.98
ubuntu@ip-10-97-24-72:~$

Example

In this case we have some files that are on a remote server on our local machine. The command for same is ?

rsync -av [email protected]:test.txt  tmp

The file test.txt is available on remote machine 12.343.11.34. We want that file inside the tmp folder on our local machine.If the server is password protected you will have to enter the password when the file transfer takes place.

Output

The output of the command is :

ubuntu@ip-xx-xx-xx-xx:~$ rsync -av [email protected]:test.txt tmp
receiving incremental file list
test.txt

sent 43 bytes  received 1,988,966 bytes  1,326,006.00 bytes/sec
total size is 1,988,385  speedup is 1.00
ubuntu@ip-xx-xx-xx-xx:~$

Remote Server to Our Local Machine

Let us also try synching the directory from the remote server to our local machine. The command for it is

Example

rsync -av  [email protected]:demo tmp

Here the directory demo exists on the remote machine [email protected]. We want to copy it in the tmp directory on our local machine.

Output

The output of the command is as follows ?

ubuntu@ip-xx-xx-xx-xx:~$ rsync -av  [email protected]:demo tmp
receiving incremental file list
demo/
demo/abc.txt

sent 47 bytes  received 200 bytes  164.67 bytes/sec
total size is 53  speedup is 0.21
ubuntu@ip-xx-xx-xx-xx:~$