How to Sync Local and Remote Directories with Rsync

Introduction

Rsync (Remote Synch) is a command line that can be used to transfer and synchronize files or directories between two locations, either within a single computer or between computers over a network. There are main features in the rsync command, including the following:

1. File and Directory Synchronization, can synchronize files and directories from one location to another by transferring only the differences (deltas) between the source and destination. This is very efficient compared to traditional file transfer methods that transfer entire files even if there are only minor changes.

2. Incremental Transfer uses an algorithm that detects differences between the source and destination files, so only the files' changed parts are transferred. This saves bandwidth and time, especially for large files.

3. Support for Network Transfer can be used to transfer files between computers over a network using the SSH (Secure Shell) protocol for security. This makes it a powerful tool for data backup and replication.

4. One-Way and Two-Way Synchronization performs one-way synchronization, where files from the source are synchronized to the destination. However, with some settings, Rsync can also be used for two-way synchronization.

5. Data Compression compresses data before transferring it, reducing the amount of data that must be sent over the network, increasing transfer efficiency.

6. Time-Based Storage supports the option to store snapshots or time-based backups using hard links, allowing older backups to be stored efficiently.

Rsync Command on Linux

To make the use of `rsync` easier, there are several commonly used `rsync` command parameters.

Command	Description
-a	–archive means that it will recursively copy files by keeping attributes on files such as permissions. This option is a frequently used option, and will generally be combined with other parameters.
-v	-verbose used to display detailed output when the rsync process is carried out. This will allow you to see what the rsync process is doing.
-n	-dry run is useful for displaying a list of files and directories to be moved, deleted, or copied based on a given command, but will not actually perform the action, thus reducing the risk of data loss by ensuring that the command you run does what you expect.
-r	-recursive this will recursively copy directories. With this parameter, the rsync process is not only performed for files in the main folder but also for all subfolders and files in it.
-z	–compress this is used to enable file compression during the transfer process. This parameter can also save bandwidth when the rsync process is running.
-P	It is a combination of two different options: --partial to store files that are not downloaded or copied partially, and --progress to display progress information from file transfers. When used you can benefit from both options at the same time.

The table above is only a small percentage of commonly used parameters. You can do the full parameters of the rsync command using the rsync -h command.

This guide describes using the rsync command in Cloudspeaker Portal Service. On the Cloudspeaker Services page, verify that you already have an Instance on Flexi Deca.

Instance-Deka Flexi

In Instance, there are two Instances available, namely rsync and rsync2. The rsync instance uses the Internal IP address 192.168.1.137, while the rsync2 uses the IP address 192.168.1.23

IP Address Instance rsync & Instance rsync2

Install Rsync

In each instance, press the More button then select Access Console. Enter the username with root and password that was registered when creating the Instance.

Instance-Deka Flexi

Access Terminal

Rsync is usually installed automatically on Linux distros. If it is not yet available on the operating system you are using. If the operating system used is Debian or Ubuntu, you can use this command syntax sudo apt-get install rsync whereas if the operating system used is RedHat or CentOS, it uses command syntax sudo yum install rsync.

Understanding Rsync Syntax

1. If rsync is already available on your operating system, the next step is to enter the home directory by executing the following syntax.

cd~

1. Add two different directories, namely directory 1, namely dir1, and directory 2, namely dir2, by running the following command.

mkdir dir1 dir2

1. To make sure the directory you created already exists on the system run the following command.

ls

Create Directory

1. After creating a directory, the next step is to create several files at once in dir1 by running the following command.

touch dir1/file{1..100}

1. To ensure files created for dir1 are available with run the following command.

ls dir1

If the syntax entered is correct, the result will be as in the image below.

Add File

1. On dir2 because it is still empty and needs to fill in the files from dir1. If you want to perform a complete synchronization that includes all files, subdirectories, and file attributes, resulting in a completely identical copy of the directory you can use the function archive by running this command.

rsync -a source-directory/ destination-directory

1. Meanwhile, if you only copy the structure of directories and files, without paying attention to detailed metadata, you cannot guarantee that the copy will be identical in every non-structural aspect, you can use the add function recursive by running this command.

rsync -r source-directory/ destination-directory

Syntax Rsync

In the command syntax above there is a slash sign (/) referred to as "trailing slash”, which has a significant impact on synchronizing content from the source directory to the destination directory. If you use a slash, it means that all files and subdirectories in the origin directory will be copied directly to the destination directory, without including the origin directory itself. So, the final result in the destination directory will immediately contain the files and subdirectories in the source directory. Meanwhile, if you don't use the slash, it means the rsync command will include the source directory itself along with all the contents that will be copied to the destination directory. To make it easy to understand, we provide examples of its use.

Syntax	Result Dir
rsync -a dir1/ dir2	~/dir2/[files_and_subdirectories_of_dir1]
rsync -a dir1 dir2	~/dir2/dir1/[files_and_subdirectories_of_dir1]

Make sure you double-check your arguments before running the rsync command. Rsync provides a method to do this by passing the -n or --dry-run option. The -v flag, which means “verbose”, is also required to get proper output. You will combine flags a, n, and v by running the following command.

rsync -anv source-directory/ destination-directory

The following are syntax results that use slash marks or without slash marks.

Rsync with Slash

Rsync without Slash

Using Rsync to Sync with a Remote System

To use rsync for synchronization with a remote system, you need to configure SSH access between the local computer and the computer to be accessed, and install rsync on both systems. Once SSH access is verified between both machines. The following are the steps for using rsync to synchronize remotely on the Cloudeka Portal Service.

1. On the Instance menu page in the Cloudeka Portal Service, access the rsync2 instance console by pressing the More button and selecting Access Console. Enter the username with root and password that was registered when creating the Instance.

Access rsync2

Access rsync2

1. Create a new directory on the rsync2 Instance, namely dir3, by running the following command

mkdir dir3

Create dir3

1. After creating a new directory in rsync2 instance namely dir3. The next step is that you can synchronize dir1 on the rsync instance to dir3 on the rsync2 instance. Because currently you are accessing the rsync2 console, you have to log in to the rsync instance using the IP address 192.168.1.137, so you need to access it via SSH by running the command syntax below.

ssh root@192.168.1.137

Access rsync

1. After you have successfully logged into the rsync instance at IP address 192.168.1.137, run the command below. Please pay attention to the command being executed, that you want to transfer the actual directory, so you will omit the slash at the end:

rsync -a ~/dir1 username@remote_host:destination_directory

Synchronization

1. Run the exit command to exit ssh on the rsync Instance and it automatically returns you to the rsync2 Instance.

Exit SSH

1. To ensure that dir1 on the rsync instance has been copied to dir3 on the rsync2 instance, go to dir3 and view the contents of dir3 by running the command syntax below.

cd dir3

ls

If successful, on dir3 there is dir1 which has been successfully copied from the rsync instance.

Dir3 file

1. Go to dir1 and see the contents of the copied file by running the command syntax below.

dir3 file

Conclusions

Rsync can speed up file transfers over network connections and increase resilience in local directory synchronization. The flexibility offered by Rsync makes it an ideal choice for a variety of operations involving files, both small and large.

By mastering Rsync, you can design complex backup operations and have granular control over the transfer process. This capability allows you to determine exactly how and what to transfer, ensuring your data remains secure and in sync.