Rclone - A Beginner's Guide

♲What is Rclone

Rclone is an open source command-line program written in Go, used to manage or migrate files on cloud storage. It supports 40+ cloud services including all the mainstream providers like Google Drive, DropBox, Mega, pcloud, S3 only to name a few. Rclone enables users to manage the cloud interface right from their terminal. It has gained so much appreciation that users call it "The Swiss Army Knife of cloud storage" and often refer it to as "Technology indistinguishable from magic".

📥 Installation

The installation process is quite straightforward actually. For Windows there is an executable that you can simply download as a zip archive and extract into a location of your choice and then open the Cmd in the same directory or optionally add the directory PATH to env. To install rclone on Linux/macOS/BSD systems, run:

$ curl https://rclone.org/install.sh | sudo bash

If you are on macOS & have Homebrew installed, run this command followed by the Enter key to install rclone.

$ brew install rclone

Now that we are done with installation, let's get familiar with some basic terminology of Rclone.

📜 Terminology

• Remote : A remote is simply the cloud service you have mounted on your system using rclone, for instance, if you have configured access to your Dropbox account using rclone then the Dropbox is your remote. If it's Google drive that you mounted, then Google Drive is your remote. Also note that we can have multiple number of remotes configured at the same time. For example, having 4 Mega, 3 Google Drive, and 2 pCloud accounts configured at the same time is completely normal.

• Commands : We use commands with rclone. The commands which make sense in the context of cloud storage can be used with rclone. For e.g, copy, sync, move, delete among many others can be used with rclone to manipulate files in cloud storage. Usually every rclone commands accepts two parameters, source and destination, now let's understand what they mean.

Suppose we have Google Drive & Dropbox accounts mounted as two distinct remotes and we are interested in copying files from Google Drive to Dropbox. In this context the Google Drive will be called a Source, which gets accepted as a first parameter of the rclone command and the Dropbox will be referred to as Destination.

Now that we all well versed with the basic terminology, let's dive into configuring the rclone remotes.

⚙️ Configuring a remote

To get started with configuration, on Windows open the Cmd in the same directory where you extracted the .exe of rclone and type the following command. In *nix based OS open your terminal & run:

$ rclone config

Because we have no remotes configured yet so we will get a response reading "No remotes found" and a prompt asking to choose an option:

Press n to create a new remote. The next prompt is to choose a name. Set a desired name for your remote and hit enter. This name will be used to identify this remote later. Next we are presented with a list of all available cloud storage rclone supports and we need to select the desired service.

Since I want to configure Google Drive, so I will type 18 and hit enter. Now the prompt will ask for Client Id first and then Client_secret. They are usually required for advanced usage like dealing with APIs, these fields are not mandatory so for now they can be skipped by simply hitting enter.

Now it asks for the scope, which has to do with access level basically. I want to have full access to all the files in drive so I will go with option 1. In essence full access will equip me with the ability to create, read, update, delete any files in my drive.

Next the prompt will ask for service_account_file which actually allows us to change the authentication method with Google Drive, simply leave it blank and hit enter.

After that we will be asked if we want to use advanced configurations, of course not, not when are we just starting, so hit n for no (which is default) or enter.

Now we will be asked whether we want to use auto config or not. If you configure rclone in your Mac, or Windows, or in another system with a GUI, you can say “yes” to this. If you were configuring rclone in a headless system (like SSHing into a Linux system with no GUI), you need to say “no” to this. Apparently I'm working on a GUI based OS so I will go with "yes".

This will launch your default browser with a Google Drive authorization page. Authorize rclone to access your Google Drive .

Now come back to the terminal and it will show the authorization status by printing Got code as a success message to the display. Next it will ask if we want to configure the remote as a Shared Drive. We will go with "no" because we want to mount our main storage as a remote and assuming you are using Google's free tier of Drive Storage (15 GB). The Shared Drives are available to Workspace accounts only, unless you are a member of one.

Finally, it will prompt if we want to save this as a remote configuration or make any edits (which we can make anytime though). Type y, hit enter and you are done. It will print the list of configured remotes on the screen and present with the interface we started with.

To exit out of the rclone interface press "q" and hit enter. Now that we successfully mounted our first remote, we are ready to interact with the files in our Google Drive.

🌲 Listing Remotes

To list all the configured remotes, run this command

$ rclone listremotes

As we keep adding more remotes, the list will be updated accordingly.

📝 Displaying contents

To display the contents of my remote (Google Drive), I need to pass the name of my remote as an argument to the rclone command along with the related option(s).

• lsd : this option is used to list directories.
• ls : this option is used to list all the files recursively in the specified path.

So the syntax of the command is:

$ rclone lsd remote_name:/
# OR
$ rclone ls remote_name:/

Here is the command in action listing the directories of my Google Drive.

There is another interactive command which *unix users might be already familiar with. It's called tree which lists all the files and directories recursively in a tree like structure within a specified path.

$ rclone tree gdrive:/

💾 Calculating size

To get the information on available or used disk space of remote, we use about option with rclone command.

So that provides us with the general disk size info about the remote. What if we are interested in knowing the size of a specific file or directory! Well in that case we use size option with rclone command as follows:

As you can see I used double quotes around the remote directory path, you get it already. It is because the directory name contains spaces in it.

📁 Copying

Copying from remote to local

In order to copy contents from remote to local directory we use following command. The -P flag, displays the download progress of the operation.

$ rclone "remote:/path/to/file/" ~/Downlods/ -P

Suppose I want to copy a file Atomic-Habits-Hindi-Audiobook.mp3 from my remote to local Downloads directory, here is how I can achieve that

Copying from local to remote

Previously we copied file from remote to local directory, This time we will do it the other way around. The syntax stays same except we switch the source and destination like this:

$ rclone ~/Downloads/file "remote:/directory/" -P

Suppose I want to copy salam.pdf from my local Downloads folder to the remote. Here is how I can achieve that

📂 Moving

Moving files between remote and local is identical to that of copying, except we use move option instead of copy.

$ rclone move "remote:/dir/" ~/Downloads/ -P --dry-run

The above command will move dir folder from remote to the local Downloads folder. The --dry-run flag at the end is advised to include if you want to test the command without actually executing it. It comes handy to check for any errors prior the operation. You can use --dry-run flag with copy and move commands as well.

🗳 Sync

This command syncs source to destination, modifying only destination in the process. Destination is updated to match source, including deleting files if necessary. Here is the command syntax

$ rclone sync ~/Documents/ "remote:/docs_backup/" -P

🗑 Delete & Purge

If you want to delete the contents of a folder without deleting the folder itself then delete command should come to the mind. Here is the command syntax

$ rclone delete remote:/tmp_files/ -P

The above command will delete all the contents of tmp_files directory but the folder itself will be retained. However if you are interested in deleting the contents as well as the directory itself, then purge command is for you.

$ rclone purge remote:/trash/ -P

The above command deletes the trash directory along with it's contents.

🗃 rclone config

You are already familiar with rclone config command. that is actually how we started. The point here is all the remotes are stored in a config file. If you run rclone config file in your shell the path to this config file will be displayed on the screen. In my case it prints /Users/cosmicqbit/.config/rclone/rclone.conf. This file is actually of great importance. It stores all the critical information about the remotes configured. This file can be used on a fresh system to operate on the remotes without actually going through any configuration process. So always make sure you keep this file close to your vest. That brings us to the end.

💌 Conclusion

We covered all the basics of rclone & I hope this guide will help you to get started with rclone. This was just scratching the surface, nonetheless these commands are what we will be relying on most of the time. Besides official documentation is always have your back. Don't think as though you have to, but if you feel like it, give an upvote and share the blog post with your friends. Thank you for taking time to read this 😊