Rclone 文件同步到云存储
概述
Rclone 是一个命令行程序，用于管理云存储上的文件。它是云供应商 Web 存储接口的功能丰富的替代品
超过 40 种云存储产品支持 rclone，包括 S3 对象存储、商业和消费者文件存储服务以及标准传输协议
Rclone 与 unix 命令 rsync、cp、mv、mount、ls、ncdu、tree、rm 和 cat 具有强大的云等效功
Rclone 熟悉的语法包括 shell 管道支持和--dry-run保护。它用于命令行、脚本或通过其API。
官网网址：https://rclone.org/
Github 项目：https://github.com/ncw/rclone
Rclone 能做什么？
将文件备份（和加密）到云存储
从云存储恢复（和解密）文件
将云数据镜像到其他云服务或本地
将数据迁移到云端，或在云存储供应商之间迁移
将多个、加密、缓存或多样化的云存储挂载为磁盘
使用lsf、ljson、size、ncdu分析和说明云存储中保存的数据
将文件系统联合在一起以将多个本地和/或云文件系统呈现为一个
Rclone安装
1、脚本安装
curl https://rclone.org/install.sh | bash2、源码安装
wget 'https://downloads.rclone.org/v1.57.0/rclone-v1.57.0-linux-amd64.zip'
unzip rclone-v1.57.0-linux-amd64.zip
cp ./rclone-*/rclone /usr/local/bin
rm -rf ./rclone-*3、查看版本&升级
rclone selfupdate --check ## 只检查最新版本
rclone selfupdate --stable ## 更新到最新稳定版本
rclone version ## 检查当前 rclone 版本Rclone配置
进入rclone菜单配置，可以链接目前主流的存储
rclone config常用命令说明
rclone config - 以控制会话的形式添加rclone的配置，配置保存在.rclone.conf文件中。
rclone copy - 将文件从源复制到目的地址，跳过已复制完成的。
rclone sync - 将源数据同步到目的地址，只更新目的地址的数据。 –dry-run标志来检查要复制、删除的数据
rclone move - 将源数据移动到目的地址。
rclone delete - 删除指定路径下的文件内容。
rclone purge - 清空指定路径下所有文件数据。
rclone mkdir - 创建一个新目录。
rclone rmdir - 删除空目录。
rclone check - 检查源和目的地址数据是否匹配。
rclone ls - 列出指定路径下所有的文件以及文件大小和路径。
rclone lsd - 列出指定路径下所有的目录/容器/桶。
rclone lsl - 列出指定路径下所有文件以及修改时间、文件大小和路径。
rclone md5sum - 为指定路径下的所有文件产生一个md5sum文件。
rclone sha1sum - 为指定路径下的所有文件产生一个sha1sum文件。
rclone size - 获取指定路径下，文件内容的总大小。.
rclone version - 查看当前版本。
rclone cleanup - 清空remote。
rclone dedupe - 交互式查找重复文件，进行删除/重命名操作。
rclone mount - 挂载云盘为本地硬盘
fusermount -qzu LocalFolder - 卸载挂载的云盘Rclone常用操作
LS
### 显示远端onedrive下的文件
rclone ls onedrive:file
OUT: 106622 header.png
### 显示远端bucket下的目录
rclone lsd s3-overseas:bucket-name
OUT: 0 2020-08-27 15:57:08 -1 new
### 以json形式列出bucket下的目录和文件
rclone lsjson s3-overseas:bucket-name
OUT: [
OUT: {"Path":"header.png","Name":"header.png","Size":106622,"MimeType":"image/png","ModTime":"2020-08-27T07:32:29.000000000Z","IsDir":false,"Tier":"STANDARD"},
OUT: {"Path":"new","Name":"new","Size":0,"MimeType":"inode/directory","ModTime":"2020-08-27T16:02:42.413393904+08:00","IsDir":true}
OUT: ]
### 以json形式递归的列出bucket-name下目录和文件
rclone lsjson s3-overseas:bucket-name -R
### 查看存储桶中500B以上的文件列表
rclone --min-size 500B lsl s3-overseas:bucket-nameSync
### 同步本地目录或文件到远端file
rclone sync <LOCAL_PATH> onedrive:file
### 同步远端file目录到本地
rclone sync onedrive:file <LOCAL_PATH>
### 远端同步到远端
# onedrive同步到onedrive
rclone sync onedrive1:file onedrive2:file
### 将本地文件同步到远端，并备份被删除或修改的文件到备份存储桶中
rclone sync <LOCAL_PATH> s3-overseas:bucket-name --backup-dir s3-overseas:backup-bucket-name/backup-dirCopy
### 拷贝本地文件到远端
rclone copy <LOCAL_PATH> s3-overseas:bucket-name/target-path/
## 拷贝远端对象到本地
rclone copy s3-overseas:bucket-name/target-path/ <LOCAL_PATH>
rclone copy --max-age 24h --progress --no-traverse <LOCAL_PATH> s3-overseas:bucket-name/target-path/Rclone 配置onedrive文件同步
1、获取Token
在Windows电脑上下载Rclone，下载地址:https://rclone.org/downloads/ ，然后解压，使用cmd进入解压后的文件夹：
rclone authorize "onedrive"整个大括号就是token，包括大括号。
2、连接onedrive
进入rclone配置菜单输入:
rclone config rclone config
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> onedrive
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / 1Fichier
\ "fichier"
2 / Alias for an existing remote
\ "alias"
3 / Amazon Drive
\ "amazon cloud drive"
4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, and Tencent COS
\ "s3"
5 / Backblaze B2
\ "b2"
6 / Box
\ "box"
7 / Cache a remote
\ "cache"
8 / Citrix Sharefile
\ "sharefile"
9 / Compress a remote
\ "compress"
10 / Dropbox
\ "dropbox"
11 / Encrypt/Decrypt a remote
\ "crypt"
12 / Enterprise File Fabric
\ "filefabric"
13 / FTP Connection
\ "ftp"
14 / Google Cloud Storage (this is not Google Drive)
\ "google cloud storage"
15 / Google Drive
\ "drive"
16 / Google Photos
\ "google photos"
17 / Hadoop distributed file system
\ "hdfs"
18 / Hubic
\ "hubic"
19 / In memory object storage system.
\ "memory"
20 / Jottacloud
\ "jottacloud"
21 / Koofr
\ "koofr"
22 / Local Disk
\ "local"
23 / Mail.ru Cloud
\ "mailru"
24 / Mega
\ "mega"
25 / Microsoft Azure Blob Storage
\ "azureblob"
26 / Microsoft OneDrive
\ "onedrive"
27 / OpenDrive
\ "opendrive"
28 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ "swift"
29 / Pcloud
\ "pcloud"
30 / Put.io
\ "putio"
31 / QingCloud Object Storage
\ "qingstor"
32 / SSH/SFTP Connection
\ "sftp"
33 / Sugarsync
\ "sugarsync"
34 / Tardigrade Decentralized Cloud Storage
\ "tardigrade"
35 / Transparently chunk/split large files
\ "chunker"
36 / Union merges the contents of several upstream fs
\ "union"
37 / Webdav
\ "webdav"
38 / Yandex Disk
\ "yandex"
39 / Zoho
\ "zoho"
40 / http Connection
\ "http"
41 / premiumize.me
\ "premiumizeme"
42 / seafile
\ "seafile"
Storage> 26
** See help for onedrive backend at: https://rclone.org/onedrive/ **
OAuth Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id>
OAuth Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret>
Choose national cloud region for OneDrive.
Enter a string value. Press Enter for the default ("global").
Choose a number from below, or type in your own value
1 / Microsoft Cloud Global
\ "global"
2 / Microsoft Cloud for US Government
\ "us"
3 / Microsoft Cloud Germany
\ "de"
4 / Azure and Office 365 operated by 21Vianet in China
\ "cn"
region> 1 # 按照你的实际情况选择
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> n
Remote config
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> n
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
rclone authorize "onedrive"
Then paste the result below:
result> {"access_token":qqqqqqqqqq","expiry":"2021-03-03T15:54:34.7442912+08:00"} # 这里就是上面获得的token
Choose a number from below, or type in an existing value
1 / OneDrive Personal or Business
\ "onedrive"
2 / Root Sharepoint site
\ "sharepoint"
3 / Sharepoint site name or URL (e.g. mysite or https://contoso.sharepoint.com/sites/mysite)
\ "url"
4 / Search for a Sharepoint site
\ "search"
5 / Type in driveID (advanced)
\ "driveid"
6 / Type in SiteID (advanced)
\ "siteid"
7 / Sharepoint server-relative path (advanced, e.g. /teams/hr)
\ "path"
Your choice> 1
Found 1 drives, please select the one you want to use:
0: OneDrive (business) id=b!B6yHyawk7kq599Kb4wYXQFe1jafkFoFBqrwS7ZFYLJsUbhule-g-Rax7IpAhk6c8
Chose drive to use:> 0
Found drive 'root' of type 'business', URL: https://imissyou5201314-my.sharepoint.com/personal/codehero_imissyou5201314_onmicrosoft_com/Documents
Is that okay?
y) Yes (default)
n) No
y/n> y
--------------------
[onedrive]
type = onedrive
region = global
token = {"access_token":"###############"}
drive_id = b!B6yHyawk7kq599Kb4wYXQFe1jafkFoFBqrwS7ZFYLJsUbhule-g-Rax7IpAhk6c8
drive_type = business
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
onedrive onedrive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
e/n/d/r/c/s/q>整个大括号就是token，包括大括号。
执行 ls 或者 lsd 命令检查是否连接成功：
列出顶级文件夹，（onedrive 是我们设置的名字） rclone lsd onedrive:
列出所有文件 rclone ls onedrive:
同步执行脚本
每天0点执行onedriveA同步到onedriveB
/usr/bin/rclone sync onedriveA: onedriveB: >> /root/rclone.log 2>&1配合 crontab 定时执行，比如每天 0:0 执行一次，日志写到 /root/rclone.log。执行 crontab -e，i，添加下面代码进去，按一下 Esc，然后输入 :wq 回车保存退出。
0 0 0 * * * /usr/bin/rclone sync onedriveA: onedriveB: >> /root/rclone.log 2>&1