ConoHa の VPS に RHEL9 をインストールしていろいろ設定してきましたが、今回は遠隔バックアップの設定を入れていこうと思います。
既に MariaDB の設定の際に、ローカルの自動バックアップは行っていましたが、何かあった時のために Google Drive にも自動でコピーを置くようにします。
Rclone のインストール
EPEL リポジトリにある Rclone は、バージョンが 1.57.0(2021年11月)で結構古いものでした。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | # dnf info rclone Available Packages Name : rclone Version : 1.57.0 Release : 2.el9 Architecture : src Size : 22 M Source : None Repository : epel-source Summary : Rsync for cloud storage URL : http://rclone.org/ License : MIT Description : : Rclone is a command line program to sync files and directories to : and from various cloud services. |
事前に調べたところ古いバージョンだと Google の認証まわりで不具合があるらしいので、新しいものを使った方が良さそうです。
公式サイトを見ると、最新バージョンは 1.66.0(2024年5月現在)でしたので、今回はこれを使用していきます。
インストールガイドによると Intel/AMD(64 Bit)の Zip ファイルをダウンロードして、バイナリファイルを適切な場所(/usr/bin/)に配置すれば良さそうですが、面倒なので今回は公式サイトの RPM からインストールしました。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | # dnf install https://downloads.rclone.org/rclone-current-linux-amd64.rpm rclone-current-linux-amd64.rpm Dependencies resolved. ============================================================================================================================================================= Package Architecture Version Repository Size ============================================================================================================================================================= Installing: rclone x86_64 1.66.0-1 @commandline 21 M Transaction Summary ============================================================================================================================================================= Install 1 Package Total size: 21 M Installed size: 65 M Is this ok [y/N]: y Downloading Packages: Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : rclone-1.66.0-1.x86_64 1/1 Running scriptlet: rclone-1.66.0-1.x86_64 1/1 Verifying : rclone-1.66.0-1.x86_64 1/1 Installed: rclone-1.66.0-1.x86_64 Complete! |
Google Drive の設定
Rcolone の設定に入る前に、先に Google Drive でバックアップデータの保存先を作成しておきます。
このフォルダが Rclone でアクセスできる、ルートフォルダになります。
作成したフォルダに移動し、フォルダの ID を取得しておきます。
※URL で folders/ 以降の文字列が、後述する Rclone の root_folder_id の値になります。
Google Cloud の設定
続いて Rclone から操作する Google Drive API の準備をしていきます。
Google Cloud でプロジェクトの作成
Google Cloud にログインします。
「API とサービス」のページに移動し、「プロジェクトを作成」を選択してください。
プロジェクト名を入力(ここでは「Rclone Backup」としました)して、作成ボタンを押します。
Google Drive API の有効化
API とサービスを有効にすると、API ライブラリを選択する画面に遷移します。
API ライブラリの中から、Google Drive API を選択します。
Google Drive API を有効にするか尋ねられますので、「有効にする」を選択してください。
Google Drive API が有効になりました。
OAuth 同意画面の設定
認証情報を作成のボタンを押したくなるところですが、左のメニューから「OAuth 同意画面」を開きます。
OAuth 同意画面が開きますので、User Type を「外部」にして作成ボタンをクリックします。
アプリ登録の編集画面が表示されますので、任意のアプリ名を入力します。
また、ユーザーサポートメールとデベロッパーの連絡先情報には、Google アカウントで使用しているメールアドレスを入力します。
保存して次へ進むと、スコープの設定画面になりますので「スコープの追加または削除」をクリックします。
選択したスコープの更新画面になりますので、フィルタに「drive」と入れて絞り込みます。
範囲が「.../auth/docs」、「.../auth/drive」、「.../auth/drive.metadata.readonly」の 3 つのスコープにチェックを入れ、更新ボタンを押してください。
機密性の高いスコープに「.../auth/docs」が、制限付きのスコープに「.../auth/drive」と「.../auth/drive.metadata.readonly」が追加されたことを確認して、「保存して次へ」をクリックします。
テストユーザーの画面に進みますが、ユーザーは追加せずに次へ進みます。
OAuth 同意画面の概要が表示されますので、間違いがなければダッシュボードに戻ります。
OAuth 同意画面のダッシュボードに戻ったら、公開ステータスの「アプリを公開」ボタンをクリックして、本番環境にプッシュします。
公開ステータスが「テスト」から「本番環境」に変化しました。
確認ステータスで「確認が必要」となっていますが、一般公開しないので無視して大丈夫です。
認証情報を作成
続いて、認証情報の設定をしていきます。
左のメニューで「認証情報」を開き、画面上部の「+ 認証情報を作成」から「OAuth クライアント ID」を選択します。
OAuth クライアント ID の作成画面になりますので、アプリケーションの種類から「デスクトップアプリ」を選択し、任意の名前(ここでは Rclone)を付け、作成ボタンを押します。
OAuth クライアントを作成しましたというダイアログが表示されました。
クライアント ID と、クライアント シークレットが Rclone の設定の際に必要になりますので、この画面を表示したまま Rclone 側の設定を行っていきます。
Rclone のセットアップ
rclone config を実行して、遠隔バックアップの設定をしていきます。
新しく設定を入れていくので「n」で New remote を指定してください。
1 2 3 4 5 6 | # rclone config No remotes found - make a new one n) New remote s) Set configuration password q) Quit config n/s/q> n |
リモートのサービスを指定
続いて、リモートに対して任意の名前(ここでは GoogleDrive)を付け、サービスの種類(Google Drive の場合は 17)を指定します。
※サービスの番号(17)ではなく、drive と指定しても大丈夫です。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | name> GoogleDrive Option Storage. Type of storage to configure. Choose a number from below, or type in your own value. 1 / 1Fichier \ (fichier) 2 / Akamai NetStorage \ (netstorage) 3 / Alias for an existing remote \ (alias) 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, ChinaMobile, Cloudflare, DigitalOcean, Dreamhost, GCS, HuaweiOBS, IBMCOS, IDrive, IONOS, LyveCloud, Leviia, Liara, Linode, Minio, Netease, Petabox, RackCorp, Rclone, Scaleway, SeaweedFS, StackPath, Storj, Synology, TencentCOS, Wasabi, Qiniu and others \ (s3) 5 / Backblaze B2 \ (b2) 6 / Better checksums for other remotes \ (hasher) 7 / Box \ (box) 8 / Cache a remote \ (cache) 9 / Citrix Sharefile \ (sharefile) 10 / Combine several remotes into one \ (combine) 11 / Compress a remote \ (compress) 12 / Dropbox \ (dropbox) 13 / Encrypt/Decrypt a remote \ (crypt) 14 / Enterprise File Fabric \ (filefabric) 15 / FTP \ (ftp) 16 / Google Cloud Storage (this is not Google Drive) \ (google cloud storage) 17 / Google Drive \ (drive) 18 / Google Photos \ (google photos) 19 / HTTP \ (http) 20 / Hadoop distributed file system \ (hdfs) 21 / HiDrive \ (hidrive) 22 / ImageKit.io \ (imagekit) 23 / In memory object storage system. \ (memory) 24 / Internet Archive \ (internetarchive) 25 / Jottacloud \ (jottacloud) 26 / Koofr, Digi Storage and other Koofr-compatible storage providers \ (koofr) 27 / Linkbox \ (linkbox) 28 / Local Disk \ (local) 29 / Mail.ru Cloud \ (mailru) 30 / Mega \ (mega) 31 / Microsoft Azure Blob Storage \ (azureblob) 32 / Microsoft Azure Files \ (azurefiles) 33 / Microsoft OneDrive \ (onedrive) 34 / OpenDrive \ (opendrive) 35 / OpenStack Swift (Rackspace Cloud Files, Blomp Cloud Storage, Memset Memstore, OVH) \ (swift) 36 / Oracle Cloud Infrastructure Object Storage \ (oracleobjectstorage) 37 / Pcloud \ (pcloud) 38 / PikPak \ (pikpak) 39 / Proton Drive \ (protondrive) 40 / Put.io \ (putio) 41 / QingCloud Object Storage \ (qingstor) 42 / Quatrix by Maytech \ (quatrix) 43 / SMB / CIFS \ (smb) 44 / SSH/SFTP \ (sftp) 45 / Sia Decentralized Cloud \ (sia) 46 / Storj Decentralized Cloud Storage \ (storj) 47 / Sugarsync \ (sugarsync) 48 / Transparently chunk/split large files \ (chunker) 49 / Union merges the contents of several upstream fs \ (union) 50 / Uptobox \ (uptobox) 51 / WebDAV \ (webdav) 52 / Yandex Disk \ (yandex) 53 / Zoho \ (zoho) 54 / premiumize.me \ (premiumizeme) 55 / seafile \ (seafile) Storage> 17 |
次に、サービスごとのオプションを指定していきます。
先ほど取得したクライアント ID を貼り付けます。
1 2 3 4 5 6 7 | Option client_id. Google Application Client Id Setting your own is recommended. See https://rclone.org/drive/#making-your-own-client-id for how to create your own. If you leave this blank, it will use an internal key which is low performance. Enter a string value. Press Enter for the default (""). client_id> xxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com |
同様に、先ほど取得したクライアント シークレットを貼り付けます。
1 2 3 4 5 | Option client_secret. OAuth Client Secret. Leave blank normally. Enter a string value. Press Enter for the default (""). client_secret> GOCSPX-XXXXXXXXXXXXXXXXXXXXXXXXXXXX |
※ここまで入力したら、Google Cloud 側のダイアログは閉じても大丈夫です。
続いてアクセス権の設定になります。
バックアップファイルを書き込むので、フルアクセス(1)を入力します。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | Option scope. Comma separated list of scopes that rclone should use when requesting access from drive. Choose a number from below, or type in your own value. Press Enter to leave empty. 1 / Full access all files, excluding Application Data Folder. \ (drive) 2 / Read-only access to file metadata and file contents. \ (drive.readonly) / Access to files created by rclone only. 3 | These are visible in the drive website. | File authorization is revoked when the user deauthorizes the app. \ (drive.file) / Allows read and write access to the Application Data folder. 4 | This is not visible in the drive website. \ (drive.appfolder) / Allows read-only access to file metadata but 5 | does not allow any access to read or download file content. \ (drive.metadata.readonly) scope> 1 |
Rclone からアクセス可能な Google Drive のフォルダを指定したいので、Advanced config で Yes(y)を応答します。
1 2 3 4 5 6 7 8 9 10 11 12 | Option service_account_file. Service Account Credentials JSON file path. Leave blank normally. Needed only if you want use SA instead of interactive login. Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`. Enter a value. Press Enter to leave empty. service_account_file> Edit advanced config? y) Yes n) No (default) y/n> y |
root_folder_id には、前述した Google Drive のフォルダ ID を入力します。
それ以外の設定は全て空エンターで、デフォルト値を設定します。
最後に、再び Edit advanced config? と尋ねられますので、今度は No(n)で先に進みます。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 | Option token. OAuth Access Token as a JSON blob. Enter a value. Press Enter to leave empty. token> Option auth_url. Auth server URL. Leave blank to use the provider defaults. Enter a value. Press Enter to leave empty. auth_url> Option token_url. Token server url. Leave blank to use the provider defaults. Enter a value. Press Enter to leave empty. token_url> Option root_folder_id. ID of the root folder. Leave blank normally. Fill in to access "Computers" folders (see docs), or for rclone to use a non root folder as its starting point. Enter a value. Press Enter to leave empty. root_folder_id> xxxxxGoogleDriveFolderIDxxxxx Option auth_owner_only. Only consider files owned by the authenticated user. Enter a boolean value (true or false). Press Enter for the default (false). auth_owner_only> Option use_trash. Send files to the trash instead of deleting permanently. Defaults to true, namely sending files to the trash. Use `--drive-use-trash=false` to delete files permanently instead. Enter a boolean value (true or false). Press Enter for the default (true). use_trash> Option copy_shortcut_content. Server side copy contents of shortcuts instead of the shortcut. When doing server side copies, normally rclone will copy shortcuts as shortcuts. If this flag is used then rclone will copy the contents of shortcuts rather than shortcuts themselves when doing server side copies. Enter a boolean value (true or false). Press Enter for the default (false). copy_shortcut_content> Option skip_gdocs. Skip google documents in all listings. If given, gdocs practically become invisible to rclone. Enter a boolean value (true or false). Press Enter for the default (false). skip_gdocs> Option show_all_gdocs. Show all Google Docs including non-exportable ones in listings. If you try a server side copy on a Google Form without this flag, you will get this error: No export formats found for "application/vnd.google-apps.form" However adding this flag will allow the form to be server side copied. Note that rclone doesn't add extensions to the Google Docs file names in this mode. Do **not** use this flag when trying to download Google Docs - rclone will fail to download them. Enter a boolean value (true or false). Press Enter for the default (false). show_all_gdocs> Option skip_checksum_gphotos. Skip checksums on Google photos and videos only. Use this if you get checksum errors when transferring Google photos or videos. Setting this flag will cause Google photos and videos to return a blank checksums. Google photos are identified by being in the "photos" space. Corrupted checksums are caused by Google modifying the image/video but not updating the checksum. Enter a boolean value (true or false). Press Enter for the default (false). skip_checksum_gphotos> Option shared_with_me. Only show files that are shared with me. Instructs rclone to operate on your "Shared with me" folder (where Google Drive lets you access the files and folders others have shared with you). This works both with the "list" (lsd, lsl, etc.) and the "copy" commands (copy, sync, etc.), and with all other commands too. Enter a boolean value (true or false). Press Enter for the default (false). shared_with_me> Option trashed_only. Only show files that are in the trash. This will show trashed files in their original directory structure. Enter a boolean value (true or false). Press Enter for the default (false). trashed_only> Option starred_only. Only show files that are starred. Enter a boolean value (true or false). Press Enter for the default (false). starred_only> Option export_formats. Comma separated list of preferred formats for downloading Google docs. Enter a string value. Press Enter for the default (docx,xlsx,pptx,svg). export_formats> Option import_formats. Comma separated list of preferred formats for uploading Google docs. Enter a value. Press Enter to leave empty. import_formats> Option allow_import_name_change. Allow the filetype to change when uploading Google docs. E.g. file.doc to file.docx. This will confuse sync and reupload every time. Enter a boolean value (true or false). Press Enter for the default (false). allow_import_name_change> Option list_chunk. Size of listing chunk 100-1000, 0 to disable. Enter a signed integer. Press Enter for the default (1000). list_chunk> Option impersonate. Impersonate this user when using a service account. Enter a value. Press Enter to leave empty. impersonate> Option upload_cutoff. Cutoff for switching to chunked upload. Enter a size with suffix K,M,G,T. Press Enter for the default (8Mi). upload_cutoff> Option chunk_size. Upload chunk size. Must a power of 2 >= 256k. Making this larger will improve performance, but note that each chunk is buffered in memory one per transfer. Reducing this will reduce memory usage but decrease performance. Enter a size with suffix K,M,G,T. Press Enter for the default (8Mi). chunk_size> Option acknowledge_abuse. Set to allow files which return cannotDownloadAbusiveFile to be downloaded. If downloading a file returns the error "This file has been identified as malware or spam and cannot be downloaded" with the error code "cannotDownloadAbusiveFile" then supply this flag to rclone to indicate you acknowledge the risks of downloading the file and rclone will download it anyway. Note that if you are using service account it will need Manager permission (not Content Manager) to for this flag to work. If the SA does not have the right permission, Google will just ignore the flag. Enter a boolean value (true or false). Press Enter for the default (false). acknowledge_abuse> Option keep_revision_forever. Keep new head revision of each file forever. Enter a boolean value (true or false). Press Enter for the default (false). keep_revision_forever> Option v2_download_min_size. If Object's are greater, use drive v2 API to download. Enter a size with suffix K,M,G,T. Press Enter for the default (off). v2_download_min_size> Option pacer_min_sleep. Minimum time to sleep between API calls. Enter a duration s,m,h,d,w,M,y. Press Enter for the default (100ms). pacer_min_sleep> Option pacer_burst. Number of API calls to allow without sleeping. Enter a signed integer. Press Enter for the default (100). pacer_burst> Option server_side_across_configs. Deprecated: use --server-side-across-configs instead. Allow server-side operations (e.g. copy) to work across different drive configs. This can be useful if you wish to do a server-side copy between two different Google drives. Note that this isn't enabled by default because it isn't easy to tell if it will work between any two configurations. Enter a boolean value (true or false). Press Enter for the default (false). server_side_across_configs> Option disable_http2. Disable drive using http2. There is currently an unsolved issue with the google drive backend and HTTP/2. HTTP/2 is therefore disabled by default for the drive backend but can be re-enabled here. When the issue is solved this flag will be removed. See: https://github.com/rclone/rclone/issues/3631 Enter a boolean value (true or false). Press Enter for the default (true). disable_http2> Option stop_on_upload_limit. Make upload limit errors be fatal. At the time of writing it is only possible to upload 750 GiB of data to Google Drive a day (this is an undocumented limit). When this limit is reached Google Drive produces a slightly different error message. When this flag is set it causes these errors to be fatal. These will stop the in-progress sync. Note that this detection is relying on error message strings which Google don't document so it may break in the future. See: https://github.com/rclone/rclone/issues/3857 Enter a boolean value (true or false). Press Enter for the default (false). stop_on_upload_limit> Option stop_on_download_limit. Make download limit errors be fatal. At the time of writing it is only possible to download 10 TiB of data from Google Drive a day (this is an undocumented limit). When this limit is reached Google Drive produces a slightly different error message. When this flag is set it causes these errors to be fatal. These will stop the in-progress sync. Note that this detection is relying on error message strings which Google don't document so it may break in the future. Enter a boolean value (true or false). Press Enter for the default (false). stop_on_download_limit> Option skip_shortcuts. If set skip shortcut files. Normally rclone dereferences shortcut files making them appear as if they are the original file (see [the shortcuts section](#shortcuts)). If this flag is set then rclone will ignore shortcut files completely. Enter a boolean value (true or false). Press Enter for the default (false). skip_shortcuts> Option skip_dangling_shortcuts. If set skip dangling shortcut files. If this is set then rclone will not show any dangling shortcuts in listings. Enter a boolean value (true or false). Press Enter for the default (false). skip_dangling_shortcuts> Option resource_key. Resource key for accessing a link-shared file. If you need to access files shared with a link like this https://drive.google.com/drive/folders/XXX?resourcekey=YYY&usp=sharing Then you will need to use the first part "XXX" as the "root_folder_id" and the second part "YYY" as the "resource_key" otherwise you will get 404 not found errors when trying to access the directory. See: https://developers.google.com/drive/api/guides/resource-keys This resource key requirement only applies to a subset of old files. Note also that opening the folder once in the web interface (with the user you've authenticated rclone with) seems to be enough so that the resource key is not needed. Enter a value. Press Enter to leave empty. resource_key> Option fast_list_bug_fix. Work around a bug in Google Drive listing. Normally rclone will work around a bug in Google Drive when using --fast-list (ListR) where the search "(A in parents) or (B in parents)" returns nothing sometimes. See #3114, #4289 and https://issuetracker.google.com/issues/149522397 Rclone detects this by finding no items in more than one directory when listing and retries them as lists of individual directories. This means that if you have a lot of empty directories rclone will end up listing them all individually and this can take many more API calls. This flag allows the work-around to be disabled. This is **not** recommended in normal use - only if you have a particular case you are having trouble with like many empty directories. Enter a boolean value (true or false). Press Enter for the default (true). fast_list_bug_fix> Option metadata_owner. Control whether owner should be read or written in metadata. Owner is a standard part of the file metadata so is |