Blog Work Stories Contact

Cloud storage for the paranoid

For years I have used distributed cloud storage (in the form of Syncthing on computers and servers I controlled). Having access to all of your files across multiple computers is very useful!

But there are some burdens and limitations when you roll-your-own cloud storage with Syncthing:

  • No slick web interface to access the shared files
  • No mobile clients
  • No “share” links to stored content
  • You are responsible for updates, security hardening, etc.
  • Syncthing is yet another service to install on your machines
  • Syncthing is probably not something you should install on your work-issued computer – it may be considered a security risk by your IT department.

I was interested in trying out a service like OneDrive or Dropbox, but one thing always held me back: the idea that at any moment, and for any reason, the company could lock me out of my files.

The problem

No one wants to have their data held hostage by a third-party. How can you get the benefits of using cloud storage while also retaining ownership rights and having a level of assurance that your files will always be accessible?

The solution

Luckily, there’s a simple solution: Perform full backups of your cloud files in an environment that you control.

"Backup your data, you say?! What a novel idea!" /S

It sounds unremarkable, but in order to be reliable the backups need to:

  • Be done regularly, without fail
  • Fully backup everything in the cloud account
  • Occur even if your primary PCs (desktop/laptop) are switched off

The setup

I use rclone to sync files from my cloud storage accounts to a VM running Alpine Linux. rclone works with over 40 cloud storage providers, has a very easy-to-use CLI, and works with modern authentication systems.

  • A cron job runs daily, pulling down any file changes into the backup.

  • I have the replication job set to exhaustively copy all files in the account to the local machine.

Benefits

There are many benefits to this setup:

  • It’s always running, even if your main computers are not on.

  • It runs in the middle of the night, while you’re sleeping. No need to interact with it!

  • Your day-to-day machines can remain lean – they don’t need to store the entire contents of your cloud storage account.

  • Because there’s a time delay between backups, you have a longer moment of opportunity to recover files.

  • You can backup the cloud account to a ZFS-backed storage pool and get historical snapshots of the files in the account. This will allow you to recover any deleted or mutilated files at any point in time.

Reminders

You’ll want to periodically verify the backup job is working. Take a peek at the files being brought down and ensure your newest files are there.

You might even want to have the service email you a log of actions it performs so that you can keep an eye on things.

Wrap-up

Now that your data is safe, you can run wild with cloud storage! You can stop managing your own servers and systems, stop worrying so much about security vulnerabilities, and sleep a little better at night knowing that there’s very little chance now that you won’t be ever unfairly locked out from getting to your files within your cloud storage accounts.