2

For backup strategies that don't provide "unlimited" storage (especially those that bill by the gigabyte like Amazon Glacier, Backblaze B2, etc.) there is a strong motivation not to backup things that can be re-downloaded from the Internet for free anyway.

For example:

  • Don't backup any of my Steam games' game data that I downloaded from the Steam servers, because if I lose my HDD and have to restore from a backup, I'd just as easily download the game again from Steam instead of pulling it from my backup service (plus I'd get the latest version which the backup might not have, and save tons of money if my backup provider bills by the GB).
  • Don't backup most program files. It's a waste of space to have a copy of MS Office, Visual Studio, Acrobat Reader, Chrome, etc. stored in a backup repository, and if you're uploading to the cloud, this could make a huge difference in the time it takes to backup regardless of whether the service offers unlimited storage or not.
  • Don't backup temp files, caches, hiberfil.sys, etc.

But then there are plenty of things that can be "mixed in" with those files that you would definitely want to backup:

  • Do backup any screenshots that happen to be saved in a game-specific location (which occasionally, is a subfolder of the game's binaries' installation folder, making a normal manual rules-based backup strategy very complicated to build and error-prone).
  • Do backup configuration files, tweaks, mods, etc. as these can be difficult to come by or reconstruct.
  • Do backup most application-specific settings in AppData\Local or AppData\Roaming, because the time it'd take to configure your apps again is outweighed by the relatively small storage cost of backing up this configuration. (Most apps' configuration data in AppData is a few megabytes at most.)
  • Do backup most things in the user directory (almost everything in Documents, Videos and Pictures folders ought to be backed up), except for replaceable downloads, like Nvidia drivers, BIOS updates, new versions of free / easily obtained program installers, Linux ISOs, etc.
    • ...

As you can see, for a power user who runs a lot of software and stores a lot of data, it can be really tedious to define all the rules manually of "I do want X" vs. "I don't want Y" in a data backup strategy that's folder/file-based.

There are three important reasons why you would want to be specific and selective:

  • Backup plans that bill based on storage usage will charge a much higher rate if you're storing everything (might be 1+ TB of data altogether) vs. just the absolutely irreplaceable data (for most people probably <500GB).
  • Unless your upstream is ridiculously fast, it could take months or longer to upload all of your data. Even being a little selective could make the difference between an 8 month time to backup and a 1 month time to backup. The 8 month one isn't even worth the effort, IMO.
  • Even if your upstream is ridiculously fast, you could be using it for something else while you're uploading that 40 GB of data you downloaded from Steam and can re-download at any time on a fresh install of Steam on a new computer. In short, constantly or frequently utilizing more of your upstream than necessary bogs down your network. This surprises exactly no one, I hope.
  • Many "metered" backup providers bill you for network traffic as well, so they "double dip" on your wasted, unnecessary uploads.

All of this is to describe a problem. I am hoping a software solution exists to assist with this problem, to at least make it somewhat easier for an end-user to select and manage which content should and should not be backed up to the cloud.

Here are some more basic software requirements I'm also looking for:

  • Must run on Windows 10 at least; if it runs on MacOS too that's a huge plus
  • Good integration with at least one cloud-based backup solution (it doesn't have to be a "general purpose" backup program; it can certainly be a client purpose built for one individual service, just name the service in your answer)
  • At least one of the cloud-based backup solutions supported by the software must have prices close to (competitive with) the cost of Backblaze B2 or Amazon Glacier (the service may be something other than these two, however, list what service you recommend to use with the program.)
  • If the plan is for unlimited storage, there should at least be an entry level tier that is available for no more than $20/month for one device.

Does anything like this exist, or are the requirements far too ambitious? I know I'm basically asking the program to be sort of a mind-reader, but I was hoping they could take the "big data" approach like we do with Ad-Blockers: instead of having to add whitelists and blacklists for each and every ad or non-ad site you come across in the world, other people have already created vast lists that you can subscribe to that infuse your browsing with instant knowledge of categorizing each domain/URL/HTML DOM element as "Ad" or "Non-Ad".

With a similar approach to backups, we could have lists categorizing each file/path as "Back it up" or "Replaceable". It's the same type of problem and could be solved the same way as Adblock solves its own problem.

  • 1
    A part of the answer is selective positioning of downloads - i.e. if you are downloading things that you don't wish in your backup such as OS ISOs or Games either download them to or move them immediately to a location that is outside of your backup tree, (rather than user/downloads), likewise move important screenshots out of Program Files, where they should never have been anyway, to somewhere that is backed up. N.B. if a program is saving to Program Files it is running as admin and can do major damage to your system. – Steve Barnes Aug 26 '17 at 6:02
  • 1
    So, what you're really asking for is some sort of predefined inclusion/exclusion list? I'd say backups are far too personal for any such list to exist - an adblocker is a poor analogy. A lot of common backup software will exclude (by default) common system-wide temp locations, page/hiberfiles, etc., but anything more specific than that has to be custom. – Bob Aug 26 '17 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.