Backing up the data on your computer is one of the most frequently given advice to computer owners, and there are a number of ways to accomplish it.
The oldest way is to copy the data to an external media. Originally this was tapes, today it will frequently be one or more external harddrive or SSD. Swapping between at least two complete backups is recommended, with the inactive drives stored off-site to avoid destruction or loss in case of fire, theft, or other disasters (and if your area is prone to major disasters, it might be an idea to occasionally store a backup copy in a safe location hundreds of kilometers away; storage over a network connection could be an option for this).
More recently, online backup storage has become more common. Personally, I am slightly skeptical of these, mostly due to the loss of access control, but also because cloud services occasionally have service disruptions, and in some cases lose the data entrusted to them. In case you use such a service, my recommendation is to make sure the data are encrypted locally with a key not known to the service before they are uploaded; this prevents the service from accidentally or intentionally accessing your data, as well as preventing other unauthorized access. Another problem with such services is that they occasionally shut down business with little or no warning, so even if you use such a service, a local backup is recommended anyway. Backing up locally is also recommended when using online application services; these services are useful for working with others, but you might lose access when you most need the access.
There are various ways to perform a backup, from just using a simple copy command, to using more advanced backup applications in the OS, to purchasing commercial backup tools. Trial or Freeware versions of many such tools are frequently included on external harddrives.
My backup system
In my system at home I swap between two external SSD harddrives, and use Windows’s Backup software to manage the backup. Previously, I used a similar system with a commercial tool, but once I moved to Windows 10, I found that the Backup software in Windows seemed to work better for my purposes and I switched to it.
Better does not mean “perfect”, though. There are a few issues, but reasonably minor: 1) Swapping drives destroys the backup configuration, so I have to re-enter it when connecting the second drive. 2) The software does not resume backing up data from where it left off on the reconnected drive, causing it to use a lot more disk space, and requires occasional cleanup to remove old backups.
All this was manageable. At least until last week.
Microsoft breaks the backup
Recently, I finally caved in and allowed Windows 10 on my home computer to be updated to Feature Update 2004. Considering the problems that had been reported about loss of data in Chromium-based browsers, maybe I shouldn’t have, but Windows was now insisting on updating.
A couple of days after the update I switched backup disks, cleaned up some very old backups that were no longer needed, and set up the backup configuration again, and started a backup. A backup that failed! No data was copied to the drive.
I found no errors reported in the normal Event Viewer logs, until I dug down into the application specific logs for “File History backup”, where I found this meaningless warning: “Unusual condition was encountered during scanning user libraries for changes and performing backup of modified files for configuration <name of configuration file>”, with no information about what the “unusual condition” was.
As I usually do when having a problem like this, in order to find out what caused the problem, I started to test with the default configuration and then add more source drives for the backup to see which one broke the system.
The default configuration did copy those files, but it also copied a directory from one of my other drives, the main data drive, the copied directory is where I store all my photos. This directory was not part of the configuration. This directory may have been included because it is the configured default destination folder for the Windows photo import software.
However, when I added the rest of that drive to the list of folders to copy, no further files were copied (although a couple of days later some of the upper level folders did get backed up, none of the important folders were copied).
Removing that drive from the list, and adding the other drives I have for various tasks, projects, and software, those drives did get copied properly.
Going back to the problematic drive, further experiments did not succeed at backing up that drive more than the mentioned top level folders. Even experimentally adding some sibling folders of the Photo folder did not work; they weren’t even added to the list of folders to backup.
Eventually, I was forced to do a manual copy of that drive to a separate area of the backup drive, to make sure I did have a copy of it.
At present my conclusion is that in Feature update 2004 Microsoft did
something to the Backup/File History software, and it broke my system for
backups.
My initial guess at the cause of this problem is that the addition of the photo folder conflicts with adding the rest of the same drive to the list of files and folders to back up. Such overlapping lists should be merged, not create a fatal error.
A backup problem like this may not be a Security Vulnerability(TM), but it is definitely a Security Problem.
I have reported this via the Windows Feedback App, as well as to the @MicrosoftHelps Twitter account, but have so far not received any information about how to fix this problem (so, no help, so far).
Microsoft, there are some systems that should never break in production systems. The file system is one, account storage is another, and the backup software is one of the others that should never break. In this release it looks like you broke two such systems. And at least one is still broken 5-6 months after the public release!
Please fix this. Immediately!
Photo by Markus Spiske on Unsplash
I almost always have issues with a lot of the ready-made, automated backup solutions. I don’t trust the one built into Windows as I seem to remember there was such a system built in to the Windows 9x series, and later versions of Windows didn’t use it, so I’m not sure how you’d access those old backups. I don’t trust them not to drop support for it. Similarly I’ve never used Timeshift on Linux Mint as I’ve never really felt the need.
Years ago, I used to just boot from a DOS floppy disk and image my entire machine’s hard disk with Norton Ghost. It was clever enough to be able to restore the image to a different sized hard disk, etc., but the more I thought about it, the more I realised that’s a bit drastic and time-consuming. Now, I only backup my personal data. It’s easy to just reinstall your operating system and programs, if you have all of your personal files and configuration sitting on a hard disk, ready to copy back. If you’re a GNU/Linux user, you can restore all of your programs with a single (possibly long) command to your package manager. If you’re a Windows user, that’s what Chocolatey is for! 😉 Then just copy-back your personal files and you’re done!
Some programs and browser-extensions let you save and restore configuration settings to a single file, meaning there’s minimal customisation for you to do once you’ve got your programs reinstalled. If no such save/restore config is available, it’s still possible to find out and copy the profile folder from popular applications such as Libreoffice, Vivaldi, Firefox and Thunderbird. If you’re a GNU/Linux user, you can copy select configuration files over from your /etc/ or ~/.config/ folders. I don’t know what the equivalents are on Windows, but they’ll be hidden in there somewhere.
I always store my documents on a separate hard disk that’s not configured to be my /home/ or “My Documents” folder. This makes it very easy to back these up as I can be certain of the location of everything.
I have a couple of external hard disks that I use for backup disks, and I do my backups sort-of manually. My method (following) will work with anything that looks like a folder or a hard disk to your operating system (for instance, cloud storage). You can optionally use encryption such as Bitlocker, LUKS, Veracrypt or Cryptomator on your external hard disk. If you use cloud storage, Cryptomator will be essential.
To make a backup, I perform the following on a GNU/Linux system:
rsync -av --delete --progress -h --exclude "System Volume Information" --exclude "lost+found" --exclude "\$RECYCLE.BIN" --exclude ".Trash-1000" --no-o --no-g --no-p --modify-window=1 --safe-links /source-dir/ /dest-dir/This also works in reverse for restoring your backup. It will mirror the directory structure, only copying across files that are newer than those at the destination. If you have deleted a file from your computer, it will delete it from the backup too. I exclude some of the Windows and Linux system directories to avoid problems and save time. The “modify-window” option is a bit of a dirty kluge to avoid constantly re-copying unchanged files, where one filesystem is a FAT filesystem (it has much more “grainy” timestamps than the likes of NTFS, EXT4, etc).
An approximate equivalent for a Windows based system would be:
robocopy /e /xo /purge "c:\source dir" "z:\destination dir"I think you can probably use an option such as /xd “System Volume Information” or /xd:”System Volume Information” (I’m not sure of the exact syntax) to make xcopy not try to backup or delete the contents of reserved system folders. You’ll have to doublecheck that as I have actively avoided using Windows whenever at all possible for about 15 years.
The first backup may take a long time, but subsequent backups are extremely fast because the only files that get copied are the ones that have been changed since the last backup.
I use several external hard disks in rotation so I have more than one backup of my files, at more than one location. If you drop your external hard disk down a flight of steps, it’s reassuring to know you’ve got a spare at your parents’/friend’s house.
A variation of this, initially using xcopy (DOS) and either cp or a tar-pipe on *nix, has served me well for approximately 25 years.
Remember:
* It’s not a backup until it’s offline AND redundant.
* There’s not such thing as the cloud, only other people’s computers.
This means:
As a minimum you should have 2 backups of your data, 1 being local.
AND
No one cares about protecting your files as much as you. If it’s on someone else’s computer, they can get hacked, decide to shut-down their service, or replace the hardware and just throw their unencrypted hard-disks in the trash for any fraudster/criminal to pick up. Always encrypt data locally before transferring it to other people’s computers, and don’t rely on other people’s computers as your sole backup.
> * It’s not a backup until it’s offline AND redundant
And until it’s tested. In the wise words of a friend, if you think you have backups and haven’t tested them, you don’t have backups, you have only hope.
I don’t know about how it was in older Windows versions, but the current Windows 10 Backup software do verbatim copies of the file to the destination drive, AFAICT the only thing it changes is to append a timestamp for when it was copied to the copy’s name (keeping the extension, of course).
As above I have always used DoS/Windows command line tools, XCopy and RoboCopy to backup.
I have found the Windows Feedback Hub is about as useful as a screen door on a submarine (my Nickel’s worth).
The other thing about cloud backups is that it requires an internet connection.
Any different with the build update of Windows 10 20H2 (the build after 2004)?