161028 Data - matthew piers robertson • photography writing creative toronto

"Back up your data," they say.

It's good advice.

Running out of hard drive space, like hard drive failures, is something that will eventually happen to everyone. For the past month or so I've been dealing with both, although I saw one of them coming for quite some time.

I've been using a two-tier drive setup for years. Xenon, my main drive, is the internal system disk (chip set, these days) in my computer that's frighteningly fast but comparatively small. All data lands on it first, and after a couple of months is routinely shuffled off to a secondary data drive called Argon, which is a big striped raid and also very quick. Both of those would back up regularly – my data drive to a slower redundant raid called Krypton, and my system drive to two different spinning drives, Radium with a bootable clone and Helium as a Time Machine volume. Time Machine has never helped when I need it, but I like the idea that newly imported images are duplicated within an hour.

And yes, my hard drive naming does occasionally confuse me.

Of course, this can't last, and my fast-but-expensive Argon data drive was nearing capacity several months ago. I eventually decided to bring in a third hard drive tier, which would be the redundant and expandable Krypton raid that has served me – very slowly – for many years. I bought another hard drive, Neon, to back up Argon to. Neon is an 8TB drive, while Argon is only 5TB, which gives a lot of overflow room for Carbon Copy Cloner and its "SafetyNet" feature. I let that new setup stabilize and be happy for several weeks while I worked on the next part of the transition.

My Lightroom library is vast and contains multitudes.

To bring this under control I've needed to do some housekeeping. I've been moving files from their daily folders into large lumps by year, but that means that I could have several images with the same name – different cameras, different dates – that are now conflicting. So I've been retroactively renaming and moving tens of thousands of files at a time. All told rather painless, if tedious, and vital to being able to break my two-hundred-thousand image collection across two different drives.

After a particularly productive morning of moving and renaming I thought to myself: "Self, I should pause for a while and back up all of the changes I've just made."

Now, backing up takes a long time, because each new file name is copied as an entirely new file, and there's tens of thousands of them to write to the backup drive. Crucially Carbon Copy Cloner had enough room on Neon that it never deleted anything from its previous backup runs.

That backup couldn't be completed because there were errors on Argon – my main data drive.

It took three more tries for CCC to be able to finish, and the drive fought and complained the whole time. Needless to say I was terrified, because I didn't know if the backup CCC was writing to Neon was compromised as well, and I had wiped my older backup on Krypton just hours earlier so that I could start migrating new data to it.

In comedy as in tragedy, timing is everything.

I relaunched Lightroom and pointed it at Argon, and it came back with over thirty thousand photos missing. 

On the bright side, that's only about 14% of the catalog, and most of what I shoot is complete dreck, so there's not much actually lost. But some of my Lakefill and Horizon photos were in there, along with a lot of family photos that I'd hate to lose. Fortunately it only seemed to be photos from the last few years, so my wedding photos weren't hurt. Honestly, that was my biggest concern – six years ago this past week, and still the best thing I've ever done. But I digress.

I'm unable to repair Argon, which I hoped might let Lightroom find those photos. Bad sectors. Most of the time I can't even unmount the drive to scan it. Eventually I concede defeat, and convince the computer to format it. That works – blank slate – and I copy the backup disk across to restore it. Thirty thousand photos are still missing.

Carbon Copy Cloner has kept everything it ever copied over to the backup disk, so I point Lightroom at Neon's CCC safety net folders that are filled with the unrenamed files and ask it to import everything it finds there. It works, creating a vast number of duplicate files, but also reporting that thousands of photos couldn't be imported because they appeared damaged. I've blocked the total number from my mind, but it was a lot.

Next I manually went through those overflow folders, gathered the photos together, and imported them into Lightroom in smaller batches. This time there are no complaints about damaged files, which is very nice, but now my library has gone from just over two hundred thousand  images to over three hundred and twenty thousand. It was barely fitting on Argon before, and now I need to use Neon to hold active and possibly unique files. This is not optimal.

And those missing thirty thousand photos? Still missing. I try to track them down, and find which folder they currently live on… and can only match them one at a time. Usually Lightroom happily finds other missing photos in the same directory, but in this case they've been renamed, and that breaks the automation.

So that wasn't my best weekend ever.

The next Monday I was at the local big-box hard drive store for another backup drive, which I named Osmium because I've run out of noble gasses, and reduplicated Argon onto it. Having a third copy on an idle drive made me feel much better, since Neon now hosts some active files, and I don't dare risk overwriting anything on it as the restoration process continues.

And it does continue.

So now I'm doing what I've been doing for weeks: hunting and deleting duplicate files. Perversely this involves doing what I was doing before the trouble started, since ensuring that each image has a timestamped filename and then consolidating multiple source folders into one is the best way to spot redundant copies. I also search for all of the -2, -3, -4, and -5 file names that Lightroom creates, being careful not to step on scans or externally edited files that also use that suffix format. And then I review them, scrolling eternally, to make sure that each one that's flagged for deletion has a matching original file that Lightroom can find – or that it's something I won't miss. Between duplicates, complete dreck, and  other obvious candidates for deletion, I've taken out about fifty thousand photos.

Half-way there, more or less.

Most of my missing photos have been found through sheer strength of duplication, so that's very good. There are still a few thousand that I can't locate, some of which I liked, but the most important ones are all safe. And soon it should all fit on one drive again, which will let me get on with splitting up my library, which was what I've been trying to do all along.

So, what have I learned from all this?

I have learned two things.

One, have multiple live backups. Two, have an inert offline storehouse of my most important files.

Step one is in progress. Once my new system is in place I'll have two backups for my main data drive, Argon: the larger Neon with its substantial overflow room will remain my daily backup and protects against human error, letting me go back and find photos that I might have accidentally deleted from my active drive. The smaller backup disk Osmium, which matches the capacity of Argon, becomes my Last Known Good clone that I manually set from time to time. This is the same process that I use for my system disk, Xenon, except that I have two daily backups of it on different spinning drives as well as a periodic LKG clone. Those three volumes are Radium, Rhodium, and Ruthenium, respectively.

All of my important photos will continue to live on Argon and its two backups, while the filler gets demoted to the slow but inherently redundant Krypton raid. I'm not backing that one up. That's risky – I call keeping a single copy of a file "passive deletion" – but Krypton has lived through a couple of individual drive failures without any data loss already, and it's easily expanded as the digital kipple increases.

The second thing that I want – a redundant offline copy of the comparatively small amount of data that actually matters – is a bit more tricky, but back when Argon failed it would have made me very happy to know that all of my wedding photos were safe. This can be done by exporting all photos above a certain rating as tiffs and then storing them on surplus SD cards or smaller hard drives… and I know it works because I tried it once, but never quite made it into a habit. So that has to change, and ideally do it with automation.

Believe it or not, small safety deposit boxes that can fit hard drives are quite cheap to rent. Maybe I'll finally get around to off-site storage, too.

Powered by SmugMug Log In