Over the last half of 2016 dealing with hard drive failures and data protection has been a recurring theme in my life. I’ve written about it previously, but think the subject deserves a retrospective, creating my first non-photographic project for this site. If you want the full process of discovery, the original posts can still be found at these links:
What I offer here is my current process to protect my photographic files. I'm not an expert on the subject, but this is what I do as a practical system, and so far it works.
My three principles of data protection:
1 - Memory cards are cheap.
2 - Hard drives are cheap.
3 - USB drives are cheap.
While the implementation is a bit more involved than that, it really is fairly easy. The goal is simply to minimize the significance of the inevitable data problems, and make the recovery as graceful as possible. So don’t erase memory cards. Have back up hard drives. Export the most important images as finished files and keep them somewhere else.
The Toronto Transit Commission’s first production Canadian Light Rail Vehicle, number 4010, boards passengers while running southbound on Spadina Avenue in February 2013. Now forty years old, CLRVs no longer run on the 510 Spadina route as the TTC phases in new vehicles across the city.
While storms of this size used to happen several times each winter, for the past several years they have been fairly rare events. Photographed with the indefatigable and weather-sealed Olympus E-1 and 50mm macro lens.
Memory cards are cheap. They’re also very stable, nearly indestructible, and easy to store. I used to keep a pool of memory cards that I’d reformat and reuse in different cameras, but no more. Now every camera has its own dedicated card, labelled with a metallic Sharpie, creating an immediate and unmodified archive of every image. When everything else fails – and it has – this is what I can go back to.
By safely keeping all of my original files the memory cards are my best archive of last resort. They protect against catastrophe, creeping bit rot, and the most nefarious and difficult of all flaws: human error. If you do nothing else, do this. But it doesn’t protect any of the post-processing work, so do more, as well.
Underneath the Bathurst Street Bridge, October 2004. Once a refuge for those without any other shelter in a quiet corner of Toronto’s lakefront, this photo was taken during the time when the residents were being forcibly evicted. Over the following decade the character of this part of the city has completely changed, and it is now home to condominium towers in the new Fort York neighbourhood.
Taken with my first ‘good’ camera, a Sony F828, this is both one of my earliest infrared photos as well as rare evidence that I’ve ever held a beer bottle. This photo only exists now as an edited tiff; the original camera raw file, like almost all of my photographs from this time period, was lost to a hard drive failure.
Hard drives are cheap. There’s just no reason not to have a backup copy of everything, and ideally more than that. My system uses tiers of hard drives that decrease in speed and increase in size as the data becomes older and less important. The system drive that’s the most important and changes data often has three different backups, while my main storage has two and the kipple drive has one.
This may be excessive, but with multiple backup drives comes the option to back up at different times. My setup is to automatically run some backups daily and others weekly, while my system drive also gets a special “last known good” backup that I update whenever things have been stable for a while and before I make any operating system changes. I do also use Apple’s Time Machine, but I’ve never found it to be useful for anything at all.
And the reason why I like doing backups at different times? The data problems I ran into last year were because of bad sectors on my main data drive, which I only found out about after it had corrupted the backup data. Two copies made at different times might have made my life a lot easier. Maybe.
New York City road crews cut through the pavement without closing any lanes to traffic. Drivers would simply wait until the massive saw was out of their lane and then proceed through. This photo was taken on the afternoon of October 27 2012 with my Canon S100 compact, which was the camera that taught me how useful built-in neutral density filter are, and how much I enjoy daytime long exposure photographs.
October 27, 2012, was just two days before Hurricane Sandy reached the city. This image was the last photo my last digital Canon camera took before it fell victim to a pervasive manufacturing fault; it would have been nice to have a pocket camera during the city’s final day of storm preparations and my late-evening escape through the jammed Port Authority bus terminal. The S100 was back from Canon Service three weeks later, but I never forgave it. After a few moths of ostracization it was replaced by the Ricoh GRDIV.
USB drives are cheap. They’re also portable, abusable, and even if they need adapters, are a format that is likely to be supported for many, many years. They’re what I use for offline and offsite storage for the small percentage of my photos that I really care about. When a spinning drive starts the click of death it’s good to know that there are other copies of the important stuff on reliable media that’s somewhere else.
I have a pair of encrypted USB drives that hold the archive of my most important photos. One stays at home within arm’s reach, where it reminds me to update it frequently as I generate new images that meet the protection criteria. The other USB drive is cleverly hidden where I work, and every few weeks I swap them. My better photos are written as jpegs or tiff files depending on their level of significance, which absolutely includes the wedding and family photos that are so personally important. This alone drastically reduces the likelihood that I’ll ever need to run back into a burning building.
I’m a big fan of star ratings, so I use them as the selection criteria for Lightroom’s ‘Publish to Hard Drive’ service. Crucially, the exported photos that Lightroom creates through this process need to be moved out of the folder where they land so that Lightroom can’t delete them from the drive if they’re ever removed from the published collection or otherwise accidentally deleted from Lightroom. Add that to the list of things that I learned the hard way, and it brings me full circle back to why I stopped reformatting my memory cards.
Former Toronto mayor Rob Ford addresses city council after he had admitted to smoking crack cocaine. Taken in November 2013, this meeting was held to strip him of all powers that he was not required to have by law. A headstrong populist, Ford powered through numerous scandals and transgressions that would have seen most politicians step down months or years earlier.
Taken with my Ricoh GR, this is the only photo out of 145 images from that day that earned the two-star rating that has it included as a jpeg file in my USB backup layer. While I don’t really want to see the other dozen-dozen photos disappear if the worst happens, it’s also true that I wouldn’t really miss them.
So that’s it: don’t erase the camera’s memory cards, have hard drive backups, and export the few actually important images as finished files and keep them somewhere else.
Three steps. Easy.
If all goes well the inevitable future data failures will be insignificant and handled with grace and aplomb.
Time will tell.