# My Basic & Practical Back-Up Strategy



## mackguyver (Aug 7, 2014)

Here's an overview of my rather basic back up strategy that I recently discussed with another member over PM. I used to manage datacenter projects for a major financial company so I'm well aware of more sophisticated storage/backup tools & technologies, but after trying a number of other methods like DVDs (too small), Blu-rays (too expensive for Dual-layer), and online back up services (not practical given my 1.5MBps upload speeds), I have come up with my own strategy that's pretty simple and quite practical. 

The key to my strategy is a focus on my preserving best work (as rated during import/edit) and accepting that I _could _lose some of the other files that are really just outtakes from each shoot.

1. *Import & Edit*: When I import my photos, I rate them and then process them. I also delete the blurry and obviously bad photos to save drive space. 

2. *Output*: The final photos are output in TIFF and 100% quality JPEGs, and sometimes as PSD files for composites or sophisticated edits. 

3. *Upload*: I upload the JPEGs to my website where they are stored as 100% JPEGs in the Amazon Cloud

4. *Local Backup*: I copy the CR2, TIFF, and JPEG files to a portable hard drive that goes into a fire safe. 

5. *Offsite Backup*: Every so often, I copy it to another drive that I take to a safety deposit box. Every year or so, I buy a new hard drive, usually doubling my capacity. 

6. *Local Storage*: When I get the new hard drive, I copy all of the photos to it, and then store the old drive in the bank. This allows me to have some protection for all of my photos, and really good protection for my best work. 

By only saving the RAW files, and final edits of my best work, I can get away with using small portable drives (WD Passports) for years, while keeping all of my work at my fingertips on the local drives that are backed up less frequently.

I have considered NAS and other tools, but I don't like the idea of storing everything locally on my network and have heard negative stories of people who thought their RAID would save them, but didn't when two or more drives failed at once. It doesn't seem like home Network Attached Storage (NAS) systems have reached the level of commercial Storage Array Network (SAN) devices, at least without spending a lot of money. That said, I have 3 hard drives in my machine at the moment, two of which are just for photos. Also, I don't do video.

It's not bulletproof, but it's worked well for me a number of years and I have found that Western Digital drives (and the former HGST drives, now owned by WD) are the most reliable.

I welcome the onslaught of inevitable techno-debates over this post ;D


----------



## tolusina (Aug 7, 2014)

No debate, just a slick bit of simple hardware to ease many SSD or HDD back up solutions.

You might like this for desktop use.
Fits in an empty 5.25" optical drive size bay.
Accepts 2.5" and 3.5" SSD or HDD drives, one of each at the same time, up to a SATA 6 interface. Has two additional USB 3 ports. There's no active electronic components to it, essentially just wires circuit traces and connectors between motherboard headers and whatever devices.
Slot in a bare drive, first time format it, partitions if and as you like, from then on SSDs and HDDs can be used pretty much like floppies (remember those?) only FAST and BIG. 
It has a power switch, do power it off when inserting or removing drives.






Newegg page.


----------



## mackguyver (Aug 7, 2014)

tolusina, I like that - it would probably be a little prettier than my current tool that I use to copy disks, which with it's FAST eSATA connection has served me well for the last 6 or so years:

Thermaltake BlacX ST0005U External Hard Drive SATA Enclosure Docking Station 2.5" & 3.5" USB 2.0 & eSATA 




http://www.newegg.com/Product/Product.aspx?Item=N82E16817153071

P.S. Do you remember punching holes in 5.25" disks to make them double-sided? Or how about ZIP disks?


----------



## RustyTheGeek (Aug 7, 2014)

I'm in IT as well and I think *mackguyver* has described a backup scenario similar to what I have set up for some clients as well as myself. It's a bit manual and it's not exactly the same as mine but it's close and in the same spirit of redundancy and off site storage. The best backup is the one you use, which offers some redundancy and most important that you understand. In addition, if part of the backup plan is automatic and requires less attention and discipline, it will likely happen with more regularity. (Because we're all inherently lazy, myself included.)


----------



## Lloyd (Aug 7, 2014)

My approach is similar. I have my photos stored in a Pegasus raid array attached via thunderbolt to my mac and this array is backed up to one of two high capacity portable thunderbolt drives using Carbon Copy Cloner which makes a daily clone of the raid drives on the attached portable drive. I rotate the portable drives with one on site and the other off site. Unfortunately, my Pegasus has much more capacity that the individual portable drives and as I start to increase the data on the Pegasus, I will either have to do a major house cleaning or have to get larger portable drives or multiple portable drives and divide the backed up data between them.


----------



## neuroanatomist (Aug 7, 2014)

My backup needs are comparatively minor. I have <300 GB of RAW images, not that much by today's standards. I got a deal on LaCie 1 TB Rugged drives (bus-powered, FW800 and USB3), so I bought a stack of them. Backup my internal 960 GB SSD to a pair of them, and just the RAW images to another pair. One of each set is at home, the other at work.


----------



## RustyTheGeek (Aug 7, 2014)

neuroanatomist said:


> My backup needs are comparatively minor. I have <300 GB of RAW images, not that much by today's standards. I got a deal on LaCie 1 TB Rugged drives (bus-powered, FW800 and USB3), so I bought a stack of them. Backup my internal 960 GB SSD to a pair of them, and just the RAW images to another pair. One of each set is at home, the other at work.



Sounds good!

<300 GB... ahhh, that would be sooo sweet. With everything I've done over the last 5 years I am starting on my 3rd 2TB RAID1 volume. (Yes, that's 4TB full + beginning another 2TB.) And yeah, I could do some cleaning. And you probably delete a lot more than I do.


----------



## neuroanatomist (Aug 8, 2014)

RustyTheGeek said:


> And you probably delete a lot more than I do.



Yes, I triage pretty heavily...in large part due to having a camera that shoots 12 fps bursts. I tend to delete anything that's not a true 'keeper'.


----------



## RustyTheGeek (Aug 8, 2014)

neuroanatomist said:


> RustyTheGeek said:
> 
> 
> > And you probably delete a lot more than I do.
> ...



Yes, I'll BET! 12 fps burst groups that probably contain multiple large RAW ~25 MB files. I can imagine... bzzzt! bzzzt! Avg 300MB each group perhaps, depending on how many frames fired off? Wow.


----------



## Halfrack (Aug 8, 2014)

Zip, Jaz, Superdisk (120mb floppy disk from 3M), Doublespace v Drivespace lawsuit (Microsoft DOS 6.2/.21/.22)

Damn you Mack, I feel old now...

So there are a few things I'll toss in on:

- Fire safe - data rated fire safes are around that have a USB bass thru, but most importantly are going to keep the contents at 120 degrees rather than normal fire safes that go up to 400 degrees (when paper burns).

- I'm up to almost 4tb of data, and it keeps getting more fun. I don't 'clone' due to a bad file on the source drive becomes a bad file on the 'clone'. I'm using a sync product (chronosync personally, but there are others) for the reason I can tell it to 'archive' anything that is 'deleted' or 'changed' on the source drive. Drive space is cheap, and I can grab anything back from the archive, no software needed.

- Single drives - if you have to pay someone like DriveSavers or Ontrack to recover data, it's much cheaper for a single drive compared to a RAID set

- To NAS or DAS otherwise - 4 most important things for a NAS are 
1) setup alerting and test it 
2) back it up to something else (and send you a task completed email)
3) have a spare drive to put in asap and 
4) have a recovery plan
Had lots of fun recovering an old ReadyNAS - back when they used Sparc CPUs, now they've done x86 and ARM as well - thankfully a client had an old one that happened to match up otherwise SOL. Current favorite is Synology since I can recover the disks attached to any pc booting off a live-cd. http://www.synology.com/en-us/support/faq/579


----------



## RustyTheGeek (Aug 8, 2014)

I'm considering Synology eventually *Halfrack*.

I only use RAID 1 for the reasons you mention. I don't need higher RAID.

Love the blast from the past! Good old iomega external drives in Parallel and SCSI flavors! I also had JAZ drives, internal and external. (Probably still do somewhere around here!) Also, don't forget about the old OnTrack hard drive emulation software when the BIOS wouldn't recognize hard drives larger than 137 GB! Wasn't that fun?

I have a couple of safes, both fire rated. My larger safe is where I keep important papers, guns, silver, etc plus a backup drive on a network cable.

I recommend these Sturdy Gun Safes hand built in California. Truly designed to withstand a long fire.
http://www.sturdysafe.com/pages/sturdy-safe-fire-liner-info


----------



## mackguyver (Aug 8, 2014)

Halfrack said:


> Zip, Jaz, Superdisk (120mb floppy disk from 3M), Doublespace v Drivespace lawsuit (Microsoft DOS 6.2/.21/.22)
> 
> Damn you Mack, I feel old now...


LOL, and remember the _click of death_ with Zipdisks? That did it for me, but I loved them up till that point. Also, my first PC used analog cassette tapes to transfer data


----------



## RustyTheGeek (Aug 9, 2014)

mackguyver said:


> Halfrack said:
> 
> 
> > Zip, Jaz, Superdisk (120mb floppy disk from 3M), Doublespace v Drivespace lawsuit (Microsoft DOS 6.2/.21/.22)
> ...



TRS-80?


----------



## neuroanatomist (Aug 9, 2014)

Halfrack said:


> Damn you Mack, I feel old now...



Anyone else remember using a hole punch to make single-sided floppy disks into double-sided floppy disks (back when they were actually big and floppy)?


----------



## Dylan777 (Aug 9, 2014)

For me, x3 5TB external HDs with USB3 speed. Keep two at home, always up to date. One at grandma house with monthly backup. 99% is photos.


----------



## FTb-n (Aug 9, 2014)

neuroanatomist said:


> Anyone else remember using a hole punch to make single-sided floppy disks into double-sided floppy disks (back when they were actually big and floppy)?



I do. I recently found one of those floppy floppies in an old box of stuff. Too bad I no longer have that fancy dual floppy Columbia PC clone.


----------



## RustyTheGeek (Aug 9, 2014)

FTb-n said:


> neuroanatomist said:
> 
> 
> > Anyone else remember using a hole punch to make single-sided floppy disks into double-sided floppy disks (back when they were actually big and floppy)?
> ...



Ditto! I think I still have a 5 1/4" floppy drive around somewhere. I have a couple of Seagate MFM/RLL 20MB full height hard drives upstairs to show people for fun. (Along with a ton of other artifacts that many folks these days probably wouldn't ever see.)


----------



## ray5 (Aug 9, 2014)

My mind is numb now. I am totally confused. Can someone please explain what's the difference between a backup and a clone? When I back up my laptop using time machine onto a external drive, the information on that drive is much smaller than what's on my laptop. Is the information compressed? Does it have everything? i.e if my laptop were to crash and loose everything, can a restore ALL information, pictures, documents from the backup?
Clone, to me sounds like an exact copy, is it uncompressed?
Is one better/safer as a backup than the other?
Thanks


----------



## RustyTheGeek (Aug 10, 2014)

ray5 said:


> My mind is numb now. I am totally confused. Can someone please explain what's the difference between a backup and a clone? When I back up my laptop using time machine onto a external drive, the information on that drive is much smaller than what's on my laptop. Is the information compressed? Does it have everything? i.e if my laptop were to crash and loose everything, can a restore ALL information, pictures, documents from the backup?
> Clone, to me sounds like an exact copy, is it uncompressed?
> Is one better/safer as a backup than the other?
> Thanks



Good questions. Lots of terminology that can sound confusing.

Forget terminology for a sec. There are several ways to back up data. In essence there are two methods - file based backups and image based backups. An image based backup, whatever it's called, is typically a single compressed file that contains the entire drive including partition information, boot sector information, etc and can be used to recover a blank hard drive back to a bootable state. File based backups usually only contain the files that the backup was configured to back up. Often there are multiple versions of the file in case an older version is needed. They are usually much smaller since they only contain chosen data files. Another type of file backup is sync based backups like DropBox, etc.

I hope this helps clear things up a bit. A 'clone' is usually an exact copy of something and often refers to a drive image that can be used to create a 'clone' (exact copy) of a drive that contains the entire operating system, etc. Once the recovered drive is booted up, since it is an identical 'clone' of the original system, it will be just as it was at the time of the 'snapshot' 'cloning' of the drive.

It's important to understand how your backup works so you know what is protected, for how long and how to get it back if necessary.

Lastly, I think RAID Arrays may have been mentioned. These are not backups, they are just more than one drive used to host a volume of data that acts like one logical drive but is actually made up of multiple physical drives in case one drive dies, the logical volume is maintained. The data files are not protected from damage, only the existence of them is. If one is corrupted or deleted, it's still gone unless it can be recovered from a backup. That's why RAID isn't called backup, it's called Fault Tolerance.


----------



## Halfrack (Aug 10, 2014)

neuroanatomist said:


> Halfrack said:
> 
> 
> > Damn you Mack, I feel old now...
> ...



That only worked on 5 1/4" floppy disks - the 8" floppy disks were truely one sided. Cassette tapes for program loading? Commodore 64/128 or Leading Edge Model D with an 8086 chip running DOS 2.11. Ah click of death - what a painful way to die... I actually used to carry around the parallel ZIP drive back when I was working tech support, Windows 98 fit on a single disk. Remember SyQuest also had their 'EZ Drive'?

To add more knowledge, a "clone" type backup is more inline with the included backup tools in Windows7/8 and Mac (Time Machine). File based backup is like what Carbonite/Backblaze/iDrive offer - where you select your directories and they back them up.

File backup is better for photos and documents, while Image backup is better for programs and personal settings. Thankfully you can stack your backup options - especially when you have multiple drives. Use a clone or image based backup for your c:\ drive, while your photos out on e:\ are backed up using a file based technology.


----------



## mackguyver (Aug 11, 2014)

Halfrack said:


> neuroanatomist said:
> 
> 
> > Halfrack said:
> ...


Yes, lots of memories there, and I didn't mention cloning, but I use Acronis True Image for that, and have for many years. It's saved my work many times and is a great program.


----------



## AcutancePhotography (Aug 11, 2014)

I remember when DEC had their single sided quad density 5.25" disks. Pretty much making sure there would be no compatability with any other computer system of the day. :-\


----------



## lexptr (Aug 13, 2014)

Thanks for the overview! I do similar backups in general, but with some differences in particular 

1. Same. But I don't rate them, just choose what is worth to process and publish and put in subfolder.

2. I output only chosen content for publishing. The rest is left as CR2.

3. Upload to the web (site/blog/Facebook) downsized jpegs, but not a 100% quality. I have tested different settings and found that there is no really visible difference between 100% and something between 96-97% but there is a big difference in file size, which is still important for web these days. So I use 97%.

4. I backup on 2,5" external drives. They are small and don't require separate power connector. I usually backup only CR2, as I don't see notable profits from saving additional formats while much more space is required for that. It is always obtainable from the original CR2s. And it will be even exactly the same, because I do save Lightroom's XMP files from processing. In some cases, if I did a complicated PP in Photoshop – I do save a PSD files too. I don't use a fire safe. Will it really help in case of fire? I believe the fire won't enter, but I think disks will just melt inside (their plastic parts at least).

5. Well, shame on me, but I still did not settled offsite backup. I plan to do it in the nearest future. House of my mother-in-law will be used as a deposit box 

6. Old hard drive in bank? An overkill level of redundancy for me 

Some additional safety measures I use: I do not delete images from CF cards until I have a double backup on hard drives.


----------



## wickidwombat (Aug 14, 2014)

are there any raid NAS devices that can be configured to replicate with a like device over the internet?
i haven't been able to find anything


----------



## nvsravank (Aug 14, 2014)

I use some thing similar I use Raid 1 as my working drive and main storage, a Drobo as my onsite backup and a seagate Goflex as my off site backup. I keep both the lightroom catalog as well as the raw files on the backups. I exclude the previews folder from the backups. 

I have set Drobo to do backup every hour so i am very current with my backups and the seagate external i do it everytime i have an event that i covered.

While this works well for regular safety, I am getting more and more concerned abut data corruption at rest for my personal photos. Any consumer oriented products that protect from data at rest errors? I believe some Raid 6 solutions can protect from data at rest errors? I know Raid 1 cant protect since the system wont know which copy is correct if there is a conflict.

At work i know the EMC centerra etc do periodic checks for data at rest corruptions. Any consumer oriented solutions out there? 

I keep reading about ZFS and have read somewhere that it does have such support, but other than a roll your own solution i dont see commercial solutions for ZFS targeted to consumers. Any ideas?


----------

