# Backup to Blu-ray



## FTb-n (Jan 2, 2015)

Anyone writing images to Blu-ray discs as part of you backup strategy? If so, which size disc?

I currently backup to multiple external hard drives. Previously, before 25+MB RAW images, I also burned images to DVD. A couple years ago I started burning to 25 GB Blu-ray, but I've fallen behind. I've also had write issues with some discs failing. Grouping images to fit 25 GB discs can also be a pain. It's now cheaper and easier to get another external drive, like the WD My Passport to use as a backup. But, will a hard drive only strategy be reliable in the long run?

There is something about write-once media without moving parts (like Blu-ray discs) that intuitively seems to be more reliable for long-term storage than moving-platter-based media. But, I've had issues with burning discs on one drive that can't be read on another (even after finalizing them). This leaves me with the fear that I could burn a bunch of discs on a drive that might drift out alignment only to find out years later that I can't read them.

I would like to incorporate the cloud, but last year (2014), I accumulated over 33,000 RAW images that consume roughly 950 GB. A TB/year can of data on the cloud can get expensive quick.


----------



## zim (Jan 2, 2015)

A cloud (hate that word now :. ) that doesn't limit you by GB would suit?

https://www.code42.com 190 for four years

https://www.vembu.com, worth a look too edit, they charge per gb

Regards


----------



## tpatana (Jan 2, 2015)

Just checked, I've shot 1.85TB during 2014.

No way I'm burning those to discs.

I use 3TB drives, and double those. So after editing on my faster SSD, I copy everything on 2 separate external drives.

Currently 3TB drives are less than $100 each, so <$200 investment takes me maybe 1.5 years. Not bad.

There's several online backups with flat fee, so in theory they'd be cheaper but uploading everything would be impossible task.


----------



## martti (Jan 2, 2015)

http://www.blu-raydisc.com/en/aboutblu-ray/whatisblu-raydisc/bdkeycharacteristics.aspx

According to the site, the technology will be relevant for at least 10-15 years from now.
As the discs have not been around for very long so it is not known how well they hold their information.
An estimate I found was 100 to 150 years. That's cool. Currently I am using hard disks. 
How long does it take to write a 25GB BlueRay disc? 

Cloud is not an option here with the current Net bandwidth. 
And what if North Korea hacks the cloud server like they (supposedly) did with Sony? Or worse still, the Lezard Squad...


----------



## tolusina (Jan 2, 2015)

For optical archiving, there's M-Disc...
http://www.mdisc.com/
Um, even with 25GB Blu-Ray discs, you'll need 40 of them for 1 TB.

For real simple HDD or SSD backups in a tower, the drive bay shown below takes any SATA interface 2.5" or/and a 3.5" drive in a slot (two at a time, one each size possible), no other enclosure needed.
Simply pop a drive in, switch on the bay's power, drive shows up as a removable USB 3 device.
http://www.newegg.com/Product/Product.aspx?Item=N82E16817998185

A couple reviewers had issues with drives not being recognized, ignore those reviewers. There are no active components anywhere in the device, it has just straight through connections from the device to the motherboard's USB headers. Those reviewers had other issues that they blamed on the bay.










.


----------



## Marsu42 (Jan 2, 2015)

No joke: If you want to preserve information, carve it into stone tablets or write it on paryrus scrolls (the ones from ancient egypt are still around because they don't contain acid as modern paper). Alas, digital image data doesn't fit on them :-\



martti said:


> An estimate I found was 100 to 150 years. That's cool.



Right. And pigs can fly :->



martti said:


> Currently I am using hard disks.



Imho the best strategy for us laymen, you just have to spin 'em up now and again - and to make sure, you actually have to *re-write* the data sooner or later to refresh the magnetic information.

If you have too much data, there's always the option for *lossy* archival (like lossy dng, jpeg, ...) for non-critical data. I know archival experts will cringe at the though, but do you really need a 1:1 raw backup of all your doggy shots as the original storage space isn't very likely to fail in the first place?



martti said:


> And what if North Korea hacks the cloud server like they (supposedly) did with Sony? Or worse still, the Lezard Squad...



Those who have nothing to hide have nothing to fear


----------



## martti (Jan 2, 2015)

Marsu42: Those who have nothing to hide have nothing to fear 
Was it Michael Hayden who said that? The "Drown them a bit"-Hayden?

Did you know that 'marsu' means 'guinea pig' in Finnish? A misunderstanding from the German word 'marschwein'.

Apple's TimeCapsule gave be very hard time and my daughter, too. That was a mechanical issue with overheating and short circuit. It is possible that they solved the problem but no thank you. I do TimeMachine BU's after every upload and before updates and every once in a while a disc image with SuperDuper on a different disc. 
I would love to have a WiFi backup disc but my IP cuts the connection several times a day. Yes, the WiFi, too.
Their excuses are funny. Ha-ha.

You pointed out the millions of doggyshots that just take up space. It takes time to only keep the keepers. 
I am too lazy. Getting a bit better, though.


----------



## Steve Balcombe (Jan 2, 2015)

The number one golden rule of backup/archive (no real difference between the two here) is it must be *EASY* so you will actually do it promptly and regularly. One of the best decisions I ever made was to invest in a good NAS box - http://www.amazon.co.uk/Synology-DS413j-DiskStation-Desktop-Server/dp/B0095RYB36 but that was a while ago, there may be a better choice now. I fitted two 3 TB drives in a RAID 1 (mirror) configuration so it is as proof against hardware failure as I can make it. At some point I will fit a further two drives to give me more capacity. 

My normal routine is to download files from camera to local hard drive (also RAID 1); delete any obvious dross then backup to the NAS. Only at that point, when I have two secure copies, do I format my memory cards.


----------



## Marsu42 (Jan 2, 2015)

Steve Balcombe said:


> The number one golden rule of backup/archive (no real difference between the two here) is it must be *EASY* so you will actually do it promptly and regularly.



Excellent point!



martti said:


> Was it Michael Hayden who said that? The "Drown them a bit"-Hayden?



Unfortunately, there are a lot of people who've said it, at least in one way or the other. But the real problem is that there are no ends of the "silent majority" (Nixon) who *think* it. Well, imho the latest hacks of Sony and celebs might do some good here - even though the outcome probably only is "Never say anything controversial or put any private data online".



martti said:


> You pointed out the millions of doggyshots that just take up space. It takes time to only keep the keepers.
> I am too lazy. Getting a bit better, though.



Same here, sorting through the shots takes longer than postprocessing and shooting itself. My strategy is to "whitelist" the definite keepers, then archive the rest as lossy dng in case I have to pull them out against expectations. The decision to blacklist and delete shots forever makes it harder for me to decide.



martti said:


> Did you know that 'marsu' means 'guinea pig' in Finnish? A misunderstanding from the German word 'marschwein'.



Interesting, esp. es "Meerschwein" (German) is a mis-conception in the first place - obviously guinea pigs are a completely different class of animal than the "oink, oink" pigs. But guinea pig fits perfectly - I've got several, two of them are right next to me on my desk, sleeping in their little houses 

As for Marsu: It's from a comic book, I just happened to change my avatar atm.


----------



## danski0224 (Jan 2, 2015)

Are CF cards a viable option for long term file storage?


----------



## dppaskewitz (Jan 2, 2015)

So far, I've been downloading to an external hard drive (I use a laptop as my main computer and don't have enough room on it for anything except my LR catalog), burn to a DVD (and more recently, BluRay) then reformat the camera card. I have a second external hard drive. Once a day, drive one synchs with drive two (any changes on drive one are repeated on drive two, whether new CR2s, changes to XMPs, deletions (from culling), etc.). The point is that I should be reasonably secure from a mechanical failure of one hard drive. And at least in theory I have a back up of the original CR2s and the initial XMPs on DVD or BluRay - so that if there is data corruption on the hard drives (and any corruption would be copied from drive one to drive two as I understand it), at least I could restore originals from DVD or BluRay. Obviously, any post processing of a corrupted file would likely be lost. My LR catalog is in my Dropbox, so is stored on the cloud as well as on my internal hard drive. And drive one is a portable, so I can take it all with me when I travel.

Holes that I see in my system: (a) loss of all drives and DVDs/BluRays from, for example, fire; (b) no "future proofing" (archiving so that my kids/grandkids can see my photos in 30 years (when I am likely not around to continue upgrading to new hardware) - not that they care; (c) the aforementioned lack of protection of post processed files from data corruption. 

I guess the points are: in the long run, storing on multiple HDDs seems to be the way to go, because they can be kept current with most recent changes. Cloud storage would solve the fire/earthquake issue, but at what cost (even as an amateur, I have over 1 TB of old dog photos) and convenience (with a fast connection, it would still take shortly less than forever to initially put everything into the cloud). Is there some way to easily (regularly and automatically) check for data corruption?


----------



## Joe M (Jan 2, 2015)

I gave up writing to disc years ago. A combination of factors from the increase in the amount of data I need to backup to the disc write times (and number of discs needed) to the plummeting cost of external hard drives has me simply writing to multiple externals. I was intrigued but only momentarily by the MDisc. The longevity claims are interesting but I would quickly fill a room with stacks of them. And the burn times...yeesh. Might be practical for someone backing up the occasional couple of gigs of stuff but not when you have tbs of stuff. 
Bang for the buck these days, no matter if you have a little info to backup or tons is externals drives. If you want to get paranoid, make sure you have a few, store some in other locations if you can or buy fireproof/waterproof ones. 
I like the dock that tolusina posted. I have an older external Thermaltake dock for hds and use that for casual personal backups on various drives too. I think my next tower build will include that dock. With the price of SSDs dropping too, that will likely become the practical backup solution to simply pop in one of those and backup a ton of stuff in seconds.


----------



## Dylan777 (Jan 2, 2015)

USB3 external HHDs are getting cheaper everyday. I'm using three 5TB USB3 as backup. Two at home(always up to date) and one at grandma house(monthly backup).

SSD will get cheaper. That will be my next option.


----------



## EchoLocation (Jan 2, 2015)

I'm a big music fan and archived a bunch of concerts(maybe 500cds) on the highest quality cd's I could buy(at a reasonable price, maybe $30 for a 50pack) about ten years ago...sony
I made double copies and kept the second copies in jewel cases in a box in a cool, dry closet. I never played them, or touched them.
ten years later, almost half of them have lost their sony paint on the top side and are now basically clear, and totally unplayable. I'm super pissed.
don't believe that cd's, dvd's or blu ray will last 100 years. that is 100% BS.
I now have triple backups of everything on different external hard drives in different locations(one at home, one at my mom's house, etc)


----------



## hgraf (Jan 2, 2015)

martti said:


> http://www.blu-raydisc.com/en/aboutblu-ray/whatisblu-raydisc/bdkeycharacteristics.aspx
> 
> According to the site, the technology will be relevant for at least 10-15 years from now.
> As the discs have not been around for very long so it is not known how well they hold their information.
> An estimate I found was 100 to 150 years. That's cool. Currently I am using hard disks.



I'm not sure you can trust what that site says, it appears to be a marketing type site. My personal opinion of marketing has always been: lie as much as you can without getting into too much trouble.

I can't speak for the longevity of BD. But I do have data from previous generations of optical media, and saying those discs will work 100-150 years from now is pure hilarity.

In my experience, user writeable optical media has a reliable life span of AT MOST 1 year. I've got discs that lasted longer then that, some much longer, but NONE have lasted longer then 10 years (CDRs), and most haven't lasted longer then about 3 years for CDRs and ~1.5 years for DVDRs.

The reason is the chemicals used in user writeable optical media isn't very stable. Since it has to react to light to change state, it means long term stability is an issue. It's just a consequence of the technology.

Again, I can't speak for BluRay, my oldest BluRay disk is about 2 years old, it's still readable, but I don't hold up much hope.

Note that PRESSED optical media is a different beast altogether, aside from some issues with early CDs (the aluminium oxidized, google "CD rot") they will I believe last many decades. I have commercially pressed CDs that are 30 years and still play fine, and commercial DVDs that are maybe ~10+ years old play fine as well.

Personally, the only viable backup media at the moment is multiple harddrives. Replace them every few years and you'll be good to go.

TTYL


----------



## LDS (Jan 2, 2015)

tolusina said:


> For optical archiving, there's M-Disc...
> http://www.mdisc.com/
> Um, even with 25GB Blu-Ray discs, you'll need 40 of them for 1 TB.



I'm going to give M-Disc a try, but only for those image subset I can't really afford to lose. I'm not going to backup everything on them, because they still require more storage space than TBs sized hard disks.
IMHO a sensible backup strategy doesn't threat each file equal - but you need to assess which ones are more important and need better, although more expensive protection, and which don't.


----------



## Mt Spokane Photography (Jan 2, 2015)

EchoLocation said:


> I'm a big music fan and archived a bunch of concerts(maybe 500cds) on the highest quality cd's I could buy(at a reasonable price, maybe $30 for a 50pack) about ten years ago...sony
> I made double copies and kept the second copies in jewel cases in a box in a cool, dry closet. I never played them, or touched them.
> ten years later, almost half of them have lost their sony paint on the top side and are now basically clear, and totally unplayable. I'm super pissed.
> don't believe that cd's, dvd's or blu ray will last 100 years. that is 100% BS.
> I now have triple backups of everything on different external hard drives in different locations(one at home, one at my mom's house, etc)


 
The early CD's often did not last a year, once the tech matured in about 2005, I've had no issues when using the best quality disks. 

Right now I use Verbatim disks, they seem to be very reliable.

However, Hard drives have been far more reliable than disks for me, even so, its necessary to copy things avery few years. I keep my backups on a NAS. That automatically keeps a backup, and it has the benefit of becoming obsolete over time, so I move things to a new NAS with new drives every few years.

I have a big box of floppies that were carefully stored in the 1980's. Many of these can no longer be read, but the program disks as opposed to home written data disks are mostly still good. I pulled out a very old Microsoft Dos 5.5 disk last year, and installed it to run some very old software. I had to buy a replacement power supply for my old computer in order to run it. (The power supply had died and I never bothered to fix it)


----------



## ajfotofilmagem (Jan 2, 2015)

I used the DVD-RAM media (rewritable professional standard) for more than 10 years, writing and rewriting again and again, and I know it really last many decades. Unfortunately, the high price made it uncompetitive, and today the market is most new readers incompatible with this type of disc.

I imagine the M-DISC have the same type of material, durable and resistant for passing years.


----------



## wockawocka (Jan 2, 2015)

Long term I use the Mileniatta 25gb Blu Rays (1000 year lifespan) - But still have HD's for fast access.


----------



## RustyTheGeek (Jan 2, 2015)

OK, IT Geek here. I'm just going to say this...

Burnable Media = DVD/CD/Blu-Ray/Whatever = NOT Long Term, the chemicals/dyes fade/die over the years.

Magnetic Media = Hard Drives ONLY = GREAT Long Term, the technology is mature and reliable over the years. NOT RAID Arrays. (RAID is fault tolerant working storage, not for long term archiving.) KISS. Simple is best. 2.5" Ext Drives, not large AC powered drives.

Pressed Media = CDs, etc that are actual metal media, not dye based = BEST Long Term, nothing to fade or corrupt as long as the physical medium, plastic, metal film, etc are not corrupted by heat, sun, or physical abuse like scratches, etc.

It all comes down to what the archive goals are. In general, I put everything on large external hard drives and store them in a cool, dark, fireproof place and check them every year or two. Refresh if necc. Create two copies if necc.

Read about how the big boys, National Archives, etc are facing this challenge and all the things they are trying to overcome. It's a fascinating topic.


----------



## rfdesigner (Jan 2, 2015)

EchoLocation said:


> I'm a big music fan and archived a bunch of concerts(maybe 500cds) on the highest quality cd's I could buy(at a reasonable price, maybe $30 for a 50pack) about ten years ago...sony
> I made double copies and kept the second copies in jewel cases in a box in a cool, dry closet. I never played them, or touched them.
> ten years later, almost half of them have lost their sony paint on the top side and are now basically clear, and totally unplayable. I'm super pissed.
> don't believe that cd's, dvd's or blu ray will last 100 years. that is 100% BS.
> I now have triple backups of everything on different external hard drives in different locations(one at home, one at my mom's house, etc)



When I ran my business I would do backups onto rewritable CDs weekly. Two different manufactuers disks copy of the backup on each type each week, swapping disks every week so I had 4 disks going with this weeks and last weeks backups. I wound up the business some years ago, and found the disks when unpacking when we moved into our new house a couple of years ago, about 7 years after writing the data.

All data was unrecoverable.

CD-ROMs might have been better, but I'm not sure they'd be good for a decade.

IMHO there are three ways.

1. Keep data in a current live format.. i.e. in immediately accessible backup drives, ensure your data is stored with one set in a fire safe or in an alternative building (what I'm setting up)

2. Print REALLY valuable data using best inks/dyes onto paper in 2D barcode style. last time I looked it was about 1Mbyte per A4 page, you should easily get 20 years from top of the line printing, probably far more with no digital degradation.

3. Print the images you really want to keep using top end inks/dyes and store carefully, as we always did with those couple of pictures of Great Great Grandpa... and those lasted >100 years.. this is what I'm starting to do for those family pictures we have so we don't suffer digital amnesia.


----------



## Lloyd (Jan 2, 2015)

rfdesigner said:


> 3. Print the images you really want to keep using top end inks/dyes and store carefully, as we always did with those couple of pictures of Great Great Grandpa... and those lasted >100 years.. this is what I'm starting to do for those family pictures we have so we don't suffer digital amnesia.


My backup procedure is to take my pictures of Great Great Grandpa and using layers in photoshop I paste in next to him a copy of a naked celebrity and post it on the internet. They tell me it lasts forever.


----------



## LookingThroughMyLens81 (Jan 2, 2015)

MDisc is the only way to go. For those of you complaining about it being 25GB, the next update to the MDisc line which comes this year will be BD-XL discs and they pack 100GB on single-sided discs.


----------



## Marsu42 (Jan 2, 2015)

PropeNonComposMentis said:


> Even just a few bits is bad enough.



Pure hashes are just there to *detect* changes, but there are recovery record schemes for data reconstruction when some parts are lost like par2 and rar. I used the latter for burning data to cdr, it served me well when some damage occurred. 



PropeNonComposMentis said:


> Analogue Storage is the only way.



+1 - I'm always thinking of those poor kids being burdened with terabyte-data of their childhood videoed and shot with high-mp cameras (containing nice bokeh, of course). Much better to print 'em or do vhs copies, at least they fade away over time and it gets less embarrassing 



LookingThroughMyLens81 said:


> MDisc is the only way to go. For those of you complaining about it being 25GB, the next update to the MDisc line which comes this year will be BD-XL discs and they pack 100GB on single-sided discs.



Somehow, simple /me doesn't buy the "one millenium" life span - that's the time of the collapse of the Maya civilization and when the Vikings first settled on American soil.


----------



## RustyTheGeek (Jan 2, 2015)

I agree with Analog as being superior. Nothing beats a few photo albums in the cabinet that are easy to pick up, look through and enjoy. Anything more becomes cumbersome at best and impossible at worst.

But our world is now digital. Most seldom print anything. So it all must be preserved somehow. Print a few, great! But wouldn't it be great if we still had the negatives to all those ancient photos from 100 years ago? Now we do as long as they aren't deleted they can live forever in a digital format as long as they are cared for and maintained.

Unfortunately the true reality is that there is now so much overwhelming data that it is a real possibility that most of what we have now will never be seen after the initial creation and will likely be deleted/discarded at some point if not by us then by someone else in the future. Sort of like a lot of the studio music tapes from the 50's, 60's and 70's of famous music artists. Tape was expensive and if the music wasn't making money or it wasn't a "keeper", studios would record over the earlier sessions because they had no desire or need to keep hundreds or thousands of tapes 'just because'.


----------



## ajfotofilmagem (Jan 2, 2015)

We have to face a hard fact:

The files that we store today in hard disc, or any other media that allows you to delete will be deleted someday.

Imagine that after his death, his children will seek the childhood photos of a million RAW files ... What are the chances of it find the file he seeks, and can view it and process it before printing?

Does anyone have the illusion that other people have the same care you have with your files?


----------



## RustyTheGeek (Jan 2, 2015)

ajfotofilmagem said:


> We have to face a hard fact:
> 
> The files that we store today in hard disc, or any other media that allows you to delete will be deleted someday.
> 
> ...



Yep. Pretty much. It's up to the 'person in charge'. At this time, that is likely you. Later, it will be someone else. If I am any indication, based on my personal OCD habits and 'save stuff' mentality, we're all ******* because as much as I hate to delete or toss things, I still don't have much (or care much) with regard to my parents' stuff. A few prized items that mean something to me but otherwise, nada.

So, I'm trying to force myself to realize deep down that 99.9% of my valuable stuff that I have worked hard to buy/acquire/care for/store/obsess over/worship/whatever will be gone in the dump or sold off in probably 20+ years. Little by little that is helping me "detach" from it since it's essentially gone at some point anyway whether I am here and in an institution or dead and buried it will all still be out of my hands and concern. :-\

That's pretty much it. Back in my teens, 20's and 30's I was going to live forever and I wanted to acquire stuff. Now that I'm in my 40's, my older relatives are dying off and I'm faced with dealing with their stuff, I'm realizing that it's all just crap that gets old and will end up in a garage sale, trash can or on eBay. :-\

Ashes to ashes and all that...

The nice thing about photography these days is that a few great photos have the potential to live on for someone who cherishes them. So in doing something that you love, you will brighten the life of someone else in the process. There aren't too many hobbies that work in that way. That's why I have so many images uploaded to zenfolio for everyone else in the scout troop, church and school to access. If those images mean something to others enough for them to download and share them, they might live on down the line for others that cherish them and all my efforts will not be in vain...


----------



## LDS (Jan 2, 2015)

Marsu42 said:


> Vikings first settled on American soil.



For which, unluckily, they didn't left any record able to reach us... bad media backup strategy theirs. Maya ones worked better


----------



## LDS (Jan 2, 2015)

PropeNonComposMentis said:


> 2.5" Ext Drives.



These are usually cheap drives designed for laptops, and not designed to last long (you can see it from their actual prices). There are drives designed for long term storage, but it's not those. They may last longer if you connect them just for backups, and then store them properly, but don't expect much. They are cheap, though. Anyway, even magnetic archiving will decay with time (flash memories included).

The quest for the definitive backup media continues....


----------



## epsiloneri (Jan 2, 2015)

There appears to be some confusion in this thread. Since long-term archiving of digital data is essential for me (and no doubt for many of you), I have investigated this in some detail:

*Hard drives.* Great for _short-term_ storage, backups, but not for long-term archives. Biggest problem is not de-magnetisation of the disks (half-life is typically ~70 yr) but aging of lubricants etc, resulting in mechanical failure after 3-5 yr. The problem is actually exacerbated by storing the drives powered off, off-line. Best strategy is to store multiple copies and re-write the data every 2 yr or so. On-line discs are ok for backup, but too volatile for archive.

*SSD/flash drives* lose their charge over time and typically last 5-10 yr unless re-written.

*Cloud storage* requires high bandwidth connection, useful for smaller amounts of shorter-term backup, but not for archives. Do you trust the cloud storage provider to still keep your data safe 20 yr from now? Will the company still be around?

*Optical media* - considered to be the ultimate digital archival solution, adopted by libraries etc. Note that there is a huge difference between pressed media and write-able discs. Write-able CDs and DVDs tend to use organic dye that deteriorate in time - could be as bad as in a few years only. There are some exceptions, *m-disc* being a notable one which uses inorganic dye (in the form of "rock powder"!), but requires specially designed writers with extra powerful lasers. Regular blue ray discs also use inorganic dyes, so should be fine. There is also an m-disc BD version, but it is not clear if there is any longevity advantage yet (although they are marketed as 1000-yr solutions). Stay clear from the *BD LTH* discs, however, which are developed to use the same manufacturing processes as CDs/DVDs with organic dye, to significantly reduce the manufacturing costs. They have similar lifetimes as the regular write-able CDs/DVDs. Note that pressed media are different and should have lifetimes on the order of 100's of years. They are typically not useful for archival purposes, though, since you need a very big series before it becomes economically feasible to press discs.

*Magnetic tape* - the most economical archival solution for huge data sets is to use magnetic tapes, such as *LTO*. There are archival WORM tapes ("write once, read many") that are certified for 40 yr storage, and you can get a 2.5 TB tape for less than $60. These tapes are typically used by huge data centers. A problem is of course that the tape drives are typically > $1500. Perhaps they are rentable. 

A good practice to protect from bit errors is to save redundancy information together with the data, in the form of error correcting codes (ECC). Popular software to generate these are* ICE ECC*, *MultiPar*, and* dvdisaster* (the last optimised for ECC on optical media).

Some* general, disorganised rants* about archiving that I nevertheless found useful.

*In summary*, hard drives are good for short term backups, (non-*LTH*) BDs for long-term storage of limited data generation rates (less than ~ TB/yr), magnetic tapes for large data rates (several TB/yr).


----------



## TexasBadger (Jan 2, 2015)

FTb-n said:


> Anyone writing images to Blu-ray discs as part of you backup strategy? If so, which size disc?
> 
> I currently backup to multiple external hard drives. Previously, before 25+MB RAW images, I also burned images to DVD. A couple years ago I started burning to 25 GB Blu-ray, but I've fallen behind. I've also had write issues with some discs failing. Grouping images to fit 25 GB discs can also be a pain. It's now cheaper and easier to get another external drive, like the WD My Passport to use as a backup. But, will a hard drive only strategy be reliable in the long run?
> 
> ...



Backblaze is $50/year if you pay for the full year at once. Unlimited backup of any drive except NAS. Does not require Java which is a great thing for computer security.


----------



## tpatana (Jan 2, 2015)

ajfotofilmagem said:


> Does anyone have the illusion that other people have the same care you have with your files?



How else after my death can they create the documentary spectacle of my life, starring Prad Pitt Jr?


----------



## Old Sarge (Jan 2, 2015)

Joe M said:


> If you want to get paranoid, make sure you have a few, store some in other locations if you can or buy fireproof/waterproof ones.


You mentioned the word paranoid so I thought I would tell a story on my son. He has always been great at backing up his system(s). It is done automatically. I believe he has a RAID set up and then a separate back-up system in another room of his house. A few years back he placed a computer on top of the gun safe at my house and at 4 a.m. it does an incremental back-up of his system (1.7 miles away in the same city) over the web. As he installed the system he asked if I thought another system at his mother's house (roughly seventy miles away) might be a good idea in case both of our houses burned at the same time. That is paranoid.



> I like the dock that tolusina posted. I have an older external Thermaltake dock for hds and use that for casual personal backups on various drives too. I think my next tower build will include that dock. With the price of SSDs dropping too, that will likely become the practical backup solution to simply pop in one of those and backup a ton of stuff in seconds.


My son, the paranoid geek, spent some of his time over the holidays to get two Synology DiskStation DS1515+ 5-Bay NAS Servers (bought from B&H) up and running. Eventually one will be at his house and one at mine, replacing the computer we are now using (an old case with minimal equipment). Like I say, he is serious about backing up data.


----------



## Mt Spokane Photography (Jan 2, 2015)

PropeNonComposMentis said:


> ajfotofilmagem said:
> 
> 
> > I used the DVD-RAM media (rewritable professional standard) for more than 10 years, writing and rewriting again and again, and I know it really last many decades. Unfortunately, the high price made it uncompetitive, and today the market is most new readers incompatible with this type of disc.
> ...


 
No,


I have never bought drives with consecutive serial numbers, I do check to see that they are from different batches, they often have different firmware versions as well, but I do generally buy them from the same manufacturer. My drives are powered up 24/7, they are on a UPS and protected from spikes and brownouts. The raid 5 arrays can withstand 2 drives failing and keep on going. My current set of six 2TB drives are reaching 5 years old, with no failures. I'll likely be replacing them with either 6TB Drives, or SSD's. The NAS before this one lived 5 years then the power supply died, but the disks were not affected. Even before that, I had a 4 drive Raid Disk Array of 150TB Drives, no failures, Before that, 4 - 72 TB Drives, Before that, 4 36 TB and before that a 4 Drive Raid Array of 13GB Disks (1990's) with no failures. All of my CD's made during the late 80's to late 90's rotted away, but I still have that old data on my hard drives. 

Where you get into trouble with hard drives is shutting them down and letting them sit for years. 
The bearings freeze up, so the drive won't start spinning.


----------



## RustyTheGeek (Jan 3, 2015)

LDS said:


> PropeNonComposMentis said:
> 
> 
> > 2.5" Ext Drives.
> ...



Umm... you're off a bit on the 2.5" drives. The 2.5" external drives are simply SATA laptop hard drives. Nothing more, nothing less. Yes, the drives are usually less expensive because they are typically not the highest performance, being 5400 RPM which is fine for their intended use. Their price is usually market driven. But they function just like any other laptop hard drive for better or worse.

I'm not sure what you mean about '_There are drives designed for long term storage_'. Hard drive technology is pretty universal when it comes to the platters and their magnetic retention. There are drives designed for enterprise level use and durability but in this case we aren't talking about MTBF, we are simply talking about magnetic retention. Hard drives don't typically fail based on any logic. They just fail whenever they feel like it. They are mechanical devices. Use them less, they will usually last longer. I spent time for years with a friend that owned a data recovery company and I know a thing or two about hard drives. And all the companies make lemon drives, sometimes on purpose (profit) and sometimes by accident (bad parts lot, etc). Regardless, hard drives are pretty reliable on the whole. Sort of like airplane travel.


----------



## Marsu42 (Jan 3, 2015)

PropeNonComposMentis said:


> I would recommend against using drives with more than One TeraByte per Platter. The Areal-Density of such drives is simply pushing the laws of physics.



That's what people said long before, but the industry came up with gmr, and it's now at 1,3TB/platter - who knows what's next in the pipeline: http://rml527.blogspot.de/2010/10/hdd-platter-database-seagate-35.html

Imho the problem is about *new* tech as new developments reach the market quickly given the competition, and that's even after only very few hdd manufacturers are left standing. And it's about the *amount* of platters inside a hd - more platters = more problems.

Problem is: Both issues are connected, because less density = more platters. When in doubt, personally I'd definitely go for a less platters, lower spindle speed and higher density - as long as it isn't the very newest platter tech.


----------



## LDS (Jan 3, 2015)

RustyTheGeek said:


> Umm... you're off a bit on the 2.5" drives. The 2.5" external drives are simply SATA laptop hard drives.
> Nothing more, nothing less.



Exactly. Not designed to last long. Most external drives are built to be cheap. Not to be long term storage devices. Good to shuffle files around, not for long term backups. There are external adapters that let you plug in the disks of your choice. IMHO far better choice than pre-assembled ones where you often don't know what disk is inside, for backups.



RustyTheGeek said:


> Hard drive technology is pretty universal when it comes to the platters and their magnetic retention.



But construction is not. There are different quality levels. The choice of materials and parts, production line specs and quality controls, etc. etc. After all, are the low end Canon DSLRs and lenses built with the same materials and specs of the high end ones? After all the technology is pretty universal when it comes to CMOS sensors, shutters and lenses... isn't it?



RustyTheGeek said:


> There are drives designed for enterprise level use and durability but in this case we aren't talking about MTBF, we are simply talking about magnetic retention.



But MTBF matters too. Sure, you can use multiple copies to reduce the risk - that's why I always alternate new sets to ensure they don't come near the MTBF together - but disks are more complex devices and thereby with more points of failure.

The quality of the controller and drive electronics/firmware has impact on the "correctness" of data written and its lifespan, as well the quality of the magnetic layer used for recording. Why high-end disks usually come in lower capacities (and longer warranties) than the lower-end ones? Because until they can warrant the same quality, they don't risk to sell you a disk that may fail too soon because the technology is not mature enough.



RustyTheGeek said:


> I spent time for years with a friend that owned a data recovery company



I spend part of my working time with Very Large Database applications running on SANs (Storage Area Networks) made of hundreds of disks (and lately, a lot of flash ones as well). Also the company I work for produces also military spec hardware, which needs to work in far worse conditions. Guess I learnt a couple of things too 

Anyway, companies like Facebook, Amazon, Google and the like are now faced with the need to store huge archives for decades (of course, unless they go bankrupt earlier) - because users in they early 20s or 30s expect them to store their "memories" for that long - and are experimenting with different technologies to keep data available at least near-line (as users are less likely to access often older contents) cheaply. And Facebook is now experimenting with blue-ray 100GB disks. If you store just the platters, and not whole disks, it's far cheaper.

Some companies (Sony and Panasonic) are working on "optical disks cartridges" able to store 1-1.5 TB each with RAID capabilities Currently Panasonic already offers the ADA cartridges, but they are only 50GB in size, and are somewhat a proprietary format. With the proper recording layer (one which won't decay if a few years, or decades) optical cartridges could become am interesting long-term storage for static data.


----------



## Valvebounce (Jan 3, 2015)

Hi epsiloneri. 
Wow, that is deep, I nodded off a third of the way down, dropped my blooming ipad. 
Very interesting, but not for reading in one session, good for reference. 

Cheers, Graham. 



epsiloneri said:


> Some* general, disorganised rants* about archiving that I nevertheless found useful.


----------



## Valvebounce (Jan 3, 2015)

Hi Folks. 
I do progressive improvements, just upgraded a pair of mirrored 500GB discs to 1TB, the old drives will be moved to my off site (is it backup or archive) storage. 
A lot of mine is JPEG only, but where I have RAW's and Dop's I will leave the processed JPEGs off to save space, yes I know the bit about not deleting stuff to make space but with the RAW's and Dop's', I figure I'm only really deleting processor time. 
By doing it this way the information gets re written regularly. 
It works for me, I hope it is sufficient to have the main data, mirrored, the backup at home mirrored and a single off site copy with the pair of drives rotated off site to home, intending to do this weekly but it may drop to monthly with the exception being after a special photo is taken. 

Cheers, Graham.


----------



## Mt Spokane Photography (Jan 3, 2015)

PropeNonComposMentis said:


> Mt Spokane Photography said:
> 
> 
> > I have never bought drives with consecutive serial numbers, I do check to see that they are from different batches, they often have different firmware versions as well, but I do generally buy them from the same manufacturer. My drives are powered up 24/7, they are on a UPS and protected from spikes and brownouts. The raid 5 arrays can withstand 2 drives failing and keep on going. My current set of six 2TB drives are reaching 5 years old, with no failures. I'll likely be replacing them with either 6TB Drives, or SSD's. The NAS before this one lived 5 years then the power supply died, but the disks were not affected. Even before that, I had a 4 drive Raid Disk Array of 150TB Drives, no failures, Before that, 4 - 72 TB Drives, Before that, 4 36 TB and before that a 4 Drive Raid Array of 13GB Disks (1990's) with no failures. All of my CD's made during the late 80's to late 90's rotted away, but I still have that old data on my hard drives.
> ...


 
Yes, once I started on TB, I forgot to shift to GB.

There is lots of good information posted here, the fact is that no one really knows how to store digital information for long periods. You can store it, but it may not be usable.

When my Father passed away, I collected hundreds of family photos and negatives, some going way back were tin types. I scanned them all, organized them by family, and sent out about 20 or more DVD's to all my cousins or others who wanted them in jpeg format.

Some will keep them, some may lose them, but it does increase the chances that they will survive a fire or loss by a few individuals.

I also posted many of the photos on Ancestry.com, where they were and still are being incorporated into family trees of more distant relations. I have also found photos there of my ancestors, so that is another way to preserve them.

Most of the photos before 1950 are black and white, older color photos and negatives have pretty much faded away. Only chrome based color slides seem to survive, and there were none of them in my family photos.

Of course, there are photos that have been over painted with oil paints that have survived. I even have a scan of a old tintype that has a bit of color showing where it was touched up with oils.

Most of the color polaroid photos I have have faded away since the 1960's and 1970's. The B&W ones are suffering as well.


----------



## RustyTheGeek (Jan 4, 2015)

Mt Spokane Photography said:


> I have never bought drives with consecutive serial numbers, I do check to see that they are from different batches, they often have different firmware versions as well, but I do generally buy them from the same manufacturer. My drives are powered up 24/7, they are on a UPS and protected from spikes and brownouts. The raid 5 arrays can withstand 2 drives failing and keep on going. My current set of six 2TB drives are reaching 5 years old, with no failures. I'll likely be replacing them with either 6TB Drives, or SSD's. The NAS before this one lived 5 years then the power supply died, but the disks were not affected. Even before that, I had a 4 drive Raid Disk Array of 150TB Drives, no failures, Before that, 4 - 72 TB Drives, Before that, 4 36 TB and before that a 4 Drive Raid Array of 13GB Disks (1990's) with no failures. All of my CD's made during the late 80's to late 90's rotted away, but I still have that old data on my hard drives.



I think *Mt. Spokane* is pretty sharp, has some great skill and experience with storage arrays. In short, he gets it, he knows what he's doing and he is covering all the bases. I also think that he is incredibly lucky. RAID 5 has diminished in popularity because it is so fragile and only tolerates ONE drive failure in the array. RAID 6 (among several other types) is becoming a more common PARITY Array type because RAID 6 can tolerate 2 simultaneous drive failures in the given array. RAID 5 performance is fair at best, RAID 6 performance is poor. Rebuilds after a drive failure in either array can take days, weeks or even months. For this reason, RAID 1+0 (10) is also popular if performance is more important. I won't go into the endless details of RAID here but I will say that unless you are ready to face a steep learning curve and a lot of stress and expense, don't implement a RAID array for yourself any more complicated than a RAID 1 mirror. It's just not worth it IMHO. Not only do you need to understand the technology, you need to understand the hardware and how to operate it. And once you venture past a simple RAID 1 mirror array, the hardware is critical for performance, acceptable reliability and even the possibility of recovery.

In my experience, drives fail in an unpredictable way and I'm amazed that *Mt. Spokane* has had such a long timeline with no failures. So while that is wonderful for him, I think it's a bit outside of the expected norm. I also predict that once he installs 6TB drives, he will probably see his first failure(s). It seems like as drive capacity/density has increased over the last few years, so has the failure rate (or at least the likelihood of data loss). And just because a the drives haven't failed, doesn't mean there can't be data corruption or loss. Backups still must be maintained and like you might expect, the more complicated the array, the easier it is for errors to creep in. Which is why you need more expensive controllers, etc for any array more complex than RAID 1.

*BUT - we are digressing. This thread is about BACKUP.* A lot of ideas have been tossed around. I stated my thoughts above and I'll repeat that if you put your faith in writable dye based media, you better test it every year or two because there is a definite history of this type of media failing after a few years. As for what kind of drive to use, I won't argue about what drives are best. All I know is that we all have a LOT of data to back up in the TB range. So external hard drives, whatever kind you prefer, are about all that is affordable and fast enough to get the job done. And while 'The Cloud' is big, it is very slow and out of your control. (Hello Mr. Dotcom?)


----------



## RustyTheGeek (Jan 4, 2015)

*LDS* - I won't argue with your replies to my points above. I think we are both right and I suspect you probably would agree with that. I'm glad you shared your knowledge because that way, we all learn something! 

In general, I think hard drive technology, after all these years/decades, is amazing and impressive but also somewhat disappointing. Because it is a mass produced item that has continued to drop in profit margins it is a dying product and yet is still necessary. So the overall quality is dropping slowly. But I do think that the quality is fairly consistent across all products with there basically being two classes, consumer/retail and enterprise. What you say is true about warranties and enterprise vs consumer lines but in this case, we are asking to copy data to a drive for a few hours or days and then store the drive away. It doesn't need to be a mil-spec 15,000 RPM database grade drive to do that. It just needs to hold the data. If someone is smart, they will check it every couple years and eventually copy the data from it to (hopefully) a better and more reliable media in the not-too-distant future. And that is all we need it to do. Hold the data for a few (5, 10, 15?) years. At least that is my realistic expectation.


----------



## epsiloneri (Jan 4, 2015)

Valvebounce said:


> Wow, that is deep, I nodded off a third of the way down, dropped my blooming ipad.


 I hope your ipad wasn't harmed in any way. An interesting thing about the internet is that no matter how fringe your query, there is bound to be a small group somewhere that makes it their life's mission to delve into that topic with excruciating detail. Archiving is such a topic.



RustyTheGeek said:


> I won't go into the endless details of RAID here but I will say that unless you are ready to face a steep learning curve and a lot of stress and expense, don't implement a RAID array for yourself any more complicated than a RAID 1 mirror. It's just not worth it IMHO. Not only do you need to understand the technology, you need to understand the hardware and how to operate it. And once you venture past a simple RAID 1 mirror array, the hardware is critical for performance, acceptable reliability and even the possibility of recovery.


I agree with your points about RAID 5 being inadequate for the big disks of today, and that RAID 6 has slow performance. But RAID 10 is very expensive if you don't need the performance (for 5 or more disks, say) and RAID 6 should be good enough for on-line back-up solutions. Any non-enterprise user should stay with software RAID, so understanding hardware quirks shouldn't be necessary. I thus think that RAID 6 is more often the better option for the usage pattern expected for those readers of this thread _with massive backup needs_. Better still would be RAID Z2, using the ZFS filesystem that protects the data against corruption through bit rot, but that is more for technically inclined.



RustyTheGeek said:


> It seems like as drive capacity/density has increased over the last few years, so has the failure rate (or at least the likelihood of data loss).


I know there is the perception that this is true, but is it indeed factually true? Hard drive failures have always been an issue, with some notable examples, e.g. the IBM deskstar 75GXP (aka the "deathstar"), and more recently the Seagate 1.5TB Barracuda.

In the end, I think this about the drive quality is a bit of a red herring. Yes, you should avoid obvious duds (such as the drives mentioned above), and yes, enterprise disks have more stringent quality control. But *in the long run, all drives, without exception, will fail*. The trick is to plan for it, and take appropriate measures.



RustyTheGeek said:


> *BUT - we are digressing. This thread is about BACKUP.*


While backup is indeed in the title of the thread, I think this thread is actually about data archiving:
_The key difference between backup vs. archiving is that data backups are designed for the rapid recovery of operational data, while data archiving stores data that's no longer in day-to-day use but must still be retained._



RustyTheGeek said:


> I stated my thoughts above and I'll repeat that if you put your faith in writable dye based media, you better test it every year or two because there is a definite history of this type of media failing after a few years.


Yes, that is why regular non-LTH blu ray discs are a much better option. Or the (more expensive) m-disc DVDs. They are not based on organic dye, so do not decay on short time scales. LTH BDs use organic dye like regular writable CDs/DVDs, so don't use them for archiving.


----------



## Oceo (Jan 4, 2015)

I do backups on the "3-2-1" plan, three copies of the file, in two different physical formats, and one copy off-site. For on-site I copy files to independent Hard Drives, one internal, one external. As to OP's question - for my off-site copy I started using 25GiB Blu-ray about a year ago. It still meant too much time waiting for disks to complete writing. After all, a few of us are shooting with 128GiB CF cards, more of us use 64GiB cards, and many more of us use 32GiB cards in our cameras. ( I shoot a total of more than 1TB/year from three cameras.) 

I've almost decided to go to LTO-3 tape. Refurbished writers are less than $1K, used but functioning writers about 1/3 of that. Beside the longevity advantages mentioned in previous comments I expect non-attended backup software must be available so I don't have to sit changing cartridges as I now do with Blu-ray disks. (Does anyone know of a consumer-quality Blu-ray writer that takes a stack of blanks and runs unattended?) 

I'd appreciate reading more from those respondents who have experience with LTO-n tape drives and media.


----------



## RustyTheGeek (Jan 4, 2015)

Oceo said:


> I do backups on the "3-2-1" plan, three copies of the file, in two different physical formats, and one copy off-site. For on-site I copy files to independent Hard Drives, one internal, one external. As to OP's question - for my off-site copy I started using 25GiB Blu-ray about a year ago. It still meant too much time waiting for disks to complete writing. After all, a few of us are shooting with 128GiB CF cards, more of us use 64GiB cards, and many more of us use 32GiB cards in our cameras. ( I shoot a total of more than 1TB/year from three cameras.)
> 
> I've almost decided to go to LTO-3 tape. Refurbished writers are less than $1K, used but functioning writers about 1/3 of that. Beside the longevity advantages mentioned in previous comments I expect non-attended backup software must be available so I don't have to sit changing cartridges as I now do with Blu-ray disks. (Does anyone know of a consumer-quality Blu-ray writer that takes a stack of blanks and runs unattended?)
> 
> I'd appreciate reading more from those respondents who have experience with LTO-n tape drives and media.



LTO Tape (or DLT, or QIC, or 8mm, or 4mm or 8 Track, or Cassette or whatever) is a bit antiquated these days. I'm not saying it's bad to think outside the box or that tape backup wasn't used in a serious way for quite a long time but I think the problems everyone experienced with tape are the reason tape isn't very popular today. In most cases when it came time to perform a recovery, there was much stress and holding-of-breath wondering if a recovery would even work at all. In general, testing was problematic and time consuming. Finding files was time consuming and cumbersome. Same with recovery. Everyone seems to forget that the most important part of a backup strategy involves testing the backup and being confident that the recovery process is not only reliable but that it is also understood how that recovery will take place and test it regularly. This is where tape was the biggest disappointment. In addition, fixing drives when tapes would get jammed or "eaten" and keeping heads clean is also a maintenance headache.


----------



## epsiloneri (Jan 4, 2015)

RustyTheGeek said:


> LTO Tape (or DLT, or QIC, or 8mm, or 4mm or 8 Track, or Cassette or whatever) is a bit antiquated these days.


On the contrary, LTO is cutting edge in enterprise digital storage, and its usage is _increasing_. The latest revision from 2012, LTO-6, holds 2.5 TB uncompressed and can be read/written at a bit rate of 160 MB/s - that's faster than single hard drives, and enough to saturate a gigabit network. Future revisions are planned to store increasingly high data densities, up to 48 TB with 1.1 GB/s for LTO-10 (granted some years into the future). Archival reliability for LTO cartridges is far better than hard drives, they are designed for long shelf life. You can easily convince yourself by googling around. Wikipedia gives some good background info on LTO.

A worry for archiving is naturally that there should be drives able to read the media in the future. Given the general (and increasing) usage of LTO and the backward compatibility of newer revisions, I think it is safe to assume that it will be possible to read tapes 30 yr from now without going to expensive extremes. The same goes for optical discs.



Oceo said:


> I'd appreciate reading more from those respondents who have experience with LTO-n tape drives and media.


I don't have my own tape drive, but at work we use LTO tapes for backups. The only advice I can give is to verify your data once written to tape, and perhaps write two copies to be stored at independent locations. I'd also recommend using WORM tapes for archiving, to avoid mistakenly over-writing data later. Make sure the drive you purchase is compatible with your system. LTO-3 stores 400 GB, and its data rate at 80 MB/s is still much better than you get for BDs. Plus, less swapping of media.


----------

