# Corrupted Images in Lightroom 5.3



## Rienzphotoz (Mar 7, 2014)

When I import images to Lightroom 5.3 (using "Add photos to catalog without moving them") some of my images are showing up with a weird color banding kinda thing at the bottom of the image (it only happens at the bottom of the image ... below is one for your reference ... you can also see a few more "damaged" images in the thumbnails at the bottom of the screen shot). However, when I download/check in Canon DPP, everything seems fine i.e. there is no image corruption/damage. 
This is happening only with the images made with my 5D MKII and 70D ... images from my other cameras, (EOS-M, G1X, Nikon D610 & Sony a7) do not exhibit this problem ... and it happens regardless of the memory cards I use (CF & SHDC). 
I import all my images to a Seagate Backup Plus USB3 Portable drive and access them from there ... but the same problem occurs even if I import directly to my MBP.
What do you think could be the problem?


----------



## philmoz (Mar 7, 2014)

Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
I can check on Windows and OS/X.

Might help narrow down if the problem is in the image file, or the LR software.

Phil.


----------



## Rienzphotoz (Mar 7, 2014)

philmoz said:


> Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
> I can check on Windows and OS/X.
> 
> Might help narrow down if the problem is in the image file, or the LR software.
> ...


Thanks for the reply ... below is a drop box link to the original file
https://www.dropbox.com/s/jpdp51b9hnfhb2f/IMG_1910.JPG


----------



## philmoz (Mar 7, 2014)

Rienzphotoz said:


> philmoz said:
> 
> 
> > Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
> ...



That looks like a corrupted file. It shows with the bands at the bottom in the dropbox preview, and in image viewers on both Windows and OS/X.
LR on Windows and OS/X says the image is damaged.
DPP on OS/X shows the thumbnail ok (it's a small JPEG embedded in the file); but hangs when I try and open the image in the editor.

Do you still have the image on a CF or SD card?

*Edit:* I've seen something like this once with a faulty card reader. It would corrupt files when copying from the card (if I copied from the camera via USB the files were fine).

Phil.


----------



## Rienzphotoz (Mar 7, 2014)

philmoz said:


> Rienzphotoz said:
> 
> 
> > philmoz said:
> ...


Yes, I still have the image in in the SD card and below is the link I've uploaded to dropbox (this one is straight from the SD card to dropbox using the MBP's internal card reader) and it does not exhibit the image corruption. 
https://www.dropbox.com/s/qou36027ernjcuk/IMG_1910%202.JPG

This is random occurrence i.e. sometimes it happens when I use the external card reader, sometimes with the internal card reader and sometimes even when it is straight from the camera ... this is just driving me crazy.


----------



## philmoz (Mar 7, 2014)

Rienzphotoz said:


> philmoz said:
> 
> 
> > Rienzphotoz said:
> ...



Ok, that means it's not the camera(s) that are the problem.

What are the common factors?

You say you can get corruption with different card readers and also via direct USB copy from the camera, so that would eliminate the card reader as the problem.

Are you sure it's happening with different cards? Do you have another corrupt image where there is still a good copy on a CF or other SD card?

If it's not the memory cards then that only leaves the external hard drive or the MBP itself. You mentioned earlier that you've seen the problem copying directly to the MBP which would imply the problem lies in the Mac.

At this point I would start doing some controlled testing and take notes as you go.
Eliminate as many variables as possible and gradually change one thing at a time to see if you can isolate the problem.

I would also suggest the following in case the problem is with the MBP:
- run a memory and hardware check on the MBP with the Apple diagnostics (http://support.apple.com/kb/PH14291)
- do a disk check and permissions verification on the MBP hard disk, and a disk check on the external drive.

Phil.


----------



## Rienzphotoz (Mar 7, 2014)

philmoz said:


> Rienzphotoz said:
> 
> 
> > philmoz said:
> ...


The frustrating part about this is that the images in the CF/SD card are all fine (no damage/corruption), it *only happens after I download/import it to Lightroom* (weather it be my MBP or iMac or even the Windows 7 laptop) ... when I download them to DPP, there is absolutely no problem ... so far, I've been able to isolate the problem only to lightroom ... my workaround for this problem is trying at least 2 different imports/downloads i.e. try directly with the CF/SD card in the external card reader or internal card reader and finally directly from the camera to the computer ... usually one of them works with corruption free images. But this is just frustrating and time consuming


----------



## philmoz (Mar 7, 2014)

Rienzphotoz said:


> The frustrating part about this is that the images in the CF/SD card are all fine (no damage/corruption), it *only happens after I download/import it to Lightroom* (weather it be my MBP or iMac or even the Windows 7 laptop) ... when I download them to DPP, there is absolutely no problem ... so far, I've been able to isolate the problem only to lightroom ... my workaround for this problem is trying at least 2 different imports/downloads i.e. try directly with the CF/SD card in the external card reader or internal card reader and finally directly from the camera to the computer ... usually one of them works with corruption free images. But this is just frustrating and time consuming



Can you outline the exact steps you take to transfer the files? 

From your original post it sounded like you are copying the files to the hard drive outside of LR and them importing them into LR after transfer to the computer.

This latest post sounds like you are using the LR import dialog to transfer the files to the computer.

I can understand the frustration - these kind of intermittent / random problems can be difficult to solve.
The only reliable way to track down the problem is with methodical testing 

If you are seeing the problem on multiple computers (and operating systems), multiple SD/CF cards, and multiple card readers then I'm really stumped, as that would seem to have eliminated all possible common elements. Unless you've found some very obscure Lightroom bug.

Phil.


----------



## Rienzphotoz (Mar 7, 2014)

philmoz said:


> Rienzphotoz said:
> 
> 
> > The frustrating part about this is that the images in the CF/SD card are all fine (no damage/corruption), it *only happens after I download/import it to Lightroom* (weather it be my MBP or iMac or even the Windows 7 laptop) ... when I download them to DPP, there is absolutely no problem ... so far, I've been able to isolate the problem only to lightroom ... my workaround for this problem is trying at least 2 different imports/downloads i.e. try directly with the CF/SD card in the external card reader or internal card reader and finally directly from the camera to the computer ... usually one of them works with corruption free images. But this is just frustrating and time consuming
> ...


Over the past few years, this has been my regular workflow for transferring files:
1. Copy paste images from CF/SD cards (using an external Lexar Card reader) to an external hard drive
2. Start Lightroom to import using "Add photos to catalog without moving them"

I never had an issue with that workflow ... but a few months into using LR5 (in 2013), I noticed some of the images, from the CF/SD card in my 5D MK III with the same issue (no matter which computer I use i.e. MBP mid 2013 model, iMac early 2013 model, Windows 7 laptop (2012 model).

Since this was happening randomly, I thought it was the fault of SanDisk 64 GB CF or the SanDisk 64GB SDHC card, so I tried the SDHC card in the Sony a7, but did not have any problem with it ... so I thought it might be my camera's fault.

But it is also happening with my 5 days old 70D with a brand new 32GB SDHC card ... so I am assuming that it isn't the card or the camera(s) ... so I changed the external hard disk but still the same issue ... then I tried importing the images to to my computer's internal hard drive, yet the problem still continues ... what baffles me is that it only happens for just a few images and the majority of the images are fine. Luckily most of the images that get corrupted/damaged are the bad images (which would have ended up in the trash anyway) ... but occasionally a few very good images suffer from this problem ... that's when I get totally frustrated


----------



## JustMeOregon (Mar 7, 2014)

Just a shot in the dark, but maybe try importing the files that are giving you problems into a new fresh Lightroom catalog. I've had a corrupt Lightroom catalog in the past and it resulted in a lot of strange intermittent errors.


----------



## philmoz (Mar 7, 2014)

Rienzphotoz said:


> philmoz said:
> 
> 
> > Rienzphotoz said:
> ...



Very strange.

You said that you also tried using the internal SD card reader in your MBP, and still seen the problem - is this correct?
Just want to confirm as it was a Lexar card reader I had trouble with (USB 3.0 dual CF/SD slot reader).

If this is the case then it would seem that Lightroom is the only common factor left.

Since you are importing the files in their existing location, LR would not normally write to the files unless you have something that is causing an update on import.
Are you applying any settings on import that might cause LR to update the files in some way? Even if you are it's hard to see how Lightroom could be causing this as it would normally only update the EXIF data.

Trying a new catalog as JustMeOregon suggested is worth a try.

You could also try resetting all the LR preferences and see if it helps (although I'd keep a backup if you've made a lot of setting changes).
http://www.lightroomforums.net/showthread.php?14226-Resetting-(or-Trashing-)-the-Lightroom-Preferences-file

I compared the two image files you uploaded and there is a small chunk of data missing (108 bytes) towards the end of the file, and the 404 bytes immediately before the missing chunk are all 0. Everything before and after this is identical.

Phil.


----------



## privatebydesign (Mar 7, 2014)

Narrow down the possibilities.

Are the files on the HD good? 

Yes: Save your LR library somewhere , reload LR and then relink your old library file.
No: You have an OS or hardware issue, as this happens across multiple drives it probably isn't hardware.

LR rarely touches the file, and never if you just "ADD" on import with or without a preset. The only time LR touches a file is when you move, copy, or export it, even then it doesn't write changes to the original, it just moves them, or copies them, or makes a new version.

If the files are good on the HD, I'd test open some problem images in DPP, then it is LR, just reload it.


----------



## philmoz (Mar 7, 2014)

privatebydesign said:


> Narrow down the possibilities.
> 
> Are the files on the HD good?
> 
> ...



To summarise:

The images are JPEG and it appears that they are not corrupted after the initial copy to the hard disk (not using Lightroom).
After subsequent importing into Lightroom the image files become corrupted.

From what reinzphotoz has posted this happens on different computers, with different operating systems, different CF & SD cards, different external drives and different card readers (waiting on confirmation of this).

Lightroom will write back to newly imported JPEG files if you tell it to auto save changes to XMP files, and do something on import like add keywords. This should only update the EXIF data; but may change the file size causing a re-write of the entire file.

Phil.


----------



## privatebydesign (Mar 7, 2014)

Missed that they were jpegs, my mistake I haven't used them since, forever!

I'd start by removing LR from my computer and reloading it.


----------



## Rienzphotoz (Mar 8, 2014)

privatebydesign said:


> Missed that they were jpegs, my mistake I haven't used them since, forever!
> 
> I'd start by removing LR from my computer and reloading it.


The file I uploaded was JPEG, but I generally shoot RAW and this problem occurs also in RAW files as well.
I was thinking about reloading LR ... but am worried that it might be the catalog that is causing the problem ... but I'll reload it one of these days (anyway the catalog is safely backed up in several places. But if it is the bloody catalog, then I'd need to sort (around 40000+ images, which is what I shot since LR5 release) and re-edit ... now that is making me tired already.


----------



## Rienzphotoz (Mar 8, 2014)

JustMeOregon said:


> Just a shot in the dark, but maybe try importing the files that are giving you problems into a new fresh Lightroom catalog. I've had a corrupt Lightroom catalog in the past and it resulted in a lot of strange intermittent errors.


Thanks for the tip and I have done that on several occasions ... when I re-import the images into a separate catalog, the problem disappears ... but it is really annoying, because I need to re-import everything separately and work with separate catalogs and name them separately so I can remember which catalog belongs to which image files etc ... which is just time consuming and makes the process from being enjoyable experience to cumbersome experience.


----------



## philmoz (Mar 8, 2014)

Rienzphotoz said:


> privatebydesign said:
> 
> 
> > Missed that they were jpegs, my mistake I haven't used them since, forever!
> ...



At this stage you might be better to contact Adobe.

Phil.


----------



## Rienzphotoz (Mar 8, 2014)

philmoz said:


> Rienzphotoz said:
> 
> 
> > privatebydesign said:
> ...


I was planning on doing it, but I thought I'd give it a shot at CR to see if anyone else is facing the same problem and how they fixed it ... and if they did not get it fixed, then I was planning on contacting Adobe saying that "I've got few more people who are experiencing the same problem" ... but looks like, I'm alone on this one.
Will keep you posted on what Adobe has to say ... thanks for your replies.


----------



## privatebydesign (Mar 8, 2014)

Rienzphotoz said:


> privatebydesign said:
> 
> 
> > Missed that they were jpegs, my mistake I haven't used them since, forever!
> ...



If it is happening to RAW files it is 100% your catalog.

But fixing it isn't as onerous as all that. I'd start a new one, then split the old one and "Import from another Catalog....." do this in stages and you will narrow down where your issue is, you will save all your edits, ratings and keywords etc too.


----------



## Rienzphotoz (Mar 8, 2014)

privatebydesign said:


> Rienzphotoz said:
> 
> 
> > privatebydesign said:
> ...


Could you please elaborate on "_split the old one and import form another catalog_"?
Thanks in advance


----------



## privatebydesign (Mar 8, 2014)

Rienzphotoz said:


> privatebydesign said:
> 
> 
> > Rienzphotoz said:
> ...



Yes,

Open LR in your current corrupted catalog (1), select several thousand images and then go File: Export as catalog...... (3)
Then open LR with a brand new catalog (Catalog 2) and go File: Import from Another Catalog........(3)
Then go back to your first, corrupted one (1), and export another few thousand images to a fourth catalog (4), then import that via the above moves to your new main uncorrupted one (2).

Sounds more complicated than it is 

Edit: Doing this does not move or touch your actual image files at all, it is just manipulating the catalogs.


----------



## JustMeOregon (Mar 8, 2014)

I think one of Lightroom's biggest potential pitfalls is not not being aware of a corrupt database-catalog. It is entirely possible to not notice a corrupt LR catalog; and thereby to naively continue to innocently work with it! Even having multiple backups won't help you if they are all derived from the same corrupt catalog.

Sh!t like this keeps me up at night, so I long ago resigned myself to save a number of unique iterations (if that's the correct word) of my LR catalogs that are each kept for a pretty long period of time.


----------



## privatebydesign (Mar 8, 2014)

JustMeOregon said:


> I think one of Lightroom's biggest potential pitfalls is not not being aware of a corrupt database-catalog. It is entirely possible to not notice a corrupt LR catalog; and thereby to naively continue to innocently work with it! Even having multiple backups won't help you if they are all derived from the same corrupt catalog.
> 
> Sh!t like this keeps me up at night, so I long ago resigned myself to save a number of unique iterations (if that's the correct word) of my LR catalogs that are each kept for a pretty long period of time.



People worry about the strangest things. It is just a catalog, it is not the actual images. It is of little consequence and should be anticipated and mitigated by a simple but robust digital management strategy.


----------



## JustMeOregon (Mar 8, 2014)

> It is just a catalog...


I realize, of course, that the catalog leaves the actual image files untouched. And yeah, the catalog is just a catalog... But it's a catalog of all the work I've put into the post-processing...


----------



## philmoz (Mar 8, 2014)

privatebydesign said:


> Rienzphotoz said:
> 
> 
> > privatebydesign said:
> ...



I'm not sure why you are claiming this?

The JPEG files were physically corrupted on the hard disk, and I would suspect the RAW files would be as well (Rienzphotoz - do you have a sample RAW file that you can upload both the corrupted and proper version).

How would this be caused by the catalog?
I'm not saying it's not possible; but I can't imagine a scenario where it would be the case.

Phil.


----------



## philmoz (Mar 8, 2014)

Rienzphotoz said:


> privatebydesign said:
> 
> 
> > Missed that they were jpegs, my mistake I haven't used them since, forever!
> ...



Been thinking about this some more.

AFAIK LR will only write back to a Canon RAW file if you update the capture time (and enable the write back option).

If the RAW files are being physically corrupted then I can't think of any scenario in which LR would be responsible for this (I'm not saying it can't happen; but it just seems very unlikely).

A sample corrupt and good RAW file would help.

Adobe are going to ask the same sort of questions already posted here.

I do understand the frustration dealing with an issue like this; but I'd like to suggest again that finding time to do some controlled testing is probably your best bet right now.

For example:
- shoot a bunch of images on a camera & card you've had problems with.
- copy the files to you computer
- before importing into LR compare the images on the hard disk to the CF/SD card (you can use diff on OS/X or fc on Windows). If the files don't match, then try changing card reader.
- if the files match, import into LR and see if you get any corruption

You might need to do this multiple times until you can reproduce the problem. It will probably be time consuming and tedious; but there really doesn't seem any other option to me.

Phil.


----------



## privatebydesign (Mar 9, 2014)

philmoz said:


> privatebydesign said:
> 
> 
> > If it is happening to RAW files it is 100% your catalog.
> ...



Because Rienzphotoz has said RAW files are not affected, only the LR rendition of them, _"when I re-import the images into a separate catalog, the problem disappears"_. The actual image files are not being damaged, they only look damaged in one catalog of LR, ergo, it is most likely to be that catalog.


----------



## Rienzphotoz (Mar 9, 2014)

privatebydesign said:


> Rienzphotoz said:
> 
> 
> > privatebydesign said:
> ...


Thank you ... I'll give it a go when I get home.


----------

