Files are corrupted by Lightroom: White rectangle

Report · May 03, 2008

Hi there,

I updated Lightroom to 1.4.1 yesterday and some time after the update I started experiencing a potentially disastrous problem:

When I click on any photo in the Library view (e.g. old ones from before the update), when the full resolution is loaded, sometimes a white recltangle appears in the picture as if the pixel information was missing there (its size varies between photos). The images were definitely OK beforehand.

I tried opening an affected photo in IrfanView and it reported a corrupted file header and couldn't open it. I'm slightly worried now that Lightroom is (maybe irreversibly) eating my precious pictures! Has anyone got the same problem? Any solutions?

Thanks,
Matt

Report · Dec 06, 2008

I spent about 3 weeks digging into my corruption problem which sounded very similar to the corruption described by Chris Miller.

Here is what I did which ultimately narrowed my problem down to a hard drive with a single unstable memory bit that would flip when reading the file. (And probably flipped on occassion when writing files as well.) No SMART drive diagnostics were able to find this error.

These steps are what I used to completely rule out Lightroom from my corruption issues:

1. Create a test directory on the drive to be tested. (The drive with your image files, although it could be another drive too.)
2. Create a txt file in that test directory and call it big.txt
3. open the file in Notepad.
4. In Notepad hold down the letter 'a' until you have a nice long row and press return. (In my case the bit flips I was seeing were best exposed using the letter 'a'. You may wish to try repeating other letters in order to cover other potential bit flips.)
5. Now repeatedly hit CTRL-A, CTRL-C, and hit CNTRL-V 5 times. (This will Select ALL, Copy, and then Paste 5 times)
6. Repeat #5 about 8 or more times. You want a nice large file. The larger the better to discover the corruption! I used 20MB to 50MB files. Smaller files work as well, but you see the corruption much more evidently with larger files. You don't want to go too large, or Notepad will start to have trouble.

Now you should have a single large text file in your test directory. Mine was 53MB.

7. Make about 50 copies of this file and name them sequentially such as "big_ (1).txt" through "big_ (50).txt" (Windows makes this fairly easy by automatically numbering if you select a bunch of files and then rename the first one.)
8. Open up a Command Prompt and go to your test directory.
9. now type:

comp "big_ (??).txt" big.txt

9. You would expect every copy of the file to be identical. And if the Hard Drive is operating properly you should never see these files differ, however for my drive the corruption errors were immediate.

F:\test>comp "big_ (??).txt" big.txt
Comparing big_ (10).txt and big.txt...
Compare error at OFFSET 271727B
file1 = 69
file2 = 61
Comparing big_ (11).txt and big.txt...
Files compare OK

Of course there are multiple ways that files can become corrupted, and it may not be your hard drive. But for me, I initally blamed Lightroom because I was only seeing this behavior in Lightroom and it all appeared to be the preview generation. However further investigation (it took quite a fair bit of investigation) revealed an otherwise undetectable problem with the hard drive.

Good luck, and I hope these steps help you or someone else track down their corruption errors.

Briand

Report · Dec 11, 2008

Just to share my experience, I'm unable to go anywhere NEAR that big with Notepad without it crashing. I'm able to go bigger with Wordpad, but only to about 16 Megs. At this point, when I try to copy and paste again it crashes. I therefore used 16MB files, and made 99 instead of 50 copies. I saw no compare errors.

I'm actually a bit confused, however, how these compare could be happening on Briand's system without anything showing up with SMART or in the Event Viewer. On top of this, when you copy a file in windows, shouldn't the error correction stop this sort of thing -- i.e. doesn't the system confirm correct data transfer when copying to the drive? Given this behavior, I'm wondering if the problem is actually not the hard drive after all, but rather if this could be a memory error, which would be harder to detect.

Larry

Report · Dec 11, 2008

It's odd to me that Notepad would crash on you. I am running on 64-bit Vista so that could be a factor in how large a file Notepad can handle. Wordpad would be a reasonable substitute of course and 16MB files should be more than adequate.

I was able to completely isolate this problem to the actual hard drive. Not only did I purchase a SATA->USB conversion device (in order to isolate potential SATA controller card and other memory issues) I also tested this drive usign the same procedure on a completely different computer. The Hard Drive failed these integrity/compare tests on the other computer as well as when I was accessing it as a USB drive. The problem did not occur with the replacement drive (same model #) or any other drive in my system (it was under warranty and replaced by the manufacturer.)

I'm also purplexed as to why this would pass the SMART tests, and not be discovered by error checking on the transfer/CRC error check. My only conclusion is that the problem must occur at some point in the stream that it goes undetected. Perhaps a problem with the read/write cache.

At any rate, its also worth noting that there was a pattern in the offset where the bit flip would occur. It was 99% of the time at an offset multiple of x727B, and always a bit flip from 1 to 0 on the 4th bit. Never from 0 to 1. So if I had filled up my file with the letter "i", I would never have seen the compare errors since the letter i has that bit already set to 0.

Of course this type of corruption is hopefully very unusual/uncommon so others may be experiencing different issues. I'm merely pointing out that what I first thought must be a Lightroom based on the symptoms, later turned out to be completely unrelated to Lightroom. LR was just the "catalyst" for me to find the hardware failures.

Briand

Report · Jan 25, 2009

It turned out I had a defective RAM module.

Even though the RAM tested fine when I last run a diagnostic, apparently memory degrades over time.

I swapped out the faulty module about a month ago and haven't seen the "white rectangle" corruption since knock on wood.

Report · Feb 04, 2009

Same rectangles and broken images for me...happened randomly after coming off standby. Had photos on my HD for a few days imported into LR2, everything was running fine. Out of nowhere 8 photos, (all sequential if that means anything), became completely distorted and broken. Using a 40D and imported off a card reader directly onto hard drive. I tried removing from library and reimporting but that doesn't do anything.

Report · Feb 07, 2009

Victor, did you read the previous post? Have you tried replacing your RAM? If not, why are you posting in a thread where the problem was resolved?

Report · Feb 26, 2009

Hello All!!

I am having the problem of certain photos that I have imported into LR 1.4.1 from the disk having the "rectangle explamation point" saying these files are corrupt and can't be used further by LR.

One similarity I am noticing is that they all seem to be pics that I downloaded to my LT via windows and are shots taken in the vertical. I must have rotated the photos in windows photo viewer. In other words, pics that I rotated in windows are getting the exclamation point..

Is there a process to correct this? I haven't tried deleting them from the library and then perhaps re-importing??? Any help or ideas much appreciated!

Best,
B

Report · Feb 26, 2009

Try synchronizing the folder they are in.

Report · Mar 01, 2009

I am having what seems like a similar problem. I have lightroom 2.2 and when I was taking photos using RAW and saving these as CR2, some of my photos were blank. That is there were no images and in addition, there was a pink line from the upper left moving down as diagonal line to the lower right. When I tried to save these images as a pdf slideshow, I could never recover my images. The strange thing about this is that when I used my 8 GB Transend SDHC card on my laptop and down loaded this card as *.dng files, I never lost any images on my laptop. This seems to suggest that there is some problem reading this card RAW images as *.CR2 on my desk top. The card adapter in both cases was made by lexmar. I am now going back and erasing the *.CR2 images from my desktop and will attempt to reload these images as *.dng and hope that I am able to load the images correctly on my 2nd attempt as *.dng. Does anyone have any suggestions as to what might be happening beyond what I have written here.

Thank you,

Hersch

Report · Mar 01, 2009

In article <59b526e0.57@webcrossing.la2eafNXanI>,
Hersch_Pilloff@adobeforums.com wrote:

> The strange thing about this is that when I used my 8 GB Transend SDHC card
> on my laptop and down loaded this card as *.dng files, I never lost any
> images on my laptop. This seems to suggest that there is some problem reading
> this card RAW images as *.CR2 on my desk top.

If you're using different readers, it could be either that or the
computer.

You will need to proceed by eliminating one variable at a time, use the
same reader with both and see if the problem re-occurs.

Also running a thorough check of the hardware might be necessary. In my
case it turned out to be a faulty memory module that checked out fine
about half a year ago.

--
Cheers Martin

Report · Mar 01, 2009

Hi Jao,

Thanks for the reply. Sorry forgot to mention that synchronizing the folders does no good. Even re-importing the folder, certain photos that have been rotated outside of LR are coming up with the
"" ! "" with the triangle.

Any help would be greatly appreciated! Running V1.4.1
Best,

Report · Mar 15, 2009

Hi All,

It seems most are associating this corrupted file with the reader's or media of the original... These photos were imported from an external HD containing photos copied from media readers and cards. There is one common factor with my photos and the "white rectangle-exclamation point and that is they were all rotated using Windows Viewer to the best of my knowledge.

I will try some of the ideas in the above posts as well. It should be able to establish a new photo within LR as well by renaming or other tricks I would think>?.?

Does editing photos outside of LR in PS and then synch or re-import cause problems as well? I try to use LR just for organizing but may be doing so in error. Any advise would be greatly appreciated!!

Thanks!!!

Report · Jul 03, 2009

Thought i would chime in with similar issues. been using lightroom since 1.0 with no issues importing via fw400 card reader. once i picked up a lexar fw800 reader it would corrupt about 2-3 files per import. You can re import them and they files are fine. the corruption is very random. had amazon send me a new reader thinking that was the issue but same problem. I them purchase the sandisk fw800 reader and I am having the very same issues. i've tried countless cf cards (all pro grade lexar and sandisk cards) as well as cable after cable and same issue. it's always to different files. getting really frustrated. I'd move back to my fw400 reader but since the new macbook pro's only have fw800 i'm kind of stuck and importing 20-30 gigs via usb readers are painful. I have the issue on both my macbook pro and my 8 core macpro (problems go away when using a fw400 reader on the macpro). attached is what the corruption looks like (the amount varies from image to image). i've also noticed that some will initially look corrupted like this and then once the standard preview gets rendered it's fine, others are not so lucky. please help

http://homepage.mac.com/bmarvin/filechute/lightroomcorrupt.jpg

Report · Aug 04, 2009

In my observations this is directly related to Lightroom. I have just installed the newest version and it still happens. Canon 40D (RAW+jpeg).I have used different memory cards, downloaded to my desktop using the built in card reader, using a Firewire card reader, copying form the CF card; I have recently looked back at images from over a year ago which had not been edited or altered at all, had been just fine and watched as Lightroom, as it painted the thumbnail panel at the bottom, turn a good image into a piece of junk, multi-colored lines and all, right before my eyes. Occasionally, I can deleted a photo from a folder, and re-import seperately, and sometimes Lightroom leaves it alone, sometimes it jacks it up. I no longer trust the software.

This has been going on for the past 4-8 weeks I would say. Getting corrupt catalog notices had never occured until that time, now it is a never-ending process. I wait and watch Lightroom mess up images. (I have had LR since it was new and have loved how it has evolved. I really like the poroduct when it works right. I am very disappointed, to say the least). Messes up new images, old ones that have been just fine until I took a recent look at them.

The only saving grace is that I tend to shoot multiple images...some may get hijacked by Lightroom and others may be ignored. I have no idea at this point what to do. I have run MEMTEST and my machine's memory is fine, graphics card has an up-to-date driver. I do not beleive that so many variables can change, yet Lightroom not be the culprit. Ugh.

Report · Oct 01, 2009

I have the same problem and it is a ACR problem for sure.

1. I have the latest updates on LR and CS4

2. Both have the same problem when opening the same CR2 file

3. I have a Canon 40D

4. The same CR2 files are opened just fine with the Canon DPP.

I tried to delete the side car file and repimport in the hope that all changes are kept there and therefore the corruption may be contained there as well. Nothing seems to impact the corruption.

The funny thing is that when you point to the file before importing the thumbnail LR and CS4 generate looks fine. They both actually show the proper full size image for a second and then part of the image is whited out with some diagonal lines here and there.

Also, many times this problem does not occur with the first time import but later one after a few times the file has been access / processed by LR2 / CS4. But I can't make out any particlar patern to recreate. However, once it happens the file will never recover.\

I wish I could share one of these files with one of you to see if the problem occurs across different computers or is it something local to me at this point with the specific file. (size limit is 5MB and 40D raw files are larger

Report · Oct 06, 2009

I had this problem in two forms: corruption during download and corruption of old photos. By corruption I mean big blocks of white or pink or other colors--sample JPEG attached. I first blamed my new Lightroom software, but now I think that is like blaming the messanger for bad news. The first thing I did per this discussion thread was to run various Dell and Microsoft diagnostic tests on my memory and then on my hard drive where the photos are. No problems reported. After more searching and testing here is what I think caused the two problems.

1. The download corruption was due to some sort of incompatibility between my new Lexar flash cards and my Sony card reader. This was true for RAW and JPEG photos. The corrruption was random and unpredictable, sometimes 1 out of 40 photos, other times 20 out of 40 photos. And also some sort of incompatiibility between these Lexar cards and my USB powered hub by Targus. So I eliminated the Sony card reader and now use a Lexar card reader, and I plug the Lexar card reader directly into a computer USB outlet, not through a hub. No more corruption problems when downloading.

2. Corruption of old photos--this affected about 200 photos out of 13,000 on my hard drive. They were all taken before December, 2007. What happened December, 2007? I bought a new desktop computer, and tried to transfer files from the old desktop to the new using "Laplink PCmover Essentials: Migrates applications with Easy Transfer cable." This software and cable made a mess of many of my programs. Maybe XP to Vista created additional problems. Anyway, I think these 200 photos got corrupted during various transfer attempts and computer crashes from old computer to new computer. I didn't discover these corrupted photos until recently, where it appeared Elements 5.0 Organizer (before I bought Lightroom) was corrupting them before my eyes, but of course it was seeing thumbnails from memory at first, then the actual photo with corruption. I think my backups were bad because a few months ago my external hard drive failed so I had to back up all my internal hard drive photos from scratch to my new external hard drive. At the time I thought all my photos were intact and fine.

So I hope my diagnosis above is correct and I can move forward. I have revised my backup procedures. How to repair the 200 corrupted photos? I don't think I can, bummer, but may try a few ideas and am open to your suggestions.

Good luck folks!

Tom W

Report · Oct 23, 2009

YES A POSSIBLE SOLUTION (cut-to-the-chase at the bottom):

my equipment:

iMac - 2.33Ghz core2duo - 4GB RAM - Snow Leopard

External USB 1.5TB+ where the RAW files go

Lexar firewire 800 CF card reader (this was introduced into my workflow... things started to get corrupt when I factored this into the equation)

Genaric USB card reader

16GB Kingston @ 133x (a few of these... could be the problem too)

16GB SanDisk Extreeme III 30MB/s

5D mkii

Lightroom 2.5 (CAMERA RAW 5.5)

back story: I've only been searching for about a few hours (after I discovered the corrupt files). I called Adobe... not the people you want to call when you have a fustrating situation like this... cause you get and indian guys, who is hard to talk to and super slow and pretty much said there is no solution. He didn't see if there where other people who had this problem or ask another support person... just cannot fix it. Shame on Adobe for hiring people who are hard to talk to and don't really understand. Adobe support = F in my books.

Sorry, back to the backstory. I had no problems or any corrupt files until recently. I would get one here and there, but no to the point that ever twenty pictures a few in a series would be corrupt and have white and pink lines and such.

Possible problem: There could be multiple factors it could be, and I'm slowly pining them down. I ultimately think it is combination of equipment I have especially pertaining to the LEXAR Firewire 800 card and the Kingston 133x cards. I'm definitly gonna change up my combination of equipment. My speculation is I think the LEXAR 800 is ripping the files too fast from the card reader.

MY SOLUTION:

I faced the fact... this is an project from a month ago and I've used my CF cards numerous of times, so I know the pictures are gone... no way to recover them from the cards again. So I did find a program called FILE JUICER (mac only, sorry) that extracts the JPG from the RAW file. Works great... even better is that the JPGs are all 21 mega pixels, yippie! So thanks File Juicer for saving the day!

I hope the person who ends up with the same problem will find this post.

Report · Oct 25, 2009

Thanks for this tip. I believe a few of my RAW photos got corrupted on import due to a faulty USB hub and a faulty Sony card reader. I salvaged a good part of my corrupted RAW files using the free PC software called "Instant JPEG from Raw" at http://www.rawworkflow.com/

I took these photos with my Canon 50D, set up for RAW only, no jpeg. This software still managed to extract a hidden 1 mb jpeg in each corrupted RAW file, and the jpeg was miraculously not corrupted. So a 1 mb jpeg is better than nothing! I imported these extracted jpegs into Lightroom 2 by specifying~~Copy photos to catalogue without moving, Information to apply~~ none and none. Also Preview--Minimal. I hope this never happens again!

By the way, I have some old corrupted photo files that are jpeg. This software extracted a small jpeg that was not corrupted from these jpegs, but the extracted jpeg is too small to be of much use to me.

Good luck!

Tom Wilberding

Report · Oct 25, 2009

On Sun, Oct 25, 2009 at 7:45 PM, Tom_W_1947 <forums@adobe.com> said:

>

Thanks for this tip. I believe a few of my RAW photos got corrupted on import due to a faulty USB hub and a faulty Sony card reader. I salvaged a good part of my corrupted RAW files using the free PC software called "Instant JPEG from Raw" at http://www.rawworkflow.com/
I took these photos with my Canon 50D, set up for RAW only, no jpeg. This software still managed to extract a hidden 1 mb jpeg in each corrupted RAW file,

That's the embeded JPG preview, the thing you see on the LCD screen on

the back of your camera, I believe. All RAW files have an embedded

JPG, don't they?

and the jpeg was miraculously not corrupted. So a 1 mb jpeg is better than nothing! I imported these extracted jpegs into Lightroom 2 by specifying~~Copy photos to catalogue without moving, Information to apply~~ none and none. Also Preview--Minimal. I hope this never happens again!

Congratulations! Glad you were able to get back at least most of what

you needed.

By the way, I have some old corrupted photo files that are jpeg. This software extracted a small jpeg that was not corrupted from these jpegs, but the extracted jpeg is too small to be of much use to me.

>

Good luck!

>

Tom Wilberding

>