I have experienced this aswell. As mentioned in:
http://forums.adobe.com/message/4138626
and http://forums.adobe.com/thread/948417?tstart=0
My lightroom has after two weeks of playing with a fully working raw file corrupted it.
This is what the photo looked like originally http://500px.com/photo/14737987
As of today, I tried to reexport it with a diffrent watermark, the raw file corrupted. It exports a corrupted jpg and when I delete the .xmp metadatafile and try to import it again (same happens with it now when trying to import it to photoshop) the raw file now renders as corrupted in all software.
I have tried every possible way to reimport it, its just plain broken. If this should happen to my whole portfolio I would cry.
66
Replies
Rob, you may like to anwers the question........... pigs can fly as well, anything is possible
Here's evidence that LR is rewriting an entire JPEG file when I add metadata and do Save Metadata To File, just as Rob has described.
First, I imported a JPEG in which all the metadata had been stripped using Exiftool. Exiftool's dump shows that the image data starts at around hex offset 10:
Immediately after importing, I did Save Metadata To File, and the image data now starts at around hex offset 0cf0:
Next I added a short caption and did Save Metadata To File, and the image data now starts at around hex offset 0dd0:
I used the Microsoft Sysinternals' Process Monitor to trace file reads and writes to the JPEG file while LR was doing Save Metadata To File, and that confirms it is writing the entire 1 MB file:
Obviously, if there's a hardware fault anywhere in the path from the disk to memory and back, the contents of the image data could get corrupted.
And just as Rob has patiently explained, the fact that the file could get rewritten doesn't constitute any criticism of LR.
I have a friend who worked at Apple when it was just a startup, then continued on for several years after as a consultant. He is semi-retired now, but has *tons* of Apple stock - promotes and defends them zealously, right or wrong - often using words cleverly to twist the facts... - he'd make a good lawyer (but he's not much fun to talk to if you're a PC (or Droid...) user - even if you also think Apple is (and their products are) great ;-}).
Anyway, if this happened to you:
* Image was fine (raw fine I mean, not just jpeg prevew fine), and hasn't been copied to a new location (note: moving doesn't count, since no image data gets re-written when moving), and went south (confirmed using more than one raw-reading app) when using Lightroom, then:
- if proprietary raw, problem is probably hard disk (probably not ram).
- if DNG or rgb, consider turning auto-write xmp off, and not saving manually either (neither xmp nor updating DNG previews), as a temporary gesture - problem could be either hard disk or ram..., so get it resolved. Afterward, consider reenabling auto-write xmp and/or resume saving manually again.
Of course, always make backups (preferrably when hardware is *NOT* failing, and when failing (or failure suspected), if you do attempt a backup, make sure *NOT* to overwrite previous backups) in case hardware failure (or any other snafu) causes your working copies to become corrupt.
Lastly: once upon a time, ram test upon startup was mandatory. why? - to avoid having corrupted files (and/or other misbehavior). Why did that change?:
* Ram became more reliable.
* Ram amounts have become so big that ram tests take a lot longer.
The moral of that story? - consider enabling the option to always do ram test upon boot. If you're in front of your computer, just abort the test when things have been going smoothly, otherwise: let it run - odds are better this way that you'll catch ram problems before they'll catch you 
.
Rob
CB-Images wrote:
Hi Guys
I'm just arriving on the end of this discussion, I have to admit most of it is way over my head although I follow the principles.
We've tried to weave simplified summaries in with the highly technical stuff so you can get the gist without understanding everything.
CB-Images wrote:
after going through my Library of downloaded images everything appeared OK. I opened a few images into the Develop module, processed them, exported to Photoshop - again everything was fine.
This means your problem is not due to card, nor card reader, nor cable, nor USB port... i.e. images not corrupt after import and after raw rendering (raw rendering is done in the background, after initial import may appear ok, based on embedded jpeg previews), were not corrupted upon transfer.
CB-Images wrote:
Yesterday when I opened up the Library containing the same RAW (CR2) images the thumbnails appeared to be redrawn as the thumbnails came up and then moments later the images changed slightly, I assumed Lightroom was applying the tone mapping corrections I had made...
I assume so too: - sounds like (lib) previews were being recreated, in response to develop setting changes.
CB-Images wrote:
I was initially worried but then as I had a back-up and it wasn't a particularly important file I thought - 1 problem in 3 years of using Lightroom isn't too bad.
Indeed, many of us will go a lifetime without ever experiencing such corruption, which is why whether any particular user has or has not experienced it may not seem relevant to others who are experiencing it.
CB-Images wrote:
After reading through this discussion I opened Canon's DPP software and the same RAW file appears OK in DPP.
*Iff* Canon's DPP was showing you a valid raw rendering (i.e. we're *not* talking about the thumbnail, which can show a valid image despite having a corrupt file, since it only needs the embedded jpeg to be ok for that, which is a small part of the file compared to the raw data - I can't tell from your statement which kind of "appears OK" we're talking about), then file is not generally corrupt on disk. I say "generally" because it's possible if disk is marginal for it to be read ok one time, then not the next.
CB-Images wrote:
Today I have been processing some star trail images I took 2 nights ago.
These images have been imported into Lightroom and basic corrections carried out to the RAW files without problems.
I tried opening all 180 images into Photoshop for merging but Photoshop ground to a halt under the strain so I decided to open them in batches of 20. I selected 20 images in Lightroom, used 'Open As Layers in Photoshop', processed them and then on my 6th visit back to Lightroom one of the next batch of 20 images suddenly became corrupted as I selected it. All the thumbnails appeared to be updated/redrawn again as they all changed slightly everytime I returned to Lightroom. Lightroom also seems to have created a .tif file of this corrupted RAW file.
I couldn't tell - is tif file ok? or does it look wonky? - it would be helpful to show screenshots of wonkiness (corruption) - hardware failures have a telltale appearance - discolored rectangles and stripes, like the one originally posted in this thread. One thing to note: if file appears wonky sometimes but not other times, or not in other software, then the problem (assuming hardware failure for the moment) is occuring upon reading the image. In other words, it is not corrupt on the disk.
CB-Images wrote:
I have tried saving it with a different name and importing it into Lightroom, it appears OK as a thumbnail but when I try to open it, it is corrupted.
This is the case I was talking about previously - embedded preview can be read OK, but not the raw data: not that unusual, when this sort of thing is happening.
CB-Images wrote:
As I understand it, this could be due to faulty RAM, a failing HDD, a faulty card reader or cable.
1) Faulty card reader - fairly inexpensive, so I will try replacing it.
Logically wouldn't I be seeing more corrupt files if the Card Reader was responsible and not just 1 out of the 200 - 300 I have downloaded on each of the last few days?
Your problem is not from card reader nor cable... - images that are completely OK after import (e.g. can be processed in develop module / Photoshop / ACR), were transfered OK - forget the reader...
If you had a faulty card reader, it would be entirely possible for the failure to be intermittent, occasional... - do not assume... - they can be borderline / thresholdy or totally wonked.
CB-Images wrote:
2) Faulty RAM - someone suggested trying Memtest86, I looked at the web site and it looks far too complicated, I'm a photographer not an IT consultant.
Is there any easier way to check RAM or an idiots guide to using something like Memtest86 on a Mac?
Please report back after you succeed or fail with the bootable Memtest86 CD. As DdeGannes can attest, passing this test does not necessarily mean you have no ram problems, however failing it does mean you have bad ram, without a doubt. - usually bad ram will fail this test, but not always.
CB-Images wrote:
3) Replacing the HDD - I know HDD's can fail at any time but wouldn't I be getting all sorts of other problems if the HDD was begining to fail?
HDD's usually (read: not always) fail slowly at first, then go to hell in a bucket fairly shortly thereafter. So again - make no assumptions: it is entirely possible for hard disk to have bad spots that only affect some files... - disk can fail, or supporting electronics can fail, more often the former. Even when the electronics is failing it's possible for it to only have problems with some areas, not the entire disk - granted is is more often a problem disk-wide in the case of failing electronics (control circuitry).
CB-Images wrote:
3) Replacing the HDD - I know HDD's can fail at any time but wouldn't I be getting all sorts of other problems if the HDD was begining to fail?
This will be a pain to change as I believe I have to take my iMac to an Apple store, a friend had his HDD replaced and it took over a week but I want to be sure that this is where the problem lies before I go down this route.
I would like to discount the other 2 possible problem areas first.
It may be a pain to replace the HDD but it's *not* a pain to check it - I recommend doing that (checking it) ASAP.
CB-Images wrote:
I do think having a Forum like this is excellent to help us all find ways of solving the problems we all encounter from time to time, even if they do get too technical for the average person now and again.
Please report back. Much of what those of us who are not having problems learn, comes from the findings of those of you who are having problems...
Maybe somebody else will see something I missed. Generally though: problems can occur writing or reading. If OK sometimes, then problem is reading, if OK never, problem may have occurred while writing.
Good luck,
Standing by,
Rob
Hi Rob
Thank you for your comprehensive reply its much appreciated.
1) Images viewed in Canon DPP - yes you are right I was just looking at the .jpg thumnail, when I checked again it wouldn't/couldn't open the RAW file.
2) See attached .jpg of the corrupted file
3) I haven't yet run Memtest86 as I need to read and understand what I'm supposed to do and how to do it.
I did run Rember, I understand its not as thorough as Memtest86 but it was a start, I ran it 3 times and each time it stopped after just a few seconds with errors reported - see attached Log files.
This might strongly indicate RAM as opposed to HDD but I'm going to run Memtest86 later today as a double check.
I have also contacted Crucial about the results of Rember to see if my situation falls under their 'Lifetime' guarantee scheme.
4) I have also noticed small problems, initially I hadn't associated them with the corrupt file issue but now I think they may be related.
Yesterday when importing my batches of 20 images from Lightroom into Photoshop for blending, Lightroom would refuse to select a couple of images in the 'Library module'. I could select 19 of the 20 images but there would be just one image that would fail to be selected, in the end I closed down Lightroom and re-opened it and the problem went away. This did happen with 2 or 3 individual images.
Today when I was working in Photoshop I tried to save an image, while I could open the Save dialogue box I couldn't navigate to the folder I wanted to save the image in. When I clicked on the folder nothing happened, I had to close Photoshop down and re-open it again.
There have been a few small issues like this over the last few weeks but I didn't recognise a connection until now.
***NOTE TO OTHERS***
My backup software 'Deja Vu' obviously recognised the change in my corrupted file and over wrote the good image on my external back-up drive with the corrupted one.
In future if you have an automatic backup system in place and notice a corrupt file it might be worth disableing your automatic backups until you have corrected the cause of the corruptions otherwise you may loose your 'good' back up file.
Looks like you got a ram problem alright. And image is definitely wonked on the disk. Also, since you made develop changes to the images OK in Lightroom, after import, the images went south post-import, not during import (and weren't corrupt pre-import). Begging the question: how did those images go bad? They are CR2 files, thus the mechanism was *not* writing xmp. Did you copy the images on disk? e.g. restore from backup, or anything (e.g. update capture time in raw file)? I mean, corruption when being read/written is not far-fetched when hardware is failing, however far less likely (but not impossible) when image is only being read. This is when some might say things about flying pigs soaring from black holes and such stuff, but perhaps it is not so far fetched as we thought. I mean, assuming your assessment is accurate, as I understand it, it sounds like that is what happened (proprietary raw went bad, due to failing ram, without being re-written). - Not trying to stir up any undue FUD or anything (honestly, you gotta believe me ), but is there another explanation? Obviously failing hard disk can cause images to deteriorate where they sit, but other than that? Consider giving your disk a check too, but beware, disk-check may not be valid if ram is failing, although odds are still fair, since disk check only uses a miniscule portion of ram.
AdChoices