What causes 'Unexpected end-of-file' error?

[RogierBos]

Hi,

I am a photographer and an avid Lightroom user. I have two camera's (Nikon D700 and D300) and two computers, a desktop PC and a MacbookPro. Lightroom is installed on both computers.

For the last three photoshoots I have had this problem where 1 in 3 to 1 in 5 images in Lightroom on my desktop PC suddenly gives a 'Unexpected end-of-file' error. First I thought it was my camera. But then I loaded the images onto my MacbookPro and they worked fine. Then I took some images with my other camera, loaded them on to my Desktop PC, and, voila, same problem.

The problem specified: the image stops rendering a preview. A dull preview appears. At the top of the screen the message appears: "An unexpected end-of-file occurred." The histogram window stays empty and the image can not be developed.

I have browsed through this forum some looking for the cause and the fix. There are some mentions of it, but they all date back to 2007.

I run LR 2.2 on Windows XP SP2 and on a MacbookPro with OS X, version 10.5.8.

Please help me understand the cause of this problem, and help me find a fix?

Thanks,

Rogier

Rotterdam, The Netherlands

www.rogierbos.com

[Winjeel]

Nope

[IanHylands]

There seem to be several causes, one of them is faulty cards or card errors, the other is a pretty big problem with LR itself that Adobe refuses to comment on. I get file errors, missing preview images etc. fairly regularly on several different machines with different versions of LR. I sometimes have to import a file 3 or 4 times to get it to import properly without problems. It happens to me most frequently when synching or updating folders or when there is only 1 file to import. The file is already on my computer in the folder, nothing wrong with it, yet when LR imports it there are errors. This also happeens to random files on import. I usually then copy them over using Bridge or something to make sure they're alright and then keep trying til LR gets it right. I really wish Adobe would fix or even just acknowledge this issue.

[Winjeel]

Thanks for the insight. I'll try a few things and see what works.

Cheers.

[josephallannz]

I believe some of the problems above are related to Edit > Catalog Settings... > Metadata > EXIF > Write date or time changes into proprietary raw files.  If this is selected and you change the capture time on your images then around 1 in 256 photos will become corrupted.  The problem is one in 256 files will end with character 26 (or character 1A in Hex).  When updating the files Lightroom removes the trailing character 26.  First of all turn off this switch.  Secondly if you don't have a good original copy you can download and use HEdit to append the trailing character 26 to your file.

I first posted this problem in September 2007 on the forum.  It is still an issue in the 3.2 RC.  I have reported the bug through adobe labs (correct channel rather than this forum) and they are looking into the problem.

http://forums.adobe.com/message/1399193#1399193

If you are having problems with lots of images then obviously the issue is something else.

[Jochen_F__Uebel]

We fixed the problem (regarding saving on a server) with one small trick:

no file preview.

Jochen Uebel

[awesumtoy]

This is the first time I have received this problem. Not only do I get end-of-file error, most pictures after it get distorted lines. I shoot with a Canon and a UDMA 7 high grade card. I have checked all my memory on the PC, no problems there. I have checked my disk drive and all sectors were good (I have a 2 TB hard drive so this took all night). I have no hard ware problems as some have said. I can open up all my Raw files in other software, just not Lightroom 5. I have even copied all the Raw files to another directory and imported them into lightroom, and I have the same problem. I then started with a new catalog and the problem got worse. I'm lost on how to fix this.

[josephallannz]

What other software are you using to open up the Raw files?

[awesumtoy]

I was using Lightroom V5. I also tried it in Canon's DPP and Photoshop. All the photos are bad in whatever program I use. I am now thinking it is the CF memory card.

[jesse63]

Hi all

I'm having the same problem with my Nikon D700 NEF-files and Lightroom 4. This is definately a Lightroom issue, and the reason I say this, is based on the following observation:

I have made a Ligthroom-installation on both my stationary iMac and my MacBook Pro 15". Only difference is that I recently upgraded my MBP OS from Snow Leopard to Mountain Lion. A week ago, I transfered hundreds of NEF-files through a USB card reader to the iMac. All files showed perfectly in Lightroom on this machine. I subsequently copied the files from the iMac to  MBP via local network, imported to Lightroom and bingo – end-of-file errors. So this has nothing to do with the CF-cards.

The problem is relatively new to me. I've had a very rugged and stable setup for years, but somewhere along different upgrades, things have turned unstable. I can't help thinking it might be connected to the recent Moutain Lion upgrade. But as I read things posted here, even brand new hardware + new installations are giving headaches.

I think it would be the least Adobe could do, to point out some solutions and address this serious issue, that damages vital image files, and leaves customers in despare.

PLEASE, Adobe  –  speak to us!

[2bike]

Out of interest - have you tried converting the NEF files to DNG and then transferring them? Do they still get corrupted?

[jesse63]

I have most likely solved my problem with the corrupted RAW-files. The most logical explanation in my case is, that in conjunction with an upgrade to Moutain Lion, I also installed an 8 Gb RAM upgrade. This was a well known and recognized brand, sold by a widely trusted vendor. I was advised in another forum to check the RAM with the shareware REMBER, wich instantly diagnosed the RAM as defective.

So looking back, the math seems to be quite simple. The defective RAM have messed up the file handling, both when reading RAW files from flash cards, and when archiving to external hard drives. I haven't encountered problems since I replaced the RAM with new ones, that passed the REMBER test.

[Wodyetia]

Have just encountered this problem over the last few weeks.

I was unable to import into LR 5 or to convert to DNG with Adobe converter.

I have D800Es x3 and ceased using the CF cards hoping this may be the answer but still occurred affecting the first 7 images on card.

I have downloaded direct from cards to iMac using card slot - no external card reader

After reading the posts I tried some experiments.

It appeared impossible to open these images with Photoshop from desktop or from within Bridge. or to import into LR 5

The images opened in Capture One without a problem.

I made a few adjustments in C1 and saved as NEF - this file would not open in PS

Saved as tiff and the tiff file opened in PS CC no problem.

Perhaps someone from Adobe will read this & do something about it.

I imagine I am like many photographers who were dismayed by Adobe's new policies. I have been using LR 5 and fairly happily, but if I have to go through this rigmarole every time the fault occurs then my choice is simple as I can still use PS 6 on the few occasions C1 does not suffice.

[ssprengel]

Can you post an example raw image that won’t work in Adobe products? Upload to DropBox and post a public download link.

[Wodyetia]

https://dl.dropboxusercontent.com/u/63361338/_WTX3153.NEF

[ssprengel]

The data in that NEF file appears to be corrupted, as seen in RawDigger a raw-analysis program:

There is a full-resolution JPG embedded at the beginning of the raw file.  Can you tell if C1 is just truncating the corrupted area or if it is, perhaps, ignoring the raw data, and showing the extracted JPG?

[Wodyetia]

A similar file imported into C1 has dimension 7360x4912 and when saved as dng is 72Mb.

the above NEF file saved as tiff is 202Mb so it would appear that C1 is opening the NEF file and not the jpg

the dng and tiff  files saved from C1 will now open in Photoshop without the 'could not complete...unexpected end of file was encountered' message

The original NEF file opened with Graphic Converter as full size file

[ssprengel]

I do get an error when I open your corrupted NEF in RawDigger, it just a little more forgiving, and shows what it can decode:

My suspicion would be that C1 and Graphic Converter are either silently ignoring what errors there are and showing a smaller image that is missing the data along the bottom that cannot be decoded OR are merely showing  you the embedded JPG and ignoring the raw data altogether.

To tell which type of image you're looking at in C1, look at the bottom of the image you posted to dropbox without any color-noise-reduction nor chromatic-aberration correction applied.  If you turn off both of these corrections, the raw data will have both defects while the camera-embedded JPG will not.  Here is what I see zoomed to at 100%, in Windows Picture Viewer (top) which is showing the in-camera-corrected JPG and RawDigger (bottom) which is showing the raw data:

Finally, in C1, do you have absolute white-balance adjustments in Kelvin temperature units or do you have relative WB adjustments +/- from 0, only?  Absolute Kelvin numbers, like 2650K for Tungsten or 5000K for daylight, are only available for raw data.

[Wodyetia]

Thank you for your time.

in C1 the white balance is 5410K

the missing pixels along the bottom are shown

I turned off chromatic aberration reduction and set colour noise reduction to zero. I could not convince myself that there was any appreciable difference when zoomed to 100% and also 250% compared with the image with chromatic aberration turned on and some colour noise reduction.

[ssprengel]

The aqua fringe CA along the top of the rocks is obvious in the raw file without CA correction turned on as is the red-green color-noise mottling in the rock texture.   It sounds like you are looking at the JPG data to me unless C1 has poor CA removal and there is always some no matter what you do.  You could post a screenshot of what you see in C1 at 200% to better communicate what you're seeing.

Can you adjust the WB in C1 to 9000K or 2500K?

Can you post a DNG saved from C1, to dropbox, here?  This would be a DNG without any noise-reduction or CA-corrections.

[Wodyetia]

9000K and 2500K as requested saved from C1

no corrections or noise reduction in these files

both files open in PS

Dropbox - _WTX3151 copyCO-9000K.dng

Dropbox - _WTX3151 copyCO-2500K.dng

[ssprengel]

C1 is reading the raw data and filling in garbage for the part that is corrupted.  Is this what you want Adobe to do as well?

[ssprengel]

In any case, the files are corrupted somewhere between and including the camera and the computer.  Only you can determine what is wrong, by changing cards and seeing if it stops, or by transferring the images from the same card, repeatedly, using different methods and hardware (cables, card-reader, computer, and LR vs Finder/Explorer to transfer) and see if any of those changes make a difference in how many or which images are corrupted.  If one of the things you change does change which or if the images are corrupted then maybe you have found what the problem is.  Most likely it is the card going bad, but it is possible the camera, or one of the components involved in the transfer is also bad up to and including memory or the hard-disk in the computer.

[Wodyetia]

After reviewing all discussions I conclude your last post has the answer.

I would prefer to use Lightroom but at least I am reassured that if I do get a corrupted file I can still access it using C1.

I suppose if Adobe provided the error warning but opened the image with the corrupted part highlighted it would give the user more

understanding of what is happening and alert them to the need to check the component steps in the workflow.( and obviating the need to use Brand X )

If it is an end of file error and only results in loss of last part of image on margin then image is still useable

I will prepare a protocol to keep track of my workflow. Label all cards with a code and record

card number

camera bodies ( 3 x D800E)

external card reader

built in card reader in iMac

built in card reader in Macbook air

LR vs Finder transfer.

We are going to NZ in a few weeks and will be using all cameras and cards so theoretically if I keep a careful record I should be able to track down the problem step.

Camera -Card- Card reader /import-Computer

Hopefully it is not the dreaded 'intermittent fault'

I will keep you posted & thanks again

[markalanthomas]

It's pretty spooky to see this thread bumped to the top of the list today. Last night I had an image spontaneously develop this same end of file error. Camera didn't do it. SD card didn't do it. Card reader didn't do it. It's been in Lightroom and working fine for several days, and then suddenly last night it had an exclamation point. I ran Disk Utility on my SSD, no problems. Booted from a backup and ran Disk Warrior. Then used TechTool Pro to confirm SMART status, scan the SSD, check my RAM. Zero problems reported. I restored the image from the previous night's backup, but it would have been really sucktastic if I hadn't noticed the exclamation point and the next scheduled backup had overwritten the good file with the corrupt one. I wonder how many more damaged photos are in my library, waiting to be discovered. So here's my question:

How can I quickly view in Lightroom every file with an exclamation point icon? Smart Album?

(I'm using Lr 5.4 on OS X Mavericks with Nikon NEFs.)

[ssprengel]

I don't think LR provides a way to look for corrupted files.  DNGs do have a checksum that may work for doing this, but regular raw files don't.

You might try checking www.robcole.com to see if he has something that will help with this.  You may want to send him a message offline because a quick search of the index page does not show the word "error" so it may be an ancillary aspect of some plug-in, not the main purpose.