InDesign - need a "Fix my Corrupted File" Upload Service

Report · Oct 18, 2007

There are frequently posts on the forum for people who have corrupted INDD files. This means lost work and frustrated InDesign users.

The Indesign team should offer a "Fix my File" upload link to its customers.

For each corrupted file, you will help a customer, and in the process, hopefully improve the overall stability of the product (which has issues) and benefit everyone...

Adobe: If you are afraid that you will be swamped with requests to "Fix my File", then you should probably just get out of the SW business now...

Report · Oct 18, 2007

Is this a feature request or a rant? While I agree that ID CS3 crashes
more often than prior versions, I've yet to see a file corrupted because
of it.

Additionally, I haven't seen "frequent" posts about corrupted files. Perhaps a post in the main forum with FULL system specs as well as your maintainence routines would be helpful

Bob

Report · Oct 18, 2007

While I don't agree with the tone of the post, I think it's really not a
bad idea for there to be such a service (even for pay). Corrupted files
(once in a while) are a fact of life. Sometimes a lot of work can be
lost at tremendous expense.

I'll take this as a feature request, and agree with the idea.

Harbs

Report · Oct 18, 2007

While I'm inclined to want to support this as a request, how practical is it really?

Most of the techniques for file recovery have pretty good coverage here on the forums (perhaps a FAQ would be in order, if anyone would actually read it) and files corrupted beyond what those techniques can repair are, I would guess, most likely unrecoverable, even by engineers at Adobe.

Or are you suggesting there are secrets?

Peter

Report · Oct 18, 2007

> Or are you suggesting there are secrets?
>
>
While I'm not positive, I've been lead to believe that Adobe has
utilities (which they use internally) to analyze and possibly fix
corrupted files.

While I could understand not making such utilities available to the
general public, a service such that Casey suggests does seem to be like
a very nice idea.

I don't necessarily agree that such a service should be provided for
free. We'd all end up paying for it in the long run...)

Harbs

Report · Oct 18, 2007

<<<<Or are you suggesting there are secrets? >>>

When people report files that cannot be opened, or files that keep crashing Indesign when they try to open them (we all have stacks of them) - then an engineer could step thru opening of the file and find out what the root cause is - and either solve the problem - or put in safeguards that prevent the scenario - or at least prevent the symptom from causing lost work or crashing of InDesign.

Hopefully the files would get fixed and returned... and Indesign would get more and more robust over time.

Casey

Report · Oct 18, 2007

> (we all have stacks of them)

We do?

I don't mean to minimize the issues you're experiencing but I think you
need to look deeper at your systems. What you're seeing is NOT the norm.

Bob

Report · Oct 18, 2007

I've made my living working with InDesign since version two came out. I might, in that time, have had two un-recoverable files, but I'm only sure of one. If you have "stacks" of them you either have a lot of bad fonts, bad images, or bad drivers.

Peter

Report · Oct 18, 2007

I just completed another issue of the magazine and printed out all of the individual articles. I sent the PDFs to the printer and all was good (after recovering from a few corrupt files). (21 INDD files organized in a book)

Now it is done so I went to 'package' the book so I can archive it - but every time I try to package it, it fails (InDesign crashes) on one of the articles (which was fine up until that point.) Again...

I limit my font usage to very few fonts (Formata, Myriad Pro) and I have run Font Doctor to clean up any suspects. I only link to PSD files (no EPS, no JPEG, no TIFF, no PDF, no INDD, no Word or Excel). For each issue I run pre-flight to make sure no fonts have crept in.

My system has NO STABILITY OR PERFORMANCE ISSUES - NONE (Dual 1.8 Ghz OS X 10.4.10) -not in Photoshop, not in MS Office, not in Dreamweaver, not in Flash, no disk issues, nowhere else on the system - except in InDesign - which consistently crashes and corrupts files at critical times. No other app has corrupted a file. Some apps have crashed of course, but not with the frequency of InDesign.

I use two plug-ins Teacup (Type fitter) and Font X.

As of a hour ago I have another corrupt file for the InDesign team to investigate once they turn on the "Fix my Corrupt File" service.

(note - don't send any responses telling me to delete and restore my preferences - or to repair disk permissions....)

As for the "norm" - I did a quick search of the forums (Indesign MAC only)

crash - 590 occurrences (dups will occur)
corrupt - 202
"unable to open" - 106

Casey

CS3 - InDesign 5.0.1 (thanks for finally fixing the TOC alpha-sort bug that broke TOC in 5.0.0)

Report · Oct 19, 2007

casey,

What happens if you disable those plugins? I don't know "Font X" unless you mean Linotype "FontExplorer X," but auto-activation modules particularly in Suitcase, are well-known to cause crashing problems.

Harbs,

How much is a fair price? This service would be swamped by files that could be corrected by export to .inx or other known techniques. Is Adobe's time worth less than ours?

Peter

Report · Oct 19, 2007

I doubt that typefitter is the culprit but I would avoid any
autoactivation plugins.

Bob

Report · Oct 19, 2007

With a damaged files service, people will send in badly emailed files, only the half of files, and files internally damaged by HD crashes or malware activity.

And if they have to pay for it, they'll want their originals back no matter what!

I'll keep my eyes open for any new job requests at Adobe -- and if these appear, I'll be sure *not* to react :-)

Report · Oct 19, 2007

markzware offers the service to upload files which do not work with their products - for examination.

a few more searches of the "Indesign Mac" forum:

"Cannot open the file" - 276 instances
"INX" - 500 instances (commonly used for corrupted files)

Report · Oct 19, 2007

Big difference in the kind of corruption that INX can fix and the kind
you're referring to.

Bob

Report · Oct 19, 2007

Casey, please send me any corrupt files you have along with a brief explanation of what steps are required to reproduce the crash or other incorrect behavior so our QA can reproduce the problem and get a bug report into the system. (If the files are sent as .zip attachments, youll need to remove the file extension in order to get the files through Adobes e-mail filters. If theyre too big for e-mail, you could post them somewhere for me to download or you can contact me for ftp upload information.)

If the documents are fundamentally corrupt in some way, theres no way to repair them. However, anytime a user encounters a document or set of actions that result in a crash, wed very much appreciate receiving a copy of the file (packaged if possible) and an explanation of the steps necessary to reproduce the crash.

As others have already expressed, the instability youve described is not the norm. As a first step, I recommend removing all third party plug-ins.

If you crash please submit the crash log. When on the Macintosh, please include your e-mail address. If you submit a crash log with an e-mail address I may be able to determine the general source of the crash (i.e. a corrupt graphic, a bad font, a suspect third party plug-in, a bug in the core code of InDesign, etc.).

Thank you,
- Zak

Zak Williamson
Chief Architect
InDesign Product Family
Adobe Systems, Inc.
zak@adobe.com

Report · Oct 20, 2007

October 18 2007 must be InDesign file crash and cry day (and I didn't even see it on the calendar!) I have logged probably 6 hours, and used favors with as many InDesign designers trying to recover one file. It's a 180-page doc for the UN, and the loss of time to simply use the most recent backup is about 10 work hours, so I'm hesitant to have my artist just reconstruct. Zak, if your offer to Carey still stands, then I will email you a link to the file in about 14.7 seconds!!!

SYZYGY Media

PS: Ironically the file is on landmines!

Report · Oct 22, 2007

Please do send the file. As I mentioned to Casey, it's rarely possible to repair a truly corrupt document, but we definitely want to see any document that causes a crash so we can isolate the root cause and address if for a future release.

Thanks,
- Zak

zak@adobe.com

P.S. If you send the file as an e-mail attachment, you'll need to remove the .zip file extension in order to get through Adobe's e-mail filters.

Report · Jan 31, 2008

Zak Williamson wrote: "If the documents are fundamentally corrupt in some way, theres no way to repair them." and then later "As I mentioned to Casey, it's rarely possible to repair a truly corrupt document"
I definitely don't want this to sound like a flame or a whine, but I find that statement quite flabbergasting. Somebody at Adobe must understand the format of an InDesign file well enough to be able to edit out the specific element that is causing InDesign to complain about the document. I do file repair for another company (www.qsatoolworks.com) whose file format is probably more complex than an InDesign file. In over 10 years I've only had 1 or 2 files I could not repair at least to the point of recovering the textual data.

What led me to this forum is that a customer contacted me to help them recover an InDesign file. They just migrated from a PowerMac G4 with CS2 to a Mac Pro with CS3 and a few of their files won't open. ("ID may not support the file format, a plug-in that supports the file format may be missing, or the file may be open in another application.")
We have tried them on another Mac with CS2, and they won't open there either. "Something" must have gone wrong when the files were transferred to the new Mac.
(Naturally, the old hard drive has already been reformatted.)

Knowing what I do about file formats (but understanding that my knowledge is nearly worthless in an InDesign file) I have to assume an InDesign file contains an object map with pointers to the various objects in the document. The idea of telling the customer "your 32 page booklet is lost" when the problem could simply be a screwed up pointer is horrifying to me.
Here at QSA ToolWorks we charge $120/hour for recovery service, success or failure. We've saved lots of people from their own recklessness, and created a passionate user base in the process. We've made our product more reliable (and fixed some bugs) based on the knowledge gained through file repair; Adobe could do the same, but if they don't want to provide such a service, perhaps they should consider training third parties to do it.

I'd like to address one other comment above. Harbs wrote "I've been lead to believe that Adobe has utilities (which they use internally) to analyze and possibly fix corrupted files." Of course, I don't know whether Adobe has such utilities, but I suspect they don't. Because I can do Helix database repairs, people likewise assume I have "special utilities" that can fix corrupted files. (We've even been accused of keeping them secret so we can use file repair as a source of revenue!) I don't. What I do have is a deep understanding of the file format. Beyond that, a file editor and a couple of AppleScripts I wrote to display the object fields in a human-readable format are all the 'special' tools I have. If Adobe has any sort of tools for analyzing files, I suspect they are much the same. In other words, without extensive internal knowledge, they are worthless.

Report · Feb 10, 2008

Hi Matt.

Very interesting post!

I didn't mean to infer that there are tools that a layman can use. If I
understand correctly, the tools are to help in analyzing the document
structure. I believe one of the tools shows the structure in xml format.
(I was even told the names of the tools...)

If you are in a position of pursuing this, I think you should contact
Adobe. They might be more accommodating than you think...

A file recovery service for InDesign files would be a real commodity!

Harbs

Report · Feb 08, 2008

oh dear. i have my first ever unrecoverable indesign file on my hands. it's 70 pages or so, and i can't open it. i can't believe it. 2 days work. gone.

is anybody here actually able to help me?

was zak or anyone at adobe able to fix the UN file? or Casey's?

and Matt? are you saying you may be able to fix it at $120 p/hr? because that could potentially be cheaper than re-doing this entire thing......

anyone?

Report · Feb 09, 2008

No, I can't fix a damaged InDesign file. I can fix damaged Helix database files (www.qsatoolworks.com) because I understand the structure of the file and can find and fix the corruption. Usually there is data loss, but as you said, it's better to lose e.g. one page than the whole document.
If I had access to the native InDesign document format specs, I could probably (given time) figure out how to repair InDesign documents, but I doubt Adobe wants to give that information out. If they did, I'd be on it in a heartbeat, because I think a repair service would be invaluable.
But Adobe has to make the first move.

Report · Feb 10, 2008

Rebecca, please send me your file. See my previous post for instructions on how to send me a file.

I have received 11 files since my first post in this thread ~3 months ago. I was able to repair 2 of the 11 files.

The 11 files have shown one of three types of corruption.

1) Scrambled data due to a low-level bug in Save As, Save A Copy and Package. This bug has been fixed in 5.0.1, but there's no mechanism to recover from this type of corruption in a 5.0 file.

2) File truncation due to a Mac OSX bug. We're actively working with Apple to either get a fix in an OS update, or get information for how we can effectively workaround the bug in a future InDesign update. This type of corruption can occur when a machine experiences a hard crash (power outage, hold the power button down to force shutdown, kernel panic, crash on sleep or wake from sleep, etc.) while InDesign is running with a document open. In this case a large portion of the file is missing.

3) Faulty hardware. In this case individual data blocks within an InDesign have been altered by some outside force between when they were last written and when they are read. It's this type of corruption I've been able to recover from, because only a very small amount of data in a single data block was preventing a file from opening. In others cases the contents of multiple data blocks were scrambled.

The InDesign file format is a transaction-safe object database. Something as simple as a story containing a single word stores its data across more than 25 subobjects. Each of the subobjects is referenced by an index and the data is likely to not be co-located. Randomly scrambling a few blocks on disk can impact portions of dozens of subobjects and or destroy the index that indicates which object is which. While it would be possible to salvage random bits of Unicode text, truly repairing a document to anything resembling its original form (if possible at all) would in most cases be far more time consuming than recreating even a very complex file from scratch.

That said, short of hardware or media failure, it is completely unacceptable for there to be any way to corrupt an InDesign file. Any time we hear of a corrupt file we would like to receive a copy. We make every attempt to determine the root cause in order to either fix the problem for a future update or rule out the possibility it was caused by a software bug versus faulty hardware.

Report · Feb 10, 2008

Hi Zak.

Thanks for the explanation. I find this kind of stuff very interesting!
11 files sounds like this is not a very widespread problem, but it's
nice to hear that you're taking such a proactive approach on this!

Harbs

Report · Feb 11, 2008

Zak wrote: "The InDesign file format is a transaction-safe object database."
I reply: sounds similar to a Helix database. Which is sort of ironic in that Helix began life in 1983 with the goal of creating a page layout program. The first step was to develop a database for storing & manipulating the objects in the page, but when that was done, they realized the database was a viable product in its own right.

Harbs wrote: "If you are in a position of pursuing this, I think you should contact Adobe. They might be more accommodating than you think..."
I reply: I'd be very happy to pursue it, but I doubt Adobe would be very receptive to such a pitch from somebody they've never heard of before. I have no inside contacts at Adobe. Still, Zak's obviously following this thread, so I leave it in his hands. "Call me, let's do lunch." ;^)

Zak wrote: "Any time we hear of a corrupt file we would like to receive a copy."
I reply: Through what mechanism? I have a dozen corrupt files I would be happy to send you. (But I understand via third party that you already have one of them, so the repetition may be pointless.)

Report · Feb 11, 2008

I appreciate the information on this thread. I have a similar problem.

Using CS2 5.0.4 on WinXP, I started experiencing instant crashes, where Indesign just disappeared. No crash box, no warning, just poof! Gone! I suspected a third-party plug-in (Overset Text Manager), so I uninstalled it last week.

I was working on a 200-page book last night. While looking for images on IE (the document was open in Indesign but not active), Indesign just crashed on me. I hadn't done anything with Indesign for the past minute or so. I tried to reload the document, and nothing works.

Restarted. Yes to Autorecover. No to Autorecover. Unloaded fonts. Renamed Links folder. Installed update to 5.0.5. Used other computer. Upgraded to CS3. Restarted. Made a copy of file. Nothing. Crash, crash, crash.

Hoping to hear of any success stories from the rest of you.
-Tim