Missing characters when opening MIF8 in FM8 (p273)

[_Mats_Broberg_]

Hi,

Something our translation vendor just discovered.

Since it may have major ramifications for FM users who do alot of translations, we choose to post the bug report here too. More problems in CJK, Russian and Eastern European languages and less in Western European languages.

Concise problem statement:

Opening edited (translated) MIF files in FrameMaker 8 causes some of the extended characters disappear. Text containing extended characters for different languages are missing characters in FrameMaker, although all characters are present in the MIF file. The missing characters follow no particular pattern, most of the time all characters are there. (If you need more example texts in other languages to try this on, please contact me.)

Steps to reproduce bug:

1. Create a new FM file and enter "Some text".
2. Save the file as MIF (version eight).
3. Open the saved MIF file in any text Unicode-enabled editor.
4. Replace <String `Some text'> with <String `Ce produit est protégé par des brevets, des brevets de conception, des brevets en instance ou des brevets de conception en instance. Veuillez consulter la liste complète dans le manuel de référence sur CD-ROM.'>
5. Save the edited MIF file.
6. Open the MIF file in FrameMaker

Results

The word référence is missing characters, the text says "rfrence".

Expected results

All characters entered in the MIF file should appear correctly when the MIF file is opened in FrameMaker.

Best regards,
Mats Broberg

[_Mats_Broberg_]

Hi again,

Same type of error now confirmed in Russian, Korean, Simplified & Traditional Chinese, Danish, and Portuguese translations.

Best regards,
Mats Broberg

[_Mats_Broberg_]

Hi,

Further, confirmed in Czech, German, Greek, Hungarian, Japanese, Norwegian, Polish, Spanish, Turkish languages.

Best regards,
Mats Broberg

[Arnis Gubins]

Mats,<br /><br />>Expected results<br />><br />>All characters entered in the MIF file should appear correctly when the MIF file is opened in FrameMaker.<br /><br /><br />Not really. FYI, the "strings" in all versions of MIF <String><br />statements are defined only to be ASCII. Therefore in MIF v.8, it<br />should be stored using UTF-8 encoded characters. <br /><br />So your sample French text should show something like this in MIF:<br /><br /> <String `Ce produit est protégé par des brevets, des brevets de<br />conception, des brevets en instance ou des '><br /> > # end of ParaLine<br /> <ParaLine <br /> <String `brevets de conception en instance. Veuillez consulter la<br />liste complète dans le manuel de référence '><br /> > # end of ParaLine<br /><br />The older v.7 style prefers the FM hex-values so it would then look<br />like:<br /><br /> <String `Ce produit est prot\x8e g\x8e par des brevets, des<br />brevets de conception, des brevets en instance ou des '><br /> > # end of ParaLine<br /> <ParaLine <br /> <String `brevets de conception en instance. Veuillez consulter la<br />liste compl\x8f te dans le manuel de r\x8e f\x8e rence '><br /> > # end of ParaLine<br /><br />I think your translation vendor needs to check out the compatibility<br />of their toolset, i.e. they need to enter the strings with the proper<br />encoding (or in other words: user error).

[_Mats_Broberg_]

Arnis,

You are correct. The text between the tags is, I see now, a mistake on my side when quoting the vendor's report. Here is the bug report as it was posted to Adobe:

=============================================

******BUG******
Concise problem statement:
Opening edited (translated) MIF files in FrameMaker 8 causes some of the extended characters disappear. Text containing extended characters for different languages are missing characters in FrameMaker, although all characters are present in the MIF file. The missing characters follow no particular pattern, most of the time all characters are there. (If you need more example texts in other languages to try this on, please contact me.)

Steps to reproduce bug:
1. Create a new FM file and enter "some text".
2. Save the file as MIF (version eight).
3. Open the saved MIF file in any text editor.
4. Replace <String `Some text'> with <String `Ce produit est protégé par des brevets, des brevets de conception, des brevets en instance ou des brevets de conception en instance. Veuillez consulter la liste complète dans le manuel de référence sur CD-ROM.'>
5. Save the edited MIF file.
6. Open the MIF file in FrameMaker
Results: The word référence is missing one é, the text says réfrence. All other characters are there.
Expected results: All characters entered in the MIF file should appear correctly when the MIF file is opened in FrameMaker.

=============================================

I am not in the office at the moment and don't have access to FM8 until tomorrow but can you do a test on your side with your text to see if the problem persists?

Two different vendors using two fundamentally different translation tools and workflows report the same type of errors, so I would be surprised if this is not a true bug.

But I may be wrong.

Best regards,
Mats Broberg

[Arnis Gubins]

Mats,<br /><br />Interesting. I was able to duplicate the problem by inserting the<br />entire string. However, if I split the string at the end of the first<br />sentence (see below), then the word référence comes out correctly,<br />e.g.<br /><br /> <String `Ce produit est protégé par des brevets, des brevets de<br />conception, des brevets en instance ou des brevets de conception en<br />instance.'><br /> <String ` Veuillez consulter la liste complète dans le manuel de<br />référence sur CD-ROM.'><br /><br />I think it is a bug, as it's not tied in to any length limit (such as<br />255 characters) to the <String> component in MIF, since I tested it<br />using just the word, e.g.<br /><br /> <String ` référence référence référence référence<br />référence référence référence référence référence<br />référence référence référence référence référence<br />référence référence référence référence référence<br />référence référence référence référence référence'><br /><br />These all came out fine.

[_Mats_Broberg_]

Arnis,

Yes, this is a serious bug, not the least since multilingual support was marketed as one of the major features in v8.

In 21 translations (21 languages) we have found errors in all but four, ranging from 40 to 1 missing glyphs in each file. The number of words on each translation was approx. 2,000 words each.

The errors seem non-systematic over time and character range.

Best regards,
Mats Broberg