Copy link to clipboard
Copied
I built a ScriptUI panel and suddenly all non ASCII-characters are mixed up:
The last word should read »löschen« (= delete in German).
The problem happened before and disappeared after a while.
I have absolutely no idea, wheather what caused it nor what caused it to go away.
Any idea, how to fix this (other than using ASCII only)?
I couldn’t find anything on this issue.
Thanks, Martin
1 Correct answer
One thing I would try is save the script file (actually I mean whichever file contains the string) as UTF-8 with BOM. For example, in VSCode, look down near the bottom right corner of window for the encoding. It might be UTF-8 by default. Click it and change it.
- Mark
Copy link to clipboard
Copied
One thing I would try is save the script file (actually I mean whichever file contains the string) as UTF-8 with BOM. For example, in VSCode, look down near the bottom right corner of window for the encoding. It might be UTF-8 by default. Click it and change it.
- Mark
Copy link to clipboard
Copied
Great, that did it!
I still have no clue, how this could have changed by itself.
Thanks a lot!
- Martin
Copy link to clipboard
Copied
Man, u saved my life, I was searching the entire internet thinking that the problem was with the string encoding instead of script encoding, the problem was really with the script encoding, UTF-8 with BOM!!!
THANKS THANKS THANKS!!! 🙏🙏🙏🙏🙏🙏🙏🙏🙏
Copy link to clipboard
Copied
Hi @marbay ,
I see something similar with ScriptUI. However, I do not think it's an issue with UTF-8 with or without BOM.
So my post is meant for people who cannot solve the issue by using UTF-8 with BOM.
I see the issue with InDesign 2023 on macOS. Had no chance to test it with Windows as well. Will do later…
Two cases:
[1] My string contains the German word "Maße" and shows up as "Ma√üe" in one of my ScriptUI "edittext" fields.
[2] In the same script "Maße" shows up as "Maße" in a different ScriptUI "edittext" field as it should be.
Here comes the difference:
In case [1] the string is extracted from an IDMS file using ExtendScript (JavaScript) XML's xpath() method.
I open the IDMS file with file.open("r") and write its contents to a variable with file.read().
Then the string is used with new XML(string) where xpath() is fetching a distinct attribute in the code; "Maße" is part of an XML attribute in the IDMS file. ( I cannot change that, e.g. using a double s for the ß, because I have no say how the attributes are defined.)
In case [2] the string is written from an InDesign's text frame contents property. No issue with that using it as contents of a ScriptUI "edittext" field.
Not tested yet:
As a workaroud I will substitute "√ü" with "ß" before assigning the extracted text to the "edittext" field.
I'll do that also for any transformed umlauts that will show up in XML attribute definitions.
Thank you for bringing up the issue, Martin!
Regards,
Uwe Laubender
( Adobe Community Expert )
Copy link to clipboard
Copied
@Laubender, so have you tried manually saving the IDMS with BOM and no change?
Copy link to clipboard
Copied
@m1b said: "… have you tried manually saving the IDMS with BOM …"
Good idea!
I'll come back after doing more tests.
( But it can take a while; working on different things right now…)
Regards,
Uwe Laubender
( Adobe Community Expert )
Copy link to clipboard
Copied
Hi @m1b ,
now that I am on my own Windows machine with Notepad++ I indeed can see that InDesign is exporting IDMS files as UTF-8 without BOM. So it's logical somehow, I assume a bug ( ? ) with ExtendScript's file.read() method, that umlauts or "ß" ( also other characters ) are converted. Does that make sense?
Did a little experiment:
I saved an IDMS file as UTF-8 with BOM with my Notepad++ app.
Placed that IDMS file in InDesign to see if it will still work to import a page item like a text frame.
Result: Did not work. Instead to interpret the XML code, InDesign placed the XML code as text.
So in the end I need a scripting method, apart from ExtendScript's file.read() method to fetch the contents of an IDMS file for doing some manipulatios of its contents and to extract some meaningful data from it. Or I always have to run it through a table of substitutions to get the fetched string right.
Regards,
Uwe Laubender
( Adobe Community Expert )
Copy link to clipboard
Copied
Ok. What worked is this strategy:
Export object to IDMS. Rename IDMS file to *.xml.
Import the renamed file as XML to an InDesign document.
Work with InDesign DOM xml to read out and edit the XML code of the IDMS file.
When finished export to xml. Rename the exported xml to *.idms.
And that one finally can be placed and InDesign will interpret the XML to page items.
Regards,
Uwe Laubender
( Adobe Community Expert )
Copy link to clipboard
Copied
Hi @Laubender, clever idea! I also did this test and can confirm that importing the UTF-8 (no BOM) .idms file as XML worked as you described.
- Mark
Copy link to clipboard
Copied
(I wrote this, before your last comment went live. I guess your solution makes more sense)
Hi @Laubender,
I’m not sure, if I understand your intended workflow:
You import an existing IDMS, find a specific attribute and display (part of) that attribute in a ScriptUI dialogue.
Is this correct?
In the previous comment you wrote, that you placed the IDMS in an InDesign document. Did you actually place it in the UI. I wouldn’t be surprised at all, if InDesign behaved differently in both cases (placing vs. reading from a script). Have you also tried to file.open(r) the modified IDMS?
If reading the file from script works, you could just add the BOM via JavaScript.
Regards,
Martin
P.S.: I’m not a programmer, just a graphic designer who uses some scripting. So please excuse me, in case this doesn’t make sense.
Copy link to clipboard
Copied
Hi Martin,
what I really would like to do is a file concatenation. Guess, this must be done without ExtendScript on a system specific level: A file with the BOM mark only concatenated with the IDMS file holding the XML code.
That could be read with concatenatedFile.open("r") and concatenatedFile.read() without seeing any transformation of characters.
Showing some properties of XML elements and XML attributes in a ScriptUI context is only a side project.
What I really wanted was fetching a string of the IDMS file, working with it as ExtendScript XML object, doing some things with e.g. xpath() and other methods and finally writing back the XML to another IDMS file for placing it with InDesign.
Regards,
Uwe Laubender
( Adobe Community Expert )
Copy link to clipboard
Copied
Hi @Laubender, that's interesting. I tried doing the same test except I used VSCode to save with BOM. Here's exactly what I did:
1. Typed "Maße" into text frame.
2. Exported text frame to test.idms.
3. Made this script in same folder as (2):
var file = File(File($.fileName).parent.fsName + '/test.idms');
file.open("r")
var idms = file.read();
var xml = new XML(idms);
var content = xml.xpath('//Content')[0];
$.writeln(content);
The result of this script was "Maße". Also the raw string (idms variable in script) also shows "Maße" so it isn't anything to do with XML.
4. I opened test.idms in VSCode, clicked on "UTF-8" in the bottom bar and chose "Save With Encoding" in the command palette that appeared. I chose "UTF-8 With BOM".
5. I ran the same script from (3) again. The result was "Maße".
So, in my situation (MacOS 13.4), the issue was fixed by adding the BOM and had nothing to do with XML object. However, this is still a bug with ExtendScript's File.read function I suppose. I assume this has gone away with UXP?
- Mark
Copy link to clipboard
Copied
Mark wrote:
"I assume this has gone away with UXP?"
Hi Mark,
no idea and remains to be seen; as you know UXP for InDesign is in constant development, a moving target, so to say. I just began to test the new UXP APIs that were introduced with InDesign 18.4.0. With my German account I still cannot see an update to the release version 18.4.0…
Regards,
Uwe Laubender
( Adobe Community Expert )

