Data Merge cannot import Myanmar Unicoded UTF-8 formatted csv properly.

[BSNO]

I have a csv files typed with Unicode Myanmar Font (Pyidaungsu Font) and Data Merge import can not shown correct characters in InDesign. Attached my test data files and some screen shots. 

[Eugene Tyson · Accepted solution]

Best of my knowledge Data Merge in InDesign has very limited support for Unicode encodings. Even though your CSV is UTF-8, InDesign often struggles with complex scripts like Myanmar, Hindi, Thai etc.

A few things you can try:

• Resave the CSV as UTF-16 (not UTF-8). InDesign often reads UTF-16 text files better.
• Instead of CSV, try saving it as a tab-delimited TXT file with UTF-16 encoding. That usually works better with Data Merge.
• Make sure the Pyidaungsu font is applied inside InDesign after merging, because InDesign won’t automatically assign it.
• If nothing works, one workaround is to prepare the merge in Excel or Google Sheets, then copy-paste into InDesign manually, or use a third-party plugin like EasyCatalog which has much stronger Unicode support.

[Joel Cherney]

You're mostly right on the money here, Eugene, although I'd disagree with the claim that InDesign "often struggles with complex scripts." It's more that the UI for handling complex scripts separately from other scripts is old-fashioned, and a little bit clunky. 

InDesign's Data Merge tool can only accept data sources in a very limited number of encodings, and when selecting a data source, you have to tell InDesign exactly which encoding your data source uses. You can find this menu by checking the "Import Options" box when acquiring your data source:

If your data source is in Excel, then saving out "Unicode Text" leads to a file with UTF-16 encoding and tab delimiters, so if are handling your data in Excel on Windows, you'd need to pick Tab, Unicode, PC. But if Burmese is your target language, then UTF-16 is your only encoding choice for your data source. 

[Eugene Tyson]

Cool thanks for sharing!

[BSNO]

Hi Eugene,

Tab separated UTF-16 BE and UTF-16LE works. I can import Myanmar unicode (Pyidaungsu font). One time import seamlessly works. However if I remove the imported database and import the next UTF-16BE or UTF-16LE database it does not work.

Thank you.

[BSNO]

Hi Eugene,

UTF-16 BE and UTF-16LE works. I can import Myanmar unicode (Pyidaungsu
font). One time import seamlessly works. However if I remove the imported
database and import the next UTF-16BE or UTF-16LE database it does not
work.

Thank you.

[Eugene Tyson · Accepted solution]

@Joel Cherney might have better insights but here's what I think is going on

I think it's a quirk of how Data Merge handles text encodings. When you import the first UTF-16 file, InDesign caches the encoding and file structure internally. Even if you remove the data source, it doesn’t fully reset that cache. So when you try to load the next UTF-16 file, it often won’t re-read the encoding properly.

A couple of workarounds:
Save and close the document after removing the first data source, then reopen it before importing the next file.

Or give the new data file a different name before importing it (InDesign treats it as new and reloads it cleanly).

Stick to UTF-16 LE with BOM for consistency.

If you’re just updating data, you can overwrite the existing file on disk and choose Update Data Source instead of removing and re-importing.

I don't think it’s your font or file format causing it, I think it’s InDesign keeping old import settings in memory.

[BSNO]

Save and close the document after removing the first data source, then
reopen it before importing the next file. (This works.)