Highlighted

How can you get text including Emojis into a data-merge document?

Community Beginner ,
Jul 30, 2019

Copy link to clipboard

Copied

Hi,

I'm using InDesign to create a memory book. Many people get one page each to fill with content. As past projects have shown, people are really keen about using Emojis, which I'd like to support. But I just can't get Emojis into InDesign via Data Merge, only when I copy&paste them manually.

My workflow so far:

  • In InDesign, I create a layout with sample texts, including Emojis.
  • I create a character style for Emojis, using the EmojiOne font.
  • In my paragraph style, I define a GREP Style to detect Emojis in any text and apply the character style created in the above step.

This works. But as for the data merge part:

  • I provide a sample page to help people get an idea how to fill it and what it will look like.
  • I provide a Google form. People fill it with text and upload images.
  • Google spreadsheet magic: I'm using a few formulas to prepare the texts for InDesign, and a script to determine the file names of the uploaded images.
  • I download the resulting spreadsheet (tried several formats here).
  • I use LibreOffice to convert the downloaded spreadsheet into .csv with UTF16-encoding.
  • In InDesign, I import the .csv using the Data Merge panel.

But the problem is: No matter how I format it and how I convert it, I won't get Emojis into the InDesign document. Only one of the two cases happens:

  • Emojis (as well as other non-English characters) turn into cryptic symbols (when interpreting as ASCII - that's the expected behavior).
  • Special characters survive, but Emojis are just stripped out of the text (when interpreting as Unicode, and formatting the source as UTF-16).
  • But if I open the UTF16-formatted .csv with a text editor or with LibreOffice, select the text including the Emojis, copy it and paste it into my InDesign document, it works.

Does anyone know a solution? What am I overlooking? Is there any other character encoding than UTF16 that InDesign supports?

Correct answer by joheiat | Community Beginner

Ha, I have found a solution!

 

How to get line breaks and Emojis into InDesign via Data Merge

 

InDesign doesn't support line breaks or Emojis from data sources that you import via Data Merge. With this workaround, you can encode any special characters in your spreadsheet and decode them in InDesign again.

 

In your spreadsheet:

  • Prepare all your data as usual (including Emojis and line breaks)
  • Add a second sheet to your spreadsheet
  • Use formulas to fill it, and use ENCODEURL() to encode every special character in URL notation (spaces will turn into %20)
  • Export the result as .csv and make sure to convert it to UTF-16 character encoding for InDesign (using a text editor or LibreOffice)

Capture.PNG

 

In InDesign:

  • Open a text editor, copy the below script into it, and save it as URL-decode.jsx

  • Move the script file to your scripts directory - on Windows this is: %AppData%\Adobe\InDesign\Version 14.0\en_GB\Scripts\Scripts Panel

    (you can type %AppData% into the Windows search box, press enter and navigate to the folder from there)

  • Use Data Merge as usual and import the .csv as data source
  • In the preview, you will see the URL-encoded characters
  • Click the "Create Merged Document" button
  • The resulting file will still contain all the URL-encoded characters
  • From the scripts panel (Window -> Utility -> Scripts), run the URL-decode.jsx script
  • Voilà, all your special characters are back!

 

URL-decode.jsx:

//DESCRIPTION:Decodes and replaces all URL-encoded strings in the active InDesign document.
//AUTHOR:Johannes Eichberger
//
//Example:
//    This%20is%20URL-encoded%20text!
//    This is URL-encoded text!
//
//How to use:
//    Open an InDesign document with URL-encoded texts.
//    Run the script.
//    It replaces every (!) text in every text frame.
//
//Limitations:
//    Works with text frames that contain text only. Any non-text
//    content, such as tables and linked images, will get lost.
//    If an encoded text exceeds its container (overset text),
//    only the visible part will be replaced. You can run the script
//    multiple times to solve this issue.
//
//When to use:
//    InDesign's Data Merge has only poor support of character encodings.
//    If you want to feed InDesign with Emojis, you can encode them in the
//    spreadsheet via =ENCODEURL() and decode them using this script
//    after running Data Merge.
//

#target indesign;

function main() {
    try {
        var myDocument = app.activeDocument;
        var myTextFrames = myDocument.textFrames.everyItem();
        var myContents = myTextFrames.contents;
        for (var i=0; i<myContents.length; i++) {
            var thisTextFrame = myDocument.textFrames.item(i);
            var myDecoded = decodeURI(myContents[i]);
            thisTextFrame.contents = myDecoded;
        }
        if (i==0) {
            throw "No texts found!";
        }
        alert("Texts have been replaced.");
    }
    catch(err) {
        alert(err);
    }
}
main();

 

 

Views

515

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

How can you get text including Emojis into a data-merge document?

Community Beginner ,
Jul 30, 2019

Copy link to clipboard

Copied

Hi,

I'm using InDesign to create a memory book. Many people get one page each to fill with content. As past projects have shown, people are really keen about using Emojis, which I'd like to support. But I just can't get Emojis into InDesign via Data Merge, only when I copy&paste them manually.

My workflow so far:

  • In InDesign, I create a layout with sample texts, including Emojis.
  • I create a character style for Emojis, using the EmojiOne font.
  • In my paragraph style, I define a GREP Style to detect Emojis in any text and apply the character style created in the above step.

This works. But as for the data merge part:

  • I provide a sample page to help people get an idea how to fill it and what it will look like.
  • I provide a Google form. People fill it with text and upload images.
  • Google spreadsheet magic: I'm using a few formulas to prepare the texts for InDesign, and a script to determine the file names of the uploaded images.
  • I download the resulting spreadsheet (tried several formats here).
  • I use LibreOffice to convert the downloaded spreadsheet into .csv with UTF16-encoding.
  • In InDesign, I import the .csv using the Data Merge panel.

But the problem is: No matter how I format it and how I convert it, I won't get Emojis into the InDesign document. Only one of the two cases happens:

  • Emojis (as well as other non-English characters) turn into cryptic symbols (when interpreting as ASCII - that's the expected behavior).
  • Special characters survive, but Emojis are just stripped out of the text (when interpreting as Unicode, and formatting the source as UTF-16).
  • But if I open the UTF16-formatted .csv with a text editor or with LibreOffice, select the text including the Emojis, copy it and paste it into my InDesign document, it works.

Does anyone know a solution? What am I overlooking? Is there any other character encoding than UTF16 that InDesign supports?

Correct answer by joheiat | Community Beginner

Ha, I have found a solution!

 

How to get line breaks and Emojis into InDesign via Data Merge

 

InDesign doesn't support line breaks or Emojis from data sources that you import via Data Merge. With this workaround, you can encode any special characters in your spreadsheet and decode them in InDesign again.

 

In your spreadsheet:

  • Prepare all your data as usual (including Emojis and line breaks)
  • Add a second sheet to your spreadsheet
  • Use formulas to fill it, and use ENCODEURL() to encode every special character in URL notation (spaces will turn into %20)
  • Export the result as .csv and make sure to convert it to UTF-16 character encoding for InDesign (using a text editor or LibreOffice)

Capture.PNG

 

In InDesign:

  • Open a text editor, copy the below script into it, and save it as URL-decode.jsx

  • Move the script file to your scripts directory - on Windows this is: %AppData%\Adobe\InDesign\Version 14.0\en_GB\Scripts\Scripts Panel

    (you can type %AppData% into the Windows search box, press enter and navigate to the folder from there)

  • Use Data Merge as usual and import the .csv as data source
  • In the preview, you will see the URL-encoded characters
  • Click the "Create Merged Document" button
  • The resulting file will still contain all the URL-encoded characters
  • From the scripts panel (Window -> Utility -> Scripts), run the URL-decode.jsx script
  • Voilà, all your special characters are back!

 

URL-decode.jsx:

//DESCRIPTION:Decodes and replaces all URL-encoded strings in the active InDesign document.
//AUTHOR:Johannes Eichberger
//
//Example:
//    This%20is%20URL-encoded%20text!
//    This is URL-encoded text!
//
//How to use:
//    Open an InDesign document with URL-encoded texts.
//    Run the script.
//    It replaces every (!) text in every text frame.
//
//Limitations:
//    Works with text frames that contain text only. Any non-text
//    content, such as tables and linked images, will get lost.
//    If an encoded text exceeds its container (overset text),
//    only the visible part will be replaced. You can run the script
//    multiple times to solve this issue.
//
//When to use:
//    InDesign's Data Merge has only poor support of character encodings.
//    If you want to feed InDesign with Emojis, you can encode them in the
//    spreadsheet via =ENCODEURL() and decode them using this script
//    after running Data Merge.
//

#target indesign;

function main() {
    try {
        var myDocument = app.activeDocument;
        var myTextFrames = myDocument.textFrames.everyItem();
        var myContents = myTextFrames.contents;
        for (var i=0; i<myContents.length; i++) {
            var thisTextFrame = myDocument.textFrames.item(i);
            var myDecoded = decodeURI(myContents[i]);
            thisTextFrame.contents = myDecoded;
        }
        if (i==0) {
            throw "No texts found!";
        }
        alert("Texts have been replaced.");
    }
    catch(err) {
        alert(err);
    }
}
main();

 

 

Views

516

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Jul 30, 2019 0
Community Beginner ,
Jul 31, 2019

Copy link to clipboard

Copied

Ha, I have found a solution!

 

How to get line breaks and Emojis into InDesign via Data Merge

 

InDesign doesn't support line breaks or Emojis from data sources that you import via Data Merge. With this workaround, you can encode any special characters in your spreadsheet and decode them in InDesign again.

 

In your spreadsheet:

  • Prepare all your data as usual (including Emojis and line breaks)
  • Add a second sheet to your spreadsheet
  • Use formulas to fill it, and use ENCODEURL() to encode every special character in URL notation (spaces will turn into %20)
  • Export the result as .csv and make sure to convert it to UTF-16 character encoding for InDesign (using a text editor or LibreOffice)

Capture.PNG

 

In InDesign:

  • Open a text editor, copy the below script into it, and save it as URL-decode.jsx

  • Move the script file to your scripts directory - on Windows this is: %AppData%\Adobe\InDesign\Version 14.0\en_GB\Scripts\Scripts Panel

    (you can type %AppData% into the Windows search box, press enter and navigate to the folder from there)

  • Use Data Merge as usual and import the .csv as data source
  • In the preview, you will see the URL-encoded characters
  • Click the "Create Merged Document" button
  • The resulting file will still contain all the URL-encoded characters
  • From the scripts panel (Window -> Utility -> Scripts), run the URL-decode.jsx script
  • Voilà, all your special characters are back!

 

URL-decode.jsx:

//DESCRIPTION:Decodes and replaces all URL-encoded strings in the active InDesign document.
//AUTHOR:Johannes Eichberger
//
//Example:
//    This%20is%20URL-encoded%20text!
//    This is URL-encoded text!
//
//How to use:
//    Open an InDesign document with URL-encoded texts.
//    Run the script.
//    It replaces every (!) text in every text frame.
//
//Limitations:
//    Works with text frames that contain text only. Any non-text
//    content, such as tables and linked images, will get lost.
//    If an encoded text exceeds its container (overset text),
//    only the visible part will be replaced. You can run the script
//    multiple times to solve this issue.
//
//When to use:
//    InDesign's Data Merge has only poor support of character encodings.
//    If you want to feed InDesign with Emojis, you can encode them in the
//    spreadsheet via =ENCODEURL() and decode them using this script
//    after running Data Merge.
//

#target indesign;

function main() {
    try {
        var myDocument = app.activeDocument;
        var myTextFrames = myDocument.textFrames.everyItem();
        var myContents = myTextFrames.contents;
        for (var i=0; i<myContents.length; i++) {
            var thisTextFrame = myDocument.textFrames.item(i);
            var myDecoded = decodeURI(myContents[i]);
            thisTextFrame.contents = myDecoded;
        }
        if (i==0) {
            throw "No texts found!";
        }
        alert("Texts have been replaced.");
    }
    catch(err) {
        alert(err);
    }
}
main();

 

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 31, 2019 0
Adobe Community Professional ,
Jul 31, 2019

Copy link to clipboard

Copied

Hi joheiat ,

FWIW: be aware that your script could change the contents of text frames.

Before, text frame with one anchored object, the yellow rectangle:

BeforeRunningScript-ANCHORED-OBJECT-PRESENT.PNG

After running URL-decode.jsx:

AfterRunningScript-ANCHORED-OBJECT-REMOVED.PNG

Also a problem. Tables. You'll remove tables.

Before:

BeforeRunningScript-TABLE-PRESENT.PNG

After running URL-decode.jsx:

AfterRunningScript-TABLE-REMOVED.PNG

Regards,
Uwe

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 31, 2019 0
Community Beginner ,
Jul 31, 2019

Copy link to clipboard

Copied

Hi Uwe / Laubender,

Yes, you're right. The script is on a very basic level and only works for text frames which contain nothing else but text. I might think about extending it to keep additional content placed in text frames too. Thanks for pointing me to this!

If anyone has a demand for this, please leave a reply to nudge me.

Also, feel free to extend the script.

Regards,

Johannes

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 31, 2019 0
New Here ,
Oct 18, 2019

Copy link to clipboard

Copied

Hi Joheiat,

 

I'm actually in need of exactly this — a better process for data-merging with emojis. However, when I run your script to decode the url-encoded text, I get an error message. Is the script you posted here accurate and up-to-date? Thanks!

 

Steven

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 18, 2019 0
Community Beginner ,
Jul 23, 2020

Copy link to clipboard

Copied

Hi Steven,

 

So sorry, I happened to see your reply only now. Are you still in the need of this? I tested my script again with the current version of InDesign with a minimalistic test file and it still works. Which error message do you / did you see?

 

Regards,

Johannes

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 23, 2020 0
Adobe Community Professional ,
Jul 23, 2020

Copy link to clipboard

Copied

Hi Johannes,

the script code in your correct answer was damaged last year when this thread was moved to the new InDesign forum.

That could be one cause of an error message. Look into the for loop. At first glance I see that the iterator [i] was removed on myContents. Best correct your answer and post the original code again.

 

Thanks,
Uwe Laubender

( ACP )

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 23, 2020 1
joheiat LATEST
Community Beginner ,
Jul 23, 2020

Copy link to clipboard

Copied

Hi Uwe,

 

Thanks for the hint. I've updated my answer.

 

Best,

Johannes

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jul 23, 2020 0