Multiple record data merge into paragraph styles-applies the wrong style

[LinaD36]

Hi, I've been working on this project for sometime and everytime I manage to get one part of the workflow to work another seems to break. My agency publishes catalogs in multiple formats: large-print, audio, braille, and HTML. I've been trying to redesing our work process so that the catalogs will be laid out from merged data out of comma-separated file. The data merges have worked fairly well in Word, but InDesign is a challenge. I'm merging multiple records on a page, like a mailing label. The paragraphs need to be formatted and I'm trying to apply a paragraph style to them. After much work, I think I've finally got the data merge to work correctly, but the wrong paragraph styles are applied. I'm going to apply a new master page to the data once the data is merge that uses the paragraph styles for text variable running headers and I need to build a table of contents based on the paragraph styles so I need this to work. Attached are some screenshots.

Master page set up for data merge with paragraph styles:

Here is the merged document with the wrong paragraph styles applied:

I have only a few weeks to get this process ironed out to keep to our rigorous production schedule. If anyone can help I would really appreciate it.

thanks,

Lina

[LinaD36]

Hi I am getting together the pictures for posting and trying to run the script the Uwe posted to get rid of the XML tagging. I will also check my extenstions, but there should be no 3rd-party extension running since we receive this license from the Library of Congress server downtown and they are very strict about not letting install any third-party software that hasn't been thoroughly vetted. I just hope to be able to run the script that removes the XML tagging.

Lina

[LinaD36]

And we have success! Wow this has been a long process thanks so much to everyone for their help. Uwe was right it was the XML codes. I've cut and paste the whole process below and will check my extensions. But I had two minor requests related to things that have been pointed out to me in previous posts:

1)Mike--You had mentioned there is a script for joining all the merged data into one story. I would like to do this. Remind me again where the script is?

2) I was going run a simple find/replace after the data was merged to put a space between an article like "The" or "a" in the title and the title, but I find that I'm a complete idiot with the Find/Replace tool. Has someone already done this and can show me how to do it?

This is the process that worked! I am running InDesign CS 6 Version 8.0 on Windows 7 Enterprise, Service Pack 1. Thanks to all of our your help:

1. I created a new data merge document in the Master page and applied paragraph styles to the merge fields:

2. I made sure that I was going to perform the data merge with blank lines inserted where there were empty fields:

3. Creating a new merged document:

4. I specified that the data would be laid out in columns:

5. I created the New Merge document:

6. Here is the new merged document:

7. I placed the javascript that Uwe created in my script panel and ran it. It got rid of the XML tags!

8. I ran the text search:

9. Then the GREP search got rid of the extra merge fields!

I am over the moon! I really can't thank you enough.

Lina

[MW Design]

One such script is here:

http://www.loicaigon.com/en/indesignsecrets-presents-our-inlinemerge-script/

Another, requires a plug-in installed, and so may not be appropriate in your circumstance is:

http://www.rorohiko.com/wordpress/indesign-downloads/textstitch/

You need to follow Rorohiko's installation procedure explicitly.

[Peter Spier]

There pretty much has to be a plugin doing the XML tagging...

1) The script for stitching together the stories is at Rorohiko.com

2) Make a copy of the file and try the following GREP:

Find ^(The|A)(?=\u) with the Find format set to the Title style, then change to $0<spacebar>

But even easier, if you want to edit the template, would be to add the space in the merge template between the article and the title, then the blank space would be eliminated with the ^\s+ query if there is no article.

[Peter Spier]

MW Design wrote:

One such script is here:

http://www.loicaigon.com/en/indesignsecrets-presents-our-inlinemerge-s cript/

Thanks, Mike, I didn't know about this one. Looks like it runs instead of the regular Data merge, though, so it Lina would have to redo her merge (but then she can add that space). Very interesting. Might get rid of the XML, too.

[Peter Spier]

BUT, it's not quite the same thing as the Rorohiko script, from what I'm reading (have you run it?). Looks like it makes a continuous flow of anchored objects, rather than a true text thread, so no record can break across columns or pages.

[MW Design]

No never have run it. I had it stored in my shortcuts after finding it. I'll give a whirl later. I have been using up time this morning without being productive. Got the flu, ain't too excited about work.

Not certain about the Rorhiko solution (which I have used many times) if Lina has issues installing the plug-in due to her work contraints.

It is just really strange about the XML tagging, including the other thread Uwe points to where it happened. Which is why attempting to find the source of the issue may well provide a smoother work-flow down the road.

[LinaD36]

Hey Peter I did check my extensions and the only ones that are installed are the Adobe extensions that came with InDesign CS 6 version 8.0.0.370. I had originally put the space in the template but I thought it would be harder to search for the empty space at the beginning of the line because (there are so many spaces in the publication) than for the relatively few instances where The and A start the title.Can you send me a picture of what the query is supposed to look like? I can't seem to run it in InDesign (as I said I seem to be completely inept about these things). Sorry.

Lina

[MW Design]

Current CS6 version is 8.0.2.413

I doubt that affects the merge issue, but...

[Peter Spier]

Sorry, my fault. <spacebar> means press the spacebar after the $0. whitepace doesn't show up well in a screen capture, and it's ambiguous if you type it here in the forum.

Mike has a point about the Rorohiko plugin. That may be a problem for you.

Any chance you could export a few records to a sample file, two or three would be enough, and do nothing more to the sample before giving us a link to it? If there's a plugin involved, it would show up as missing here when I try to open the file, even if it isn't in the extension manager.

Also, can you go to Help and hold down the Ctrl key while you choose About InDesign? Waht's the version number listed in the upper left of the dialog that pops up?

[MW Design]

Ran the loicaigon.com script. It does replace the native merge function.

Two issues.

(1) It runs within the merge document. In other words, it doesn't do its thing going into a new, merged document. Easy to overwrite a template.

(2) Suffers the same fate as ID's merge when removing blank lines. So in Lina's merge, the Title paragraph has the correct appearance applied as an override to the Genre paragraph style.

Mike

[Laubender]

@Lina, Mike and Peter – all well so far…

I'm glad this all is working for Lina.

Next step: Stitch all those text frames in the merged document.

That would be one thing.

I think it might be better for Lina, if all the texts in all the text frames would go in an well ordered way into a new story.

For Lina's special case (a freshly data merged document), it should be easy writing some lines of code to do so (both, the stitching and/or moving the texts of all text frames to a new story).

The basic idea behind this is:

Presumption:

In case of a data merge, the text frames are generated one by one in a regular fashion.

From left to right, from top to bottom, from page to page.

This means ( hopefully ) that the ID numbers of the text frames (that could be read by scripting) can be represented in an ordered array (a list). And that would be the key to thread them together or to move the texts together: you always know what text frame should be the next one by its ID number.

Let's see. First I need some evidence for this…

Oh: Inserting a blank paragraph at the end of each text frame is no problem, too.
You could do that with a GREP search/replace. Formatting as well.

Uwe

[LinaD36]

Peter,

I really all thumbs with this I can't get the GREP search to run. What have I typed in wrong?

[Laubender]

@Lina – I'd say: first get rid of the XML tags, then do a GREP search/replace…
Not tested, but the tags could distract that process.

Uwe

[Peter Spier]

I believe that is the case. Those tags seem to interfere with any find/change operation.

[LinaD36]

Hey Uwe,

It doesn't work either way. I get the same result if I run it on a file that is XML-tag free:

[Peter Spier]

That's really strange. I've run it twice here and it works fine -- 148 records changed. But I don't have the XML tags at any point...

There's no empty space after \u is there?

[LinaD36]

Hah! That was the key. There was an empty space after the \u. It works now. Bless you!

Lina

[Peter Spier]

That's what I meant about the ambiguity of empty space in the forum. It's easy to accidentally copy one that doesn't belong, or not get one that does.

Cheers.

[LinaD36]

Hey Peter,

How would I change this GREP to add the word "An" to the search? ^(The|A|An)(?=\u) ?

I'm checking with my IT folks to see if there's any way for me to download the Rorohiko plugin without having to go through the 6-month administrative process. They haven't given me much hope.

thanks,

Lina

[Peter Spier]

That modified expression should work fine.

And let's give Uwe a bit more time to work on a script. I think he's got an inkling of how to do it for you. A fresh merge is really a simpler case than all of what the Rorohiko script is designed to do, so it may not be too much work.

[LinaD36]

Wow. I guess I hadn't clued into the idea that Uwe was work on another script for me. You guys are the best. I feel so fortunate to have so much help!

Lina

[Peter Spier]

Yeah, it doesn't look like he mentioned it here, but he did say something last night in the other thread with a user having problems with XML tags, so there's some hope.

And worst case is you do it manually. It's boring, but simple.

[Laubender]

@lina – maybe you already did it manually…

But, here I'm back with a script, that is able to thread all the text frames of the datamerge together.

Run it AFTER you eliminated: the XML-tags, the <FEFF> special characters and the empty paragraphs.

A fair warning: It's not totally "watertight", but should work, if:

1. You want to thread all text frames on pages in a document by order of their creation; page by page!

Here I simply assume, that the datamerge is generating the text frames page by page from left to right, top to bottom.

So check the results!

2. There are NO text frames, that are already threaded together.

As a bonus I add a paragraph style to the document that is applied to a separator paragraph between the texts of the text frames. I named it "STORY-SEPARATOR". You could change its name if you want in line 10 of the script. Just make sure you'll leave  both quotation marks intact.

As another bonus you can undo the script's action in one go.

But as always: save the result to a different file name!

//StitchTextFramesAfterDatamerge_IN-CREATION-ORDER.jsx

//Uwe Laubender

//DESCRIPTION:Does NOT work, if some text frames are already stitched (threaded) together!!

/**

* @@@BUILDINFO@@@ StitchTextFramesAfterDatamerge_IN-CREATION-ORDER.jsx !Version! Thu Dec 05 2013 23:00:25 GMT+0100

*/

var storySeparatorParaStyleName = "STORY-SEPARATOR";

app.scriptPreferences.userInteractionLevel = UserInteractionLevels.interactWithAll;

try{

app.doScript(_StitchTextFrames, ScriptLanguage.JAVASCRIPT, [], UndoModes.ENTIRE_SCRIPT, "Stitch text frames of document together: page by page.");

}catch(e){alert("Sorry, something went wrong:"+"\t"+e.message);exit(0)};

alert("DONE! New paragraph style named \""+storySeparatorParaStyleName+"\" added. Text frames on pages (not on the pasteboard) are threaded page by page in creation order!");

function _StitchTextFrames(){

var d=app.documents[0];

if(!d.paragraphStyles.itemByName(storySeparatorParaStyleName).isValid){

    d.paragraphStyles.add({name:storySeparatorParaStyleName});

    };

d.textFrames.everyItem().insertionPoints[0].contents = "\r\r";

d.textFrames.everyItem().paragraphs[1].appliedParagraphStyle = storySeparatorParaStyleName;

var firstTextFrameID = threadTextFramesInBuildingOrderPageByPage(app.documents[0]);

d.textFrames.itemByID(firstTextFrameID).characters[0].remove();

d.textFrames.itemByID(firstTextFrameID).characters[0].remove();

//FUNCTION

//Returns ID number of first built text frame by lowest number !!

function threadTextFramesInBuildingOrderPageByPage(myDoc){

    var d=myDoc;

    var myPages = d.pages;

    var IDArrayByPages = new Array();

    var n, m, currentTF, nextTF;

    for(n=0;n<myPages.length;n++){

        if(myPages.textFrames.length >0){

            IDArrayByPages = IDArrayByPages.concat(myPages.textFrames.everyItem().id.sort());

            };

        };

    //Stitch together:

    for(m=0;m<IDArrayByPages.length-1;m++){

        currentTF = d.textFrames.itemByID(IDArrayByPages);

        nextTF = d.textFrames.itemByID(IDArrayByPages[m+1]);

        currentTF.nextTextFrame = nextTF;

        };

    return IDArrayByPages[0];

    };

}; //END function _StitchTextFrames(){}

Uwe

NOTE TO MYSELF: Code is damaged because the thread was moved from the old InDesign Forum to the new one by the end of 2019.

[LinaD36]

Hey Uwe,

No, I have not done it manually, because I am overwhelmed with editing projects and really don't have time to spend hours manually formatting my publications. Don't know how to thank you! I will run it through my administrator and give it a try. You're a godsend.

best,

Lina