Highlighted

Why is this script so slow?

Participant ,
Oct 03, 2020

Copy link to clipboard

Copied

I have a script which iterates through all paragraphs in a document and adds an anchored text frame if the paragraph has one of a specific list applied. In practice, this is to get around the limitation that numbered paragraph numbers are not available for text variables, so running headers that indicate which paragraph/section you’re currently reading can’t be easily achieved. Creating little non-printing text frames with the paragraph number as plain text gets around that.

 

I’ve used the script before with success, but on a book where each chapter (= document in a book file) was quite short, usually no more than 15 pages at the most. It performed fairly well there. Now I’m trying to use it on a much larger book where one chapter in particular is about 160 pages long, and the script is absolutely glacial. I’ve added in a counter/progress bar to see some indication (not shown in the code below), and it’s clear that it starts out all right, but then gets progressively slower as it runs.

 

Here is the script:

 

var doc = app.activeDocument;
var stories = doc.stories;
var styles = doc.paragraphStyleGroups.item("Numbered headings").paragraphStyles;
var paras, p, s, frame, ip, txt;

for (var i = 0; i < stories.length; i++) {
	paras = stories[i].paragraphs;

	for (var j = 0; j < paras.length; j++) {
		p = paras[j];
		s = p.appliedParagraphStyle;

		if (s == styles.item('Heading 1') || s == styles.item('Heading 2') || s == styles.item('Heading 3')) {
			ip = p.insertionPoints.previousItem(p.insertionPoints.lastItem());
			txt = p.bulletsAndNumberingResultText.replace("\t", '');

			frame = ip.textFrames.add({
				appliedObjectStyle: doc.objectStyles.item('Section number for running header'),
				contents: txt
			});
		}
	}
}

 

Quite simple, I should have thought:

 

  • Store the relevant paragraph styles in a variable (basically three heading styles)
  • Iterate through stories, and within each story through paragraphs
  • For each paragraph, check if the applied paragraph style is one of the specified styles
  • If it is, get the insertion point at the end of the paragraph, store the paragraph number in a variable, and add a text frame at the insertion point containing containing the paragraph number

 

This all seems quite trivial to me, and I would have thought InDesign should be able to eat through this like pancakes, but apparently not.

 

The progress thingamajig shows me that it very quickly runs through a few stories that have only one paragraph (probably headers and footers, I’d imagine), then comes to the main story, which it says has 833 paragraphs in that particularly long chapter. The first 50 or so of those take maybe four seconds for it to crunch through, and then it goes downhill from there.

 

I ran the script just before I started writing this post, maybe ten minutes ago. It is currently somewhere around 369 out of 833 paragraphs. The progress bar only updates once every twenty or so paragraphs (not sure why), and it’s shown 369 for at least a minute now, so it’s now taking at least five seconds per paragraph of text.

 

I’ve gone through the code to try to identify things that may be causing this decrease in speed, but my brain has never been particularly well-wired for logic, and if there is a logical error that makes it spawn gigantic variables or nested loops or something, I can’t spot it.

 

So in short: why is the code quoted above so extremely slow, particularly the longer it runs?

Adobe Community Professional
Correct answer by brianp311 | Adobe Community Professional

Yes, definitely use getElements. You can get all paragraphs in one fell swoop outside the loop with:

var paras = app.documents[0].stories.everyItem().paragraphs.everyItem().getElements();

Also, this might be faster for your insertion point: 

 

ip = p.insertionPoints[-2];

 

Also, limit calls to the InDesign DOM by defining your paragraph and object style variables at the start of the script, outside the for loop, ie. var h1 = styles.item('Heading 1'); at the beginning, and if (s ==  h1 ||...) in the for loop.

TOPICS
Scripting

Views

137

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

Why is this script so slow?

Participant ,
Oct 03, 2020

Copy link to clipboard

Copied

I have a script which iterates through all paragraphs in a document and adds an anchored text frame if the paragraph has one of a specific list applied. In practice, this is to get around the limitation that numbered paragraph numbers are not available for text variables, so running headers that indicate which paragraph/section you’re currently reading can’t be easily achieved. Creating little non-printing text frames with the paragraph number as plain text gets around that.

 

I’ve used the script before with success, but on a book where each chapter (= document in a book file) was quite short, usually no more than 15 pages at the most. It performed fairly well there. Now I’m trying to use it on a much larger book where one chapter in particular is about 160 pages long, and the script is absolutely glacial. I’ve added in a counter/progress bar to see some indication (not shown in the code below), and it’s clear that it starts out all right, but then gets progressively slower as it runs.

 

Here is the script:

 

var doc = app.activeDocument;
var stories = doc.stories;
var styles = doc.paragraphStyleGroups.item("Numbered headings").paragraphStyles;
var paras, p, s, frame, ip, txt;

for (var i = 0; i < stories.length; i++) {
	paras = stories[i].paragraphs;

	for (var j = 0; j < paras.length; j++) {
		p = paras[j];
		s = p.appliedParagraphStyle;

		if (s == styles.item('Heading 1') || s == styles.item('Heading 2') || s == styles.item('Heading 3')) {
			ip = p.insertionPoints.previousItem(p.insertionPoints.lastItem());
			txt = p.bulletsAndNumberingResultText.replace("\t", '');

			frame = ip.textFrames.add({
				appliedObjectStyle: doc.objectStyles.item('Section number for running header'),
				contents: txt
			});
		}
	}
}

 

Quite simple, I should have thought:

 

  • Store the relevant paragraph styles in a variable (basically three heading styles)
  • Iterate through stories, and within each story through paragraphs
  • For each paragraph, check if the applied paragraph style is one of the specified styles
  • If it is, get the insertion point at the end of the paragraph, store the paragraph number in a variable, and add a text frame at the insertion point containing containing the paragraph number

 

This all seems quite trivial to me, and I would have thought InDesign should be able to eat through this like pancakes, but apparently not.

 

The progress thingamajig shows me that it very quickly runs through a few stories that have only one paragraph (probably headers and footers, I’d imagine), then comes to the main story, which it says has 833 paragraphs in that particularly long chapter. The first 50 or so of those take maybe four seconds for it to crunch through, and then it goes downhill from there.

 

I ran the script just before I started writing this post, maybe ten minutes ago. It is currently somewhere around 369 out of 833 paragraphs. The progress bar only updates once every twenty or so paragraphs (not sure why), and it’s shown 369 for at least a minute now, so it’s now taking at least five seconds per paragraph of text.

 

I’ve gone through the code to try to identify things that may be causing this decrease in speed, but my brain has never been particularly well-wired for logic, and if there is a logical error that makes it spawn gigantic variables or nested loops or something, I can’t spot it.

 

So in short: why is the code quoted above so extremely slow, particularly the longer it runs?

Adobe Community Professional
Correct answer by brianp311 | Adobe Community Professional

Yes, definitely use getElements. You can get all paragraphs in one fell swoop outside the loop with:

var paras = app.documents[0].stories.everyItem().paragraphs.everyItem().getElements();

Also, this might be faster for your insertion point: 

 

ip = p.insertionPoints[-2];

 

Also, limit calls to the InDesign DOM by defining your paragraph and object style variables at the start of the script, outside the for loop, ie. var h1 = styles.item('Heading 1'); at the beginning, and if (s ==  h1 ||...) in the for loop.

TOPICS
Scripting

Views

138

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Oct 03, 2020 0
Adobe Community Professional ,
Oct 03, 2020

Copy link to clipboard

Copied

Hi,

 

Not a full answer but you could simplify your code a little by removing the loops for stories and paragraphs and using this line and looping through the results

 

var a = app.activeDocument.stories.everyItem().paragraphs.everyItem();
 
Regards
 
Malcolm

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 03, 2020 1
Participant ,
Oct 03, 2020

Copy link to clipboard

Copied

I like using the everyItem() collections when I can, but the trouble with them – if I understand things correctly (and I still find them rather tricky, so perhaps I don’t) – is that once you need to start filtering within the collections, you can’t use them because everything you do gets applied to the entire collection, and I don’t want to add frames to every paragraph in the document.

 

But then again, reading Marc Autret’s brilliant guide to them for the trillionth time, it strikes me that perhaps the slowness is indeed (partly?) due to using dynamic collections, so InDesign needs to interact with the script for each iteration. Using a static array built from …everyItem().getElements() may well be faster. Will have to test and see if it works.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 03, 2020 0
Adobe Community Professional ,
Oct 03, 2020

Copy link to clipboard

Copied

Yes, definitely use getElements. You can get all paragraphs in one fell swoop outside the loop with:

var paras = app.documents[0].stories.everyItem().paragraphs.everyItem().getElements();

Also, this might be faster for your insertion point: 

 

ip = p.insertionPoints[-2];

 

Also, limit calls to the InDesign DOM by defining your paragraph and object style variables at the start of the script, outside the for loop, ie. var h1 = styles.item('Heading 1'); at the beginning, and if (s ==  h1 ||...) in the for loop.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 03, 2020 1
Participant ,
Oct 03, 2020

Copy link to clipboard

Copied

Holy crêpes suzette, that was some difference – it went from still not even being halfway through the 833 paragraphs when I finally got fed up and cancelled the execution, to running through the whole thing in about four seconds.

 

Lesson learnt: interacting with the InDesign DOM is expensive!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 03, 2020 0
Participant ,
Oct 03, 2020

Copy link to clipboard

Copied

Sorry, too eager there. When I got fed up and cancelled after 20+ minutes.

 

Many thanks to both of you. You’ve both helped immensely, though I’ll award Brian the tick for the more complete answer.

 

(Aren’t we supposed to be able to edit posts here? The “… More” button just brings up a tiny, empty, white square for me at the moment.)

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 03, 2020 0
Participant ,
Oct 03, 2020

Copy link to clipboard

Copied

There’s a slight issue caused by working with a static array rather than a dynamic collection: the insertion point slowly moves backwards as you go on.

 

Insertion points refer to (points between) characters, which essentially just have an absolute numeric offset in the story (?) as a whole, so the last insertion point in each paragraph is really just a reference to point between two characters with particular indexes. Since adding an anchored frame also adds a character to the paragraph, the index of all the characters after the anchored frame is increased by one for every iteration in the loop – so every time you move on to the next paragraph, your insertion point is no longer at the end of the paragraph, but n+1 characters further back from it. Eventually, if you have enough headings to work through, it will move back to the preceding paragraph and the frame will end up in the wrong place.

 

The solution to this is quite simple: start at the end and work backwards. Instead of starting the loop like this:

for (var i = 0; i < paras.length; i++)

– just start like this instead:

for (var i = paras.length - 1; i >= 0; i--)

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 03, 2020 0