Why is manual link folder replacement faster than script in InDesign?

@Aprking this is nothing you don't know, but I realised we have not explicitly answered your main question:

> Why is manual link folder replacement faster than script in InDesign?

The unsatisfying answer is: native features are always faster, in terms of raw speed, than going via the scripting API, which introduces performance overhead, sometimes quite significantly. The "manual link folder replacement" you mention relies on a native update links feature so it is faster.

We have given the topic a good workout on this thread though.

- Mark

Hi @Aprking and everyone,

Well, I'll mention this because it hasn't been discussed, and I'm not sure if it's sensible or not, but I just did a proof-of-concept test of editing the links as plain text via .idml file. I'm using MacOS so you will have to replace command line tools with Windows equivalents (eg. 7zip?).

Disclaimer: I know hardly anything about IDML files! What worked for me may not work in your case for reasons I didn't consider, so, for example, in step 4 your files might be differently-named or -structured. Thanks to @Peter Kahrel for teaching me recently that IDML files were .zip files.

What I did:

1. export demo.idml

2. make a folder called "expanded" in the same directory as 1.

3. decompress idml contents into the "expanded" folder—in a terminal: unzip demo.idml -d expanded

4. opened "expanded/Spreads/Spread_uce.xml" in VSCode (is this always the same file?)

5. performed find/replace targeting the link file paths (this would probably match the string replacement you did in your script).

6. save and close Spread_uce.xml file.

7. re-compress the .idml file—in terminal again: cd expanded; ls | zip -@ -r ../demo2.idml (be careful with this step, the re-zipped file must have the exact same structure as the original, eg. the default behaviour of zip is to compress with a root directory—which we don't want, and Indesign won't open an IDML with an unexpected structure).

8. open demo2.idml (created in step 7).

9. the links paths will be now changed but they won't be updated (similar to using reinitLink), so you must ask Indesign to do that in one operation.

So that's the idea—good or bad. It rests on the fact that VSCode is very fast at performing the find/replace. However step 9 (updating modified links) will be slow no matter what, so ultimately this may not be so fast for you. Also if this is something you want to fully automate, then this is probably too much of a stretch. Still it adds to the discussion. 🙂

- Mark

Calling your script from another script with an appropriate undo mode might be worth a test.

P.

===

It doesn't need to another script, just the function.

Final statistical report: Document size 752,016KB, 365 pages, with 40 links per page, totaling 14,595 links. As per guidance from @Peter Kahrel , among these, 729 are actual links, while the remaining 13,866 are pointers.

=======================================================================

Using @m1b  method, due to the ineffectiveness of directly using link.update() in the code, the reinitLink method was first employed to change the links, taking 26.133s. Then, updating the links using link.update() in a subsequent loop took 295.888s, totaling 322.021s, occupying 1GB of disk space. Since direct use of link.update() in the code wasn't possible, the method had to be abandoned after two script executions. Although this method was fast, I couldn't fully grasp the technical details, rendering it impractical.

=======================================================================

I employed a method of splitting links.length:

var links = doc.links.everyItem().getElements();
var batchSize = 100; 
var totalLinks = links.length;
for (var i = 0; i < totalLinks; i += batchSize) {
    var batchLinks = [];
    var endIndex = Math.min(i + batchSize, totalLinks);

    for (var j = i; j < endIndex; j++) {
        batchLinks.push(links[j]);
    }
    
    for (var k = 0; k < batchLinks.length; k++) {
        var link = batchLinks[k];

This method took 275.817s and occupied approximately 1GB of disk space.

=======================================================================

Using @rob day  code:

var p = app.activeDocument.pages.everyItem().getElements()

This method successfully reduced the processing time from over an hour to 280.275s and occupied approximately 1.1GB of disk space.

=======================================================================

@rob day  solution had the shortest processing time and minimal disk space usage, making it the champion and the most technically superior solution.

=======================================================================

My solution is experimental and requires extensive testing to ensure there are no bugs. However, its advantage lies in its flexibility in splitting links.length and adjusting batchSize according to different document requirements.

=======================================================================

In conclusion, the three solutions differ mainly in specific details, but their techniques are similar, all involving dividing the entire document into smaller chunks for processing. The observed differences in processing time are not stable and should be taken as reference only.

Once again, I extend my gratitude to @rob day @m1b @Robert at ID-Tasker @leo.r  and @Peter Kahrel  for their guidance and participation.

Thank you.

@rob day @Robert at ID-Tasker @leo.r  @m1b @Peter Kahrel This topic may be the most lively in the InDesign community recently. Thanks to all the teachers for their participation. I will test various solutions promptly and provide detailed test data.

> the processing time increases exponentially when it reaches a threshold

I've seen that in other areas as well, e.g. adding hyperlinks. After creating about 500 links InDesign starts to slow down dramatically. There's not really very much that you can do about it.

@Aprking :

> What could be the reason for this significant difference in speed?

Scripts will always be slower for these kind of operations than native InDesign actions (as others also mentioned). To match the speed of InDesign you'll need to write a C++ plug-in (while, of course, all the great suggestions for script optimization you received here can help speed it up).

And some changes in the code to reduce the number of actions. If this code doesn't execute faster, at least it reads quicker 🙂

// Add the slash to the path names

var originalFolderPath = "F:/source/";
var targetFolderPath = "f:/target/";

// Work on arrays as much as possible, 
// collections process much slower

var links = app.activeDocument.links.everyItem().getElements();

var updatedLinksCount = 0;
var targetFile;
var regex = /.+[\\\/]/;  // Precompile regular expressions
for (var i = 0; i < links.length; i++) {
  targetFile = File (targetFolderPath + links[i].filePath.replace(regex,''));
  if (targetFile.exists) {
    links[i].relink(targetFile);
    updatedLinksCount++;
  }
}

The first step to speed up the script is to change this line:

var links = doc.links;

to

var links = doc.links.everyItem().getElements();

@Peter Kahrel , @rob day - would you like to pitch in?