ExtendScript libraries

[frameexpert]

Hello All,

I want to create a library of commonly used functions, etc. that I can use in my ExtendScript scripts. I created a script that I put in the JavaScript startup folder that has this content:

var CPF = {};
CPF.getText = function () {
    var text = "";
    return {
        getText: function (textObj) {
            // Get a list of strings in the text object.
            var textItems = textObj.GetText(Constants.FTI_String);
            // Concatenate the strings.
            for (var i = 0; i < textItems.len; i += 1) {
                text += (textItems.sdata);
            }
        }
    };
}

Now when I want to get the text of a text object, I can use this:

alert (CPF.getText(pgf)); // pgf is a paragraph.

One important note: I am using ExtendScript with FrameMaker here, not InDesign, so the code itself may look unfamiliar. I am posting here because the principles should be the same in InDesign and there are a lot of smart people here :-). I have been looking at JavaScript libraries and reading about good design patterns, but I am having a hard time translating this to the ExtendScript environment.

My questions are: Is this the best way to do this kind of thing? Or, is there a better way to have a library of functions that can be used in my scripts? Thanks in advance.

Rick

[Peter Kahrel]

You could include your libraries:

#include mylibrary.jsx

Peter

[frameexpert]

Hi Peter,

Thanks for your reply. I was asking more about how to structure the library and functions, and how to call them. Also, if I put them in the startup folder, then I shouldn't have to include them. Thanks.

Rick

[Marc Autret · Accepted solution]

Hi Rick,

Here is the general design pattern I use for any library or sublibrary I create in ExtendScript:

$.global.hasOwnProperty('MyLibrary')||(function(HOST, SELF)
{
    HOST[SELF] = SELF;
    // =================================
    // PRIVATE
    // =================================
    var INNER = {};
    INNER.myPrivateData1 = "foo";
    INNER.myPrivateFunc1 = function(){ /* . . . */ };
    /* . . . */
    // =================================
    // API
    // =================================
    SELF.myPublicData1 = 123;
    SELF.myPublicMethod1 = function(){ alert(INNER.myPrivateData1); };
    /* . . . */
})($.global,{toString:function(){return 'MyLibrary';}});
// Usage
// ---
MyLibrary.myPublicMethod1();

I came up with that general structure after a number of tries and approaches. This one is the most convenient to me for several reasons:

• It supports persistent engines (i.e. does not uselessly re-create the library from scratch).

• It supports to be included, and still works as jsxbin.

• The body uses a single 'closure' variable (INNER), where I can load any required private data or func.

• The same pattern can be used to include a submodule within an existing one: just adjust the HOST argument to your needs.

• I can change the name of a library very easily: just replace the two occurences of 'MyLibrary' by the new name.

• Finally, that pattern is very generic and uniform (based on HOST, SELF and INNER). All modules I've written so far are now implemented this way: DOM helpers, ScriptUI stuff, data manipulation, and so on.

@+

Marc

[frameexpert]

Fantastic, Marc, thank you very much!

Do you recommend using "include" or putting it in the startup folder? Thanks.

Rick

[Marc Autret]

frameexpert wrote:

Do you recommend using "include" or putting it in the startup folder? Thanks.

Rick

It depends on what you are doing. Personnaly I tend to avoid loading startup scripts unless there is a serious reason to (e.g. menu manager).

@+

Marc

[Harbs.]

I use a slightly less structured approach similar to what Marc uses. The main difference is that I version my library and reload it if the version is higher than the old one. This allows easily reloading the library in a persistant engine after making a bug fix.

Here's the idea:

var curVersion = 2.00326;

if(typeof HarbsLib == "undefined" || HarbsLib.version < curVersion){

[Justin Putney]

I liked @Harbs' idea with the versions (and was in need of it).

I also really like @Marc_Autret's library setup.

If you replace the top line of Marc's library template with the following, it

1. adds version checking (even versions of the library with the old format that lack the version property) and
2. means only writing 'MyLibrary' once at the bottom.

(function(HOST, SELF) {
    var VERSION = 1.01;
    if(HOST[SELF] && HOST[SELF].version > VERSION) return HOST[SELF];
    HOST[SELF] = SELF;
    SELF.version = VERSION;

[Dirk Becker]

Beware, in ExtendScript closures produce memory leaks. This is less of a problem with single instance libraries but if you frequently create new objects, or with persistent sessions in long term running InDesign Servers, you'll produce multiple objects per instance (add a "workspace" scope object for your private variables) and a copy of each method also per instance. See the output of $.summary() for details.

Besides the closures are a pain to debug, because those "private" variables hidden to the outside are also hidden from the ESTK debugger.

Finally, performance may be an issue to you. Do some measurements and you'll be surprised how much overhead it is to invoke getters and setters, over straight variable access. Even though in other languages I also prefer them for better programming style, each additional function in big objects - and a getter and setter pair counts for two - will also cause a speed penalty just by their existence.

Dirk

[Jongware]

Dirk, $.summary()? It doesn't appear in http://jongware.mit.edu/idcs6js/pc_$.html ...

[Harbs.]

That's right. There's a number of undocumented methods...

Check out $.reflect.methods

[Dirk Becker]

Yes, you can't always trust the omv.xml . For example I refer to that xml to produce a Java (rather than JavaScript) binding. Just past hour I found after attempts to debug my java generator code that in the OMV, only since after CS4, XMLElements is missing the method itemByName even though ExtendScript still supports it. I also have trouble with the ambiguity of argument and result types.

[Marc Autret]

Thanks Dirk,

I'm very careful with memory leaks and I often check the $.summary() report. Indeed each library will add one (workspace) as soon as functions are loaded in the related closures (i.e. the INNER local variable and the SELF argument). That's the cost of the pattern, I agree, but it doesn't seem prohibitive so far, since each module is loaded once per session and never duplicated. About performance issues, I've always found they were negligible compared to DOM access time and ScriptUI LayoutManager intrinsic latency! However, when a INNER.xxx() or a SELF.yyy() routine needs to be invoked within a huge loop, you are right that the slowdown is noticeable. In such case, it's always better to create in the client code a local func variable that references the required method, then to call func() from within the loop.

In the vast majority of situations my modules just encapsulate helpers, algorithms and everyday routines. They behave as pure singletons, do not re-instantiate inner objects and usually need to remain available to the client script during a whole #targetengine session. So I simply do not kill the related workspaces. But it is also possible to entirely remove a specific module (and relax the corresponding workspace) when its API is known to be accessed once per session. I then provide an unload() method having the following form:

// generic unload pattern
// ---
SELF.unload = function()
    {
    var k;
    for( k in INNER )
        {
        if( !(INNER.hasOwnProperty(k)) ) continue;
        INNER=null;
        delete INNER;
        }
    for( k in SELF )
        {
        if( !(SELF.hasOwnProperty(k)) ) continue;
        SELF=null;
        delete SELF;
        }
    INNER = SELF = null;
    };

After invoking myLibrary.unload(), $.summary() should show that the (workspace) is properly released. (There are some exceptions though, especially when we are extending ScriptUI objects, but after many many helpless $.gc() I definitely suspect ScriptUI to have built-in memory leaks 😉

> "private" variables hidden to the outside are also

> hidden from the ESTK debugger.

Well, this is a serious point to consider for those who use ESTK. (I do not.)

@+

Marc

[Harbs.]

Marc Autret wrote:

However, when a INNER.xxx() or a SELF.yyy() routine needs to be invoked within a huge loop, you are right that the slowdown is noticeable. In such case, it's always better to create in the client code a local func variable that references the required method, then to call func() from within the loop.

Nice.

[Dirk Becker]

Marc, Harbs,

local variables are faster indeed. I frequently use them for quick access to such libraries as in this thread, and that speeds things up severely in my case with hundreds of classes. I also have moved a large share of other classes into their own namespace object for the same reason, in order to relieve the global namespace. Besides if I do cross-library calls frequently, I also store a pointer to that other library as private member of the using library, just to not go thru the globals object.

[yonatanl25270460]

Marc, can you please explain  what you mean by:

However, when a INNER.xxx() or a SELF.yyy() routine needs to be invoked within a huge loop, you are right that the slowdown is noticeable. In such case, it's always better to create in the client code a local func variable that references the required method, then to call func() from within the loop.

What is the "client code" ? Can you give a brief example of "problematic code" and the recommended way to  code it ?

Thanks

[Marc Autret]

Hi yonatan,

The "client code" is the program which relies on (and uses) the library or framework, assuming that the library/framework provides a general API usable in different projects. For example, if your library implements a String module which encapsulates various string oriented routines (trim, md5, base64, who knows…), then you can create a client script which can invoke those routines for its own purpose, e.g. a script that cleans up text frames and will use MyLibrary.String.trim.

Now, about “always better to create in the client code a local func variable that references the required method”, this is in fact a very general trick that may improve performance in some cases. Suppose you have, in the client code, a function that massively involves MyLibrary.String.trim, typically a complicate deep loop with conditions and so on, where MyLibrary.String.trim(xyz) is literally invoked again and again. This may have no noticeable side effect if ExtendScript's engine makes good assumptions and is well optimized, but as we aren't absolutely sure of that, better is to suppose that resolving "MyLibrary.String.trim" takes some time to the interpreter (find MyLibrary in the [[global]] scope, find the key 'String' within MyLibray, find the key 'trim' within MyLibrary.String). So the idea is to resolve it once before entering the loop:

const trimFunc = MyLibrary.String.trim;

and invoke the local trimFunc reference instead of repeatedly resolving the whole path. Note that the trick applies to native references as well, e.g. const chr = String.fromCharCode.

I don't know where and when local function references make huge performance gaps. (I just know they have no performance cost!) My guess is, it makes most sense to use that trick if the remote function is ultra-fast, because the time it takes to access to it becomes comparable to the time it takes to run it.

@+

Marc

[yonatanl25270460]

Thanks for a very clear answer

[DBarranca]

Marc,

fashinating pattern! Could you please explain how the immediatly-invoked function expression works?

I guess the thing that I don't get is how the object literal you pass as the parameter is involved. Basically the

HOST[SELF] = SELF;

translates in

$.global[{toString: function(){ return 'myLibrary'}}] = {toString: function(){ return 'myLibrary'}}

and eventually 'myLibrary' is a property of $.global - why, is a bit over my head.

Thank you!

Davide Barranca

[Marc Autret]

Hi Davide,

You are very close to the answer. In fact, the assignment

HOST[SELF] = SELF;

is a shortcut of

HOST[SELF.toString()] = SELF;

because SELF is an Object and then must be coerced to a String when it takes the place of a key in an associative array. That's it.

Finally, the assignment leads to HOST['myLibrary'] = SELF—i.e. $.global.myLibrary = SELF—in the specific case I've illustrated.

What is practical in the pattern HOST[SELF] = SELF is that the particular name of the library is not explicitly used here, which makes the code more generic.

@+

Marc

[DBarranca]

Thank you Marc!

Yes, it makes sense and looks obvious... now!

I've got interest in patterns only recently and I've run into this thread - there aren't many similar discussions, at least in the PS ecosystem which I come from.

Kind regards

Davide

[tobiask43046735]

Based on Marc Autret's great pattern, I created a function such that a library can be declared with less boilerplate:

function library(lib, namespace, forceReload) {
    typeof(namespace)=='undefined' && namespace = $.global;
    typeof(forceReload)=='undefined' && forceReload = false;
    lib.toString = function(){return lib.name;};
    if(namespace.hasOwnProperty(lib) && !forceReload) {
        // store a reference to the attempted re-association for possible reloading
        namespace[lib].reload = function(namespace){library(lib, namespace, true)};
        return;
    }
    namespace[lib] = lib;
    lib(lib);
    // and here you can add global things like lib.unload = function(...
}

Now one can just use something like

library(function MyLibrary(SELF){
    var INNER = {}; // if needed
    INNER.myPrivateData1 = "foo";
    // ...
    SELF.myPublicData1 = 123;
    // ...
}

i.e. you basically pass a singleton constructor to the library function. Now it becomes trivial to globally change the library interface, and as an added bonus you can declare sublibraries by passing the parent as namespace parameter. And for development there's the forceReload parameter or the lib.reload() function.

Now if only there was a way for library to figure out the name of the script which called library, you could even pass it an anonymous function in a Python-like one file per library (or module) approach...

[yonatanl25270460]

Excellent Marc - just what I was looking for !!In my version. I put the various my* declarations and the call to /MyLibrary.myPublicMethod1();   in a comment, so that I can since I invariably forget to remove them......