workflow design

[_Andrew-Hall_]

I'm interested in various approaches to controlling workflow with a view to conserving system resources.

Say you have a typical situation where you are going to collect a list of files in bridge and then process that list in Photoshop.

Imagine that the number of files could be quite large, 1000, even 10,000.

There are various ways of working with this. Easiest is probably (I):

1. Collect files in Bridge.
2. Send them over as myFiles.toSource() in a string and regenerate them as soon as they arrive via BridgeTalk.
3. Process targets one after the other in PS.

Given File Objects are more complex than a string, the next variation (II) would be the same only send the files as just the fullName string, and regenerate one at a time in PS for each iteration. This is pretty efficient as it is just a moderately long string being held in memory most of the time.

A quite different approach is to collect the file addresses in Bridge and send them over for processing in PS one at a time (so the for loop of myFiles is in Bridge instead of Photoshop). This is very efficient since at best Bridge need only have one file Object at a time. But it does require more two-way interaction between Bridge and PS as Bridge has to be told - 'ready for the next file' - I guess that is equivalent (but much simpler) to processes within Bob's Open-Close script which show a progress-bar in Bridge. It may be that the processes required in having Bridge running a JS and PS running one at the same time, with one waiting for the other, may end up being just as demanding as (II) where the Bridge script ends once it has sent the files over.

More generally I am interested if use of system resources is something for script writers to be concerned about (when they are not doing processing of obvious very high resource demand such as attempting to work with image pixels one at a time). On the one hand, compared to the crunching PS must do in running a complex operation, JS's seem very minor, but on the other hand when you have a big script running that has geenrated a large number of variables, there's an aweful lot of stuff there in the ESTK sometimes.

Andrew

[_Robert_Stucky_]

We could talk on this topic for weeks....

My preference is the way I did it (obviously!). But I do want to point out that it's only my opinion, there is no definitive answer to this.

If you do some heavy lifting in Bridge, you'll notice that the screen will freeze, eventually going white. Some time after I wrote the import from camera script, we figured a way around this using scheduled tasks.

This really isn't that much of a problem in the point apps as they are essentially modal when scripts are running, but Bridge is not.

In general, I always attempt to write scripts in a manner that reduces demand on system resources. Sometimes I succeed too.

As for the methods you laid out for discussion:

Method 1,

On large bunches of files, it takes significant time to get them ready to send to PS and to regenerate on the other side.

For example - on my machine, it takes 237ms to "toSource()" an array of 1000 files, and 415ms to regenerate the array.

At 10,000, it's 5406ms and 4116 ms respectively.

Obviously it's not linear. Sending fewer files is better than sending large blocks of files, albeit you have to be talking very large numbers of files before it gets significant.

With a large number of files, it behooves us to show something that looks like a progress bar. If you do that in one long PS process, you'll find it's much more difficult to implement, and there's no way to cancel the operation in PS once it's started except by a force quit. Even if you set up progress messages (like BridgeTalkLongProcess in lib1), you'll find that PS, ID, AI, and GL do NOT process the onResult handlers from your progress messages until after the entire script is executed. So any cancel message will only be delivered after the entire script runs.

Method 2:

I don't think there's any real advantage to this one, but I could be wrong. There is the gotcha on having to encode and decode the file names on either side to make sure special characters and higher order unicode characters get evaluated correctly.

Method 3:

BridgeTalk messages are actually pretty quick and have little impact on system resources. As I might have mentioned, recently I've been sending BT messages on the order of 250K - 500K in size. There's no real noticable impact on the system or on Bridge performance and response while messages of that size are being sent.

Since PS will only process 1 at a time, there's little performance hit by sending them one at a time. BridgeTalk.send() is asynchronous, so bridge isn't frozen while you send messages. On Bridge, the onResult and onError handlers are actually kicked off in new threads, so you can do the one at a time, chained thing, without losing the ability to use bridge.

If you look in the libraries and use the BridgeTalkIterator object, like the Open/Close script does, it will even put up a progress bar for you, and report specific errors on each individual file back to the user.

You can even kick off more than one BridgeTalkIterator at a time, it will end up interleaving the execution of both iterators in PS (so both will march towards completion together).

And now the new news...

We have just (today) received word back from legal that you can definitely ship the libraries with your scripts. There is new language being generated as we speak for the license information in the top of the library files.

So you can use BridgeTalkIterator, and you can distribute the libraries.

We'll post the language when it becomes available, and ask you to update the headers of the library files with it. (rather than ship out new copies of the libraries to everyone).

Bob
Adobe WAS Scripting

[_Andrew-Hall_]

Thanks Bob, lots of food for thought. Do you consider your method akin to Method 3.

I find that PS scripts generally halt after a few Esc's even when they are not opening dialogs.

I don't generally go for progress bars in bridge - though I do sometimes want to show something is happening in PS (like if PS is getting a large list of files from Bridge that takes 20 seconds I will [kludge] open a special-purpose image in PS - since PS does not have the palette type).

I must have a closer look at the BridgeTalkIterator.

There are other possible useful variations on the above - namely combinations of restarting the scripts at each end as each iteration finishes and also using strings written to file as intermediate steps.

Good news about the Library files. Well done and thanks. I must admit I find them most useful as a source for 'ways to do things' rather than as the doers themselves - I tend to want to make sure the specific code I use is purpose made and does things by my own idiosyncratic rules (lol).

Andrew

[_Andrew-Hall_]

I've read your post at least another 5 times, there's a lot of really useful stuff in there. I think I will stick with my current structure for now but I can see myself going your way on the next iteration.

I quite like the idea of opening target files directly from Bridge (no sending file objects at all) but I need to convince myself I can force the open to the right app (rather than the default app) - using openwith - in a safe cross-platform way.

I am doing some testing on getting target file lists directly from bridge, versus doing the same from PS, with some interesting results so far, but I need to convince myself that I am doing it right before I say more.

If I send a script from say Photoshop using BridgeTalk to Bridge, is there a way to get that script to report errors and especially timing profile info in the ESTK. Similarly can I put breakpoints in the script that will be caught in the ESTK even though the file is being run from PS.

Andrew