Tips for investigating memory leak / bottlneck? (scripting / SDK)

[videowizard.io]

Hey, wondering if anyone expert in the SDK can advise on an issue I'm having. 

First and foremost, I am using Python to automate workflows in Premiere using a package called Pymiere. 

I have a function that loops through an array of ~180 image paths, imports each, and places them on the timeline one at a time. The operation works, but is SLOW and gets slower and slower as the loop progresses, until it is basically at a standstill and I terminate the process. For what it's worth, the way Pymiere works is it basically compiles python into ES and sends it to a local server that controls Premiere.

The obvious inneficiency is that each call of the "insertClip" function is actually compiling a unique ES script and sending it to premiere. It is not a single batch call. But 180 individual calls. Memory management, "garbage collection" etc is beyond my current understanding, but it feels like it could be something to do with that.

I am hoping that this issue lends itself more generally to SDK troubleshooting and not specifically unique to Pymiere, but it may well be a Pymiere issue. In any event, I figure this community may have sone insights!

Does anyone have any thoughts on where a memory leak or bottleneck might occur in a workflow like this? Or how to even troubleshoot it? 

Thanks.

[Bruce Bullis]

Other than insertClip(), what other PPro ExtendScript API calls are getting made, by each insertion? 

[videowizard.io]

It looks like there is some reading / writing of ES code, and the following:

-line 77, checks that Premiere is running, which basically means:

      -checks for PID with the system

      -checks the Premiere "panel" to make sure it is reachable (post request to Premiere)

-line 97, sends code to premiere (post request to Premiere)

[Bruce Bullis]

Those seem like good specifics, but I don't see any ExtendScript calls, in/near those links; am I missing something?

What other ExtendScript calls are occurring, while importing and placing each of those ~180 images?

[videowizard.io]

Oh I see. From my python script, for each loop of the function, there are several:

self.project.importFiles(files_to_import,
                         suppressUI=True,
                         targetBin=import_bin,
                         importAsNumberedStills=False)

In this case "files_to_import" is an array with one file in it. I do not think this one is the issue. 

The big one is:

def add_image_to_timeline(self, 
                          ProjectItem,
                          timeline_in,
                          duration_seconds,
                          set_scale_to_frame_size=True,
                          vid_track_index=0,
                          aud_track_index=0):

    duration_ticks = self.round_ticks_to_nearest_frame((self.sequence_timebase * self.sequence_frame_rate * duration_seconds)) 

    timeline_in_ticks = self.round_ticks_to_nearest_frame(time_from_seconds(timeline_in).ticks)

    ProjectItem.setInPoint(0, mediaType=4)
    ProjectItem.setOutPoint(duration_ticks, mediaType=4)
    if set_scale_to_frame_size:
        ProjectItem.setScaleToFrameSize()

    self.active_sequence.videoTracks[vid_track_index].insertClip(ProjectItem, timeline_in_ticks)

 Each "round ticks to nearest frame" has 2 calls of the Time() function. And each call of time_from_seconds has one call of the Time() function as well. 

[Bruce Bullis]

Thanks! Those make sense.

Speculative: I wonder whether you'd get better performance if you [warning: hand-waving description of desired behavior] queued up the 180 "import these stills and place them on the timeline" jobs, at the CEP/JavaScript level (which is, I guess, also your "Python level"), and looped over them at that level...? That way, you'd be making a bunch of (relatively) smaller, atomic ExtendScript calls, and not forcing PPro to "eat the whale" of all 180 jobs, within one ExtendScript control session (which CEP afficianados might call 'one evalScript call')...?

Potential "enlightened laziness" possibility, if you're importing only stills, and if they should all have the same duration: Instead of setting the in and out points for each projectItem representing a still, could you not set the PPro Prefs --> Timeline --> Still image default duration to be correct? If so, that'd save (in your example) 360 undo-able ExtendScript calls, right there...

The pref you're looking for is 'BE.Prefs.StillImages.DurationInSeconds'; here's how PProPanel changes pref values. 

[videowizard.io]

Thank you! All great suggestions.

I will see about batching requests via pymiere. I am using pymiere without really writing any jsx, but to your point, it would almost certatinly work more efficiently if I call the eval_script function directly and feed it my same function just as a jsx script. That way it's just one "whale" each time. I'll give it a shot!