I wish PPRO had even better PERFs optimization: It is slow when inserting clips on filled sequences

[Bed30321077z973]

The observation is very simple and easy to reproduce:

Take a sequence that has 0 clips in it, and take the same sequence that has about 200 items in it:

If you add a clip though a script, then the operation will be much slower if you do it on a sequence that has already 200 clips than if you do it to an empty sequence. I am observing it because I try to insert multiple clips at once, the usual user probably has never been bothered to see that an insertion took 2-3 seconds instead of 0.1 sec.

You must know that creating a new sequence for the experiment (and leaving the 200 clips sequence aside) does not help. It is more about the number of "inserted clips" on all "sequences of the project" that affects this speed I think.

Perhaps it can be easier to observe if you do multiple insertions at once:

If run a simple script that add 200 differents other clips (actually could be probably the same media inserted in 100-200 differentes places) to the sequence:

- If you use the empty sequence to run the script, it will run faster (I will provide example of Speed % proportion below)

- If you use the sequence that already has 200 media in it, it will be slower! The more there is items there the slower it gets!

I had already posted about this here (https://community.adobe.com/t5/premiere-pro-discussions/is-there-a-way-to-turbo-speed-up-premiere-pr...)

Example:

If my sequence is empty, then i can add 200 clips in 60 seconds.

If I empty it again I can add another 200 for another 60 sec => Total is 400 clips in 120 sec (approx).

Now if I choose to add 400 clips on an empty sequence this is what happens:

- the more we advance in inserting clips, the more clips there are already in the sequence the slower it gets. The total would be 200 secs or porbably a bit more (instead of 120 seconds like we did earlier)

That's like 30-40% slower (it will get worse below).

Now if we try to insert in empty sequence 350 clips: 

- same concept happens, the more clips present in sequence, the slower it is to insert another one. So by clip 300, isnerting clip 301 is way way way slower than inserting clip 1 and 2..

About 600+ seconds

If we inserted 200 by 200, that would be 105 sec (60 for 200 + 45 for 150, hence 105 for 350 clips) , way faster than the 600 sec, so we are at what 80%~ decrease in speed?

it is kind of bothering/bugging.

And throughest the whole operations, my GPU and RAM usage is very "cool" and calm:

What I would have wanted:

I am ready to use MORE GPU and/or RAM (even if that means activating a temporary turbo mode where PPRO start doing lot of calculations in the background) to ensure PPRO is prepared to treat any request instantly or faster (at least make inserting an item on an filled sequences as fast inserting it on an empty sequence or close to it or at least way faster as much as possible).

End result: Inserting a clip to a project that has a filled sequences (in items) as fast as inserting the same clip to a project that has empty sequence(s).

Use whatever needed: GPU, RAM, CPU, all in combination, all fine by me. I just want speed.

Thanks.

[Ivan Stepanov]

I wonder do you have "Text" panel open when you run your script?

[Bed30321077z973]

Yes.

You think if I do a run without that panel, it could improve the speed?

[Ivan Stepanov]

Text panel apparently also uses ExtendScript engine and it triggers automatically, whenever clip is selected (when you insert a clip, it becomes selected).
For example, when I run my extension Grave Robber on a big timeline, to un-nest multiple sequences, if Text panel is open, speed of operations will be dramatically slower.
In any case, this is something to test as well.

I've read in your other post that you run script file to do insert operations?

[Bed30321077z973]

yeah. I Tried to experiment by removing the text panel, the captions tracks, restarting PPRO etc.

I noticed a decrease in speed everytime I did the same insertion again, so when I was removing captions then removing the text panel, up to removing all panels that are next to the text panel location, I noticed it got slower by 8 seconds 😮

Closed all panels that are next to text, only the panels for showing the medias/imported files /info etc are there:

it got slower buy an additional 10 seconds!

I tried seeing if me running a browser (with youtube on it) or pausing it or closing the browser entirely changed anything, the answer is no.

RESTARTING PPRO, removed the extra 8+10 seconds I lost from earlier.

Restarted PPRO AND visual code: lost 3 seconds for some reason (maybe some libraries being downloaded or charged at extend level)

then I closed ALLLLLL panels (even the ones showing medias (imported files)), nothing on screen except the tracks of the sequence and the tool palette:

Gained all the seconds lost + extra 10 sec speed!

(for a difference of 25-28 sec for a particular instance)

I will experiment more later with having every single panel closed to see if it really helps in longer things and if its consistent.

Don't know what i clicked but by mistake a did a "CTRL+someKey" (around K) and a big perpendicular | Line showed on my code, and when I run the code later I got an error related to JSON (library?)

Wonder if I did a shortcut that destroyed my intial config?

Usually I dont need to touch this part and it works well!

Why would it suddently stop recognizing libraries?

Maybe it is unrelated. Maybe it is related to the fact the Panel "project" where you can see your items and files need to be open to be able to interact with some files, weird.

[Ivan Stepanov]

As Bruce suggested in another thread, it would be great if you can share your script to see if there are any culprits there.

Also if you have any extensions installed, you might try to uninstall them, as the can also keep Extendscript busy.

[Bed30321077z973]

I have no extension as far as I know, in any case I think this statement is always true:

"Inserting an item to a sequence of a project that has one or multiple "filled" sequences, is always slower than inserting it to a project that has empty sequence(s), it gets SLOWER AND SLOWER, the more filled the sequence(s) of a project are!"

[Ivan Stepanov]

I can repeat this behaviour manually, without scripting (I would separate scritpting and Premiere Pro performance, so it is easier to track the issue).

I just created 2 timelines: one is empty another one has 1000 "bars and tone" clips.
When I run macro to trigger "insert" command 5 times, to insert the same "bars and tone" clip 5 times, I can see that in empty timeline it is fast, in full it is slower.
On macOS the difference is not that bad, but on Windows it is more dramatic (however I run Windows on Macbook arm chip, so it can have performance issues).

I think it is related more to data structure of a sequence, not to GPU. It is harder to add items to big data file, then to empty one.

[Bed30321077z973]

When I run macro to trigger "insert" command 5 times, to insert the same "bars and tone" clip 5 times, I can see that in empty timeline it is fast, in full it is slower.

By @Ivan Stepanov

THANK you!

See that @Bruce Bullis ?

Any script can reproduce this issue.

Any idea how to fix this?

I want it as fast as possible with no lose in "perfms"

I am on windows indeed, don't know if's a machine issue like Ivan might alluded to pc vs mac thing, but in any case I shared my machine consumption screenshot during the inserting earlier, and none of my RAM/ CPU/ GPU were used that much. (I want any solution that can fix this problem, and I am ready to pay the price of perfs, of any time (ram/cpu/gpu) even if that means activating a temporary mode that only some of us can use (that might benefit others unexpectedly in the future ofc, for new use cases)).

[bbb_999 · Accepted solution]

> Any idea how to fix this?

No. 

> I want it as fast as possible with no lose in "perfms"

Not every process is CPU-bound, or GPU-bound. Similarly, not every process can be made faster, by moving it to GPU. 

[Bed30321077z973]

Hello bbb,

What make a mode where the PPRO is always on "hyper calculation" where it is always pre computed thins on the background:

So in the normal mode:

My assumption is that: if my timeline has 2000 clips, and I perform an "insert" with my code, it usually start computing in the background something making it slower (don't know what) then it does the insert and that process takes time, way more than if the sequence was empty (if it's empty the computations are quick almost non existant).

Instead in the hyper mode: the calculations/computation that the function .insert() provokes would have been MADE PRIOR (and stocked in the background) prior to a random incoming api .insert() request.

This makes the .insert() instantenous because PPRO had already the answer to whatever the .insert() function needs to perform its action. if that makes sens? hence making the action (request) runs as if the sequence was empty (same speed)

(The hyper mode obviousely would make the machine heat up more and consume more ressources and would be entierly the choice of the user, which he can activate only when neccesary, it would personally give me an increase of 80% speed, even more, and I would be finished was sooner and would have deactivated the mode quickly after using it a bit)

This is all me asuming that the insert function needs to compute things BEFORE adding the media to the track/sequence.

Perhaps the insert function  will actually compute things AFTER the clip was added, hence everytime we add a clip, PPRO will do something to "recheck" every item "status" on that 2000 items sequence? Idk, but in that case the hypermode would not be able to do the same pre computation in the background like I am assuming, at least not the same way.

Obviousely only the conceptor or maintainer of PPRO could know what is the better answer.

I know this seems like undeeded to most users, but I am pretty confident this will help in the future for unexpected use cases.

Did you have time to simulate it yourself like Ivan did?

(I just created 2 timelines: one is empty another one has 1000 "bars and tone" clips.
When I run macro to trigger "insert" command 5 times, to insert the same "bars and tone" clip 5 times, I can see that in empty timeline it is fast, in full it is slower.)

(quote from Ivan)

Good day/evening

[Bruce Bullis]

> What make a mode where the PPRO is always on "hyper calculation" where it is always pre computed
> thins on the background

Such a mode would do little except burn electricity, and make wrong guesses. 🙂

Insert edits take longer with 2000 track items, because PPro needs a copy of the timeline from before the insertion, for the Undo stack. 

[Bed30321077z973]

As I said I only activate it before in a session of quick massive inserts. 

The mode would essentially constantly copy the timeline in the background prior to me calling insert().

Although Maybe that would only improve speed of the first insert, as I am request a second insert the mode would not had enough time to do another copy? I wonder

Or maybe modify the way it copies if it knows it is going to have a list of incoming inserts, but since extend script treats once command at a time that might not be possible? (unless we create a new function (that reads lists)) I guess.

In any case, I hope the idea or the seed has been planted for a possible modification to allow this if it's ever possible? (maybe an eureka in shower thoughts haha, I don't know how ideas emerges XD)

Thanks for the answers in any case.

[Bed30321077z973]

In may case it is inserts that are at the end of a timeline

So I wonder if PPRO can only copy a croped version of the timeline , do its work then recoup the cut content with added content in an "addition" way.

[bbb_999]

Attempting such optimizations would leave PPro open to inadvertent data loss, which we will not do.

[Bed30321077z973]

Ah okay, if you ever find a solution that feel safe for PPRO, please inform me.

Thanks again for the communication, this is a very poweful point PPRO offers (suggestions/questions forum).

[Bed30321077z973]

And if you missed my second message this is full image of my suggestion (assuming the disabled area when used to recoup the new timeline would not consume as much as the copy it does when an insert happens otherwise it would be as slow and would not work):

(sorry for the passion I showed for this xD).