Creating small proxy jobs becomes unusable the larger the source folder on disk becomes

Yeah, I just don't understand it. The extra files in the source directory have never been imported to Premiere, but for some reason Premiere seems to be "distracted" by them.  I'll try experimenting with smaller vs larger movie vs non-movie files.  I feel like it's scanning them. (and my source directory isn't a watch folder or anything, or does AME "watch" a directory a source file comes out of even if it's not on the list of "watch folders"?) If this were the case I could see it causing things to fall over.

(sorry, my wife and I share a computer and was logged into her account)

Also, it starts slowing down when there are >500 files in the directory.  It just takes noticeably longer the more files you add.

Ha, ha, ha... excellent analogy.. love it. 🙂

I don't think it's right regards PP, but the analogy stands!

It's over a NAS currently, but I'll try it on a local disk also (one directory with 10 files, another directory with 2000 files) to take that out of the equation.

I don't have this problem with any other Adobe software (eg Lightroom or Bridge) on the NAS with directories much, much larger.

I'm using the same Proxy target location each time in both tests (which is a directory on a completely different drive).

It seems like...something...in the Premiere->AME handoff is scanning/analayzing/processing the entire source disk directory of files, even if those other files have never been imported into Premiere and should be ignored entirely.

The only real world analogy I can make is someone tells you to go to a party and pick up Bob and give him a ride to his hotel.

You show up at the party, immediately find Bob and tell him it's time to leave.

However! Instead of leaving, you decide to talk at length to all the other people at the party whom you've never met nor should be talking to, you are talking to them simply just because they are there. The more crowded the party is, the longer you are going to be there before leaving.

@Humanclock Also what is the exact amount of files that are in the folder that you start to notice the proxy media to no longer run as quick as you would expect it to? Are you on HDD,SSD, or NAS? Premiere it self should always function the same and does not change performance on making the proxies. However the OS may have certain limitations depending on how many files are within the specific folder you are looking at. So if your example of 2000 files is the exact number that is causing the process to slow down or hang then it could be that the filesystem/OS is slowing down at managing the files there, and that could slow down Premiere. But it's not that Premiere cares.

Thank you

Ian

Hi @Humanclock 

When you choose to create proxies are you actively choosing the Proxies location? In location option are you selecting Next to Original, In Proxy folder? Or are you choosing a custom location for the newly created proxies?

Thank you

Ian

No, please read my steps carefully. I am only ever trying to create 10 proxies in any scenario. I should have used "disk directories" so it can't be confused with "folders" that Premiere has.

"b. Creating proxies for the same 10 specific files that reside in a directory ON DISK that contains 2000 files (1990 of which ARE NOT IMPORTED into Premiere) will cause the proxy creation process to completely hang and fail."

Here is a visual description of the problem that might help things:

If my hard drive directory layout is this, Premiere will work great for making these 5 proxies:

d:\myFiles (contains 10 files total)

movie0001.mp4 (Imported to Premiere and making a proxy of)
movie0002.mp4 (Imported to Premiere and making a proxy of)
movie0003.mp4 (Imported to Premiere and making a proxy of)

movie0004.mp4 (Imported to Premiere and making a proxy of)

movie0005.mp4 (Imported to Premiere and making a proxy of)

movie0006.mp4 (not imported into Premiere)

movie0007.mp4 (not imported into Premiere)

movie0008.mp4 (not imported into Premiere)

movie0009.mp4 (not imported into Premiere)
movie0010.mp4 (not imported into Premiere)

Now if my hard drive layout is this, Premiere will fall over and freeze trying to make the same 5 proxies:
d:\myFiles (contains 2000 files total)

movie0001.mp4 (Imported to Premiere and making a proxy of)
movie0002.mp4 (Imported to Premiere and making a proxy of)
movie0003.mp4 (Imported to Premiere and making a proxy of)

movie0004.mp4 (Imported to Premiere and making a proxy of)

movie0005.mp4 (Imported to Premiere and making a proxy of)

movie0006.mp4 (not imported into Premiere)

movie0007.mp4 (not imported into Premiere)

movie0008.mp4 (not imported into Premiere)

movie0009.mp4 (not imported into Premiere)
movie0010.mp4 (not imported into Premiere)
.....(not shown: hundreds of files movie0011.mp4 through movie1999.mp4 not imported into Premiere)

movie2000.mp4 (not imported into Premiere)

Hi @Humanclock - I would expect that when you send a 1000 files to create proxies in Media Encoder that it would take a very long time.  If you are saying that media encoder has stopped generating proxies then that is a different issue.  

When Premiere Pro is in this state of looping progress bar can you open task manager under processes select Premiere Pro, then right click and select "Create Memory dump". If you can upload it and send me a link then I can have a dev look at it to find out what is going on.

Thanks for your help.

1. Windows 10, AMD Ryzen 5950x, 128gb RAM. NVIDIA 3080RTX 12gb card. PP v 25.0

For proxy creation, it's just the standard:

a) Highlight the 10 files in the project panel
b) Right click and select "Proxy -> Create"
c) Select half-size, Prores Quicktime

Step 4 isn't 100% necessary, it's just to avoid waiting for the proxy creation to finish to proceed to step 5, I should have specifed this.

Steps 1-4 are just to demonstrate that proxy generation of 10 files (with only 10 files in the source folder on disk) will in fact, work.

However, if the source folder on disk contains hundreds of movie files (that aren't even imported into PP), the bug/issue shows up before any proxy files are actually created by AME.  When PP goes to send the proxy job of 10 files to AME, the progress bar basically never shows up and it sits on the looping progress bar on the "Creating proxy job" (not on my PC to get the exact message).

Another way of testing is just to time how long it takes to send the proxy job of 10 files from PP to AME, when there are only 10 files in the source folder on disk (eg: D:\myTestFiles).  Start adding 100 movie files at a time to D:\myTestFiles (again, do not import these new files to PP).  Now time it again with the same 10 files.  Keep doing this and eventually everything will just hang at the handover from PP to AME on D:\myTestFiles has more than 1000 files in it.