Selecting an External drive in Photoshop save window takes forever to load

For some reason, only in Photoshop, whenever I save a file, I'll hit Cmd+S, it will bring up the save dialog, and as soon as I select an external drive to save to, it will take anywhere from 30 seconds to 3 minutes to load the drive so I can save the file. However, if I am saving to a folder that already exists, I can search for it on said drive and it will find it right away and I am able to save it. But if I click on the main folder in my sidebar, it will take it forever to load it. It doesn't do it with the internal drive. I have tried to reindex the drive, to no avail. And to throw one more strange thing in there, this problem comes and goes. Everything will work fine for like a week, and then it will just take forever to save and be like that for weeks, then randomly it will start working again.  The drive in question is an external SSD NVME, formatted to MacOS Extended (Journaled) and Photoshop has full access to the drive. I am on an M1 Max, 64GB, running Mac OS 13.6.6 and Photoshop 25.6.0..