What version of Photoshop and waht operating system are you using?
What are your Scratch Disk settings, and how much free space do you have on +your primary Scratch drive?
You might like to check the root directory of your primary Scratch drive for orphaned Photoshop temp files. These can be huge, and will be left behind if Photoshop does not close properly. Anything without today's date can be deleted.
Wow! Check this out. I went looking for a creen shot for you, and found a wee bit south of 100Gb of wasted drive space. I have not used this drive for Photoshop Scratch space for a while, and never thought to check it.
The bottom line here is that Photoshop does not always release RAM when you'd think it was no longer needed. A good Scratch space strategy will minimise RAM usage, but sometimes — and especially if your system spec is marginal — you are going to have to close and reopen Photoshop, or go Edit > Purge > All to get your RAM back. If it is a proper nusience then think about setting up an Action to Purge All and have it triggered by a Function key.
Note: In my opinion the Function keys are way too valuable to waste on their default shortcuts in Photoshop. If you look at this list and can identify a shortcut that you never use, then consider using it for often used Actions. There is something very satisfying about changing and resting your workspace, for instance, with a single key press.