Importing third-party brushes drastically slows PS on exit

Hello,

I have a series of 3rd-party brushes imported into Photoshop.  I recently acquired several others (Joel Grimes' Light Rays, Haze, Snow, etc.) and some of the .abr files are quite large (300-800MB in size).

I've noticed that when I finished importing these brushes into PS on my desktop and laptop PS lags quite a bit when I close it down - I click File / Exit or the top-right X, and I get a revolving wheel icon for several minutes while PS shuts down.  This is directly tied to the number of new brushes I have loaded in (if I delete some of the new brushes the shutdown time goes down accordingly).

So I took a look at the brushes.psp file and it's about 4GB!  Thing is, my laptop seems a bit quicker on the shutdown even though all the same brushes were installed (I even copied my laptop's brushes.psp over to my desktop and tried it out).

Long story short: is there any way I can get around this lag, or do I need to manage brushes on a per-session basis (installing/deleting groups as I need them) so that I don't spend 10 min. shutting down PS?  And why exactly is it that PS seemingly needs to access this entire file (or something) in order to shut down?

Thanks!