How to Set CPU Preview as Default in InDesign

Go to InDesign's Preferences > GPU Performance and uncheck it there. That should stick. Once you do that, the toggle under View greys out and cannot be changed unless you reactivate GPU performance.

Is the setting not persistent? (I assume you're using a Mac, since Windows doesn't support GPU acceleration — and many Mac users report glitchy operation when it's active.)

I don't Mac, but try the same thing as for all other InDesign working settings: with ID loaded but no document open, change the setting. If it's at all document-specific, that should permanently set the default to off.