While looking at the audio waveform to determine edit/events points, the timeline indicator starts accurately yet immediately loses all relevance to the audio I'm hearing. For example, if the line I'm hearing is, "Create a post in After Effects", I'll hear, "...in After Effects" while the timeline indicator is at, "a post..."
I first noticed this a couple upgrades, back and it seems to have gotten worse:\ My current solution is to hold down the Command key while holding the timeline indicator over the 'guessed' area of interest on the waveform - not ideal. Editing to accurate audio is essential! Please fix!
And yes, I've purged, restarted, checked and unchecked all the relevant boxes in the Preview & Preferences dialogues. – .wav audio, AE 22.5.0 (Build 53) / Mac OS 11.5.2 / 32GB RAM
It might make more sense to prep such stuff in Premiere/ Audition and import the edits and markers from there. Not saying that it doesn't suck, but AE never has been good at exact audio timing as far as I can rember for the last 25 or so years due to it not working sample-exact. Current issues in AE and the underlying OSs clearly don't help as you already found out yourself. Sorry I can't offer a more optimistic and positive answer.
Since they removed multi threading AE really took a nose dive in playing back audio in real time. To compensate they made it so it preloads a second or two of audio which gives the illusion that they fixed the problem. Problem is, as you've discovered, you can no longer use the waveform as a reference point. I've gotten used to it. You just have to learn to live with it. As Mylenium said, it sucks but its just the way it is. With practice you'll just get better at recognizing the waveform patterns within that short offset.
You may also notice audio drops frames all the time which, when dealing with narration, could make plurals sound like singulars or drop out articles like "as" or "the". It makes difficult to tell if the narrator made an error or the audio is dropping. In those cases I I have to shorten the playback duration to 5 or 6 second and let it loop through once. The second time through it will play the audio correctly. But if you stop and restart without letting it loop through it will always drop the same frames.