How can I control WHAT Scene Edit Detection uses to detect scene changes

So I've got a recording of a game where I view every single attack animation that now needs to be split into individual video files for each move.
When I used scene edit detection on the main recording it made cuts in wrong places, or to be more accurate there were parts where it should clearly have made cuts but it didn't.

In the recording there's a part that remains the same for the whole duration of a move animation and only changes when I selected a different move.
So I figured what if I make a separate version of the recording where you see just that part so that it can detect the scenes based on what I want it to use. The idea being that I then figure out a way to "transfer" the cuts to the main recording.

But doing that resulted in Premiere not detecting any scene changes at all, so what exactly does Scene Edit Detection use to detect things and can I somehow change what it uses for that?

Here's a quick recording I made where it didn't add cuts even though it should have: