When changing photos on library module it scrolls through the entire folder and I can't stop.

It looks like you’re on a Mac. Does it change anything if you reduce the Key Repeat Rate, as shown in the picture below?

This is just a theory, I don’t know what the real problem is, but the reason for my suggestion is in case the problem might be cause by something like holding down an arrow key on a very fast computer loading up the keyboard buffer so quickly that, for example, it loads 500 arrow key presses in a second or two in a folder with only 400 images. In that case an application would scroll to image #400 (the last image) before all 500 key presses were executed. If that was the problem, maybe reducing Key Repeat Rate would help.

macOS-15-System-Settings-Keyboard-Key-Repeat-Rate.jpg