GeForce RTX 4080 goes up on 100% EVERY time im sliding in the Timeline and while Playback (Lagging)

Hi,

while editing my Videos I'm experiencing lots of lagg. Everytime when I rewind or go forward in the timeline, the whole video is lagging. Same happens when I play the Video in the Playback mode.

My CPU is not getting above 10%, but my GPU goes instantly on 99% as soon as I slide the mouse in the timeline.

These are my Computer Specs:

Prozessor AMD Ryzen 9 7950X3D 16-Core Processor 4.20 GHz
RAM 64,0 GB 
Asus Rog Strix  GeForce RTX 4080

I have tried many options like giving more RAM priority, turning down the Playback resolution etc. But the Problem is unfortunately not disappearing. 

It would be awesome if someone versed could give me an advice.

Kind regards