MacBook Pro M1 Pro - PS becomes laggy after adding selective color adjustment layers

Since I bought my new Macbook Pro M1 Pro I have experienced a very laggy Photoshop after having added a number of layers (let's say 15+ adjustment layers). This shouldn't be a challenges for this machine right?? Well, now while I was working on an edit I noticed that adding a Selective Color adjustment layer screws up the overal performance. Sliders take 1-2 seconds to responds and it take even more time to see the actual result... Perhaps there are more types of adjusment layers affecting performance like this? I work on files store on my local harddrive, run the latest version of PS and Monterey, so what's going wrong here?