Multicamera sync lost when switching cameras

Hi, I'm pretty new to video editing so it's totally possible that I'm doing something wrong, but please help me out. I have 3 cameras + audio source muticam sequence and it's synced ok. Whenever I try to switch camera, the new angle is getting out of sync. Even more, if it's the same camera, video gets like 0.5s back every time there is a cut.