Cropped video moving from side to side- how to lock in place?

Hello, any help would be greatly appreciated;

I am making a music video with a single dancer whom I am multiplying such that it looks like several of him are dancing exactly in sync at once.
I cropped the sides of the original video in order to single in on the dancer and multiply, however at the points where the dancer starts to move outside the cropped frame, I adjusted the cropping such that it follows the dancer. This creates the effect that the video camera is following the dancer.

For example, if at the beginning the crop is:
Left: 40%
Right: 40%
Then a couple of seconds later, I adjusted the percentages such that it follows the dancer while having the same dimensions (the percentages always add up to 80) like so:
Left: 37%
Right: 43%

The problem:
Despite the "position" numbers staying exactly the same throughout, the cropped video is not staying in one place; it is moving from side to side based at the points where I changed the cropping of the original video.
I multiplied the video five times such that five dancers are dancing on the screen at once, however when these moves occur, it reveals the black background on the sides of the screen whenever the videos move a little bit.

So, how do I lock the videos in place such that they don't move from side to side like this?