Creating a long seamless panning sequence with over 50 clips?

Judging from your screenshots you there is a basic flaw in your test. You can see different planes in the shots, the floor, the walls, all in different planes. If you just slide the images across the background you will not get any parallax changes and anything you put in front of the background will look very much out of place.

To answer your question, you don't need a really wide comp, you just need to animate each shot and make the seams as invisible as you can. As soon as s shot leaves the frame, trim the layer so you don't have any extra layers in the comp.

If you have 50 shots and it takes about 10 seconds for each one to cross the screen, which would be a pan speed that should not give you any juddering or strobing problems, then your comp is going to be 500 seconds long. That's an awfully long time for a single shot in any movie. You'll have to figure out how to hold the attention of the audience.

If you can live with the weird perspective changes you'll get as the background moves the easiest thing to do would be to match up the edges of your images, parent the second image to the first, then animate the position of the first image so it drags the second one across the screen, then repeat until you have all of your images moving at a constant speed. Then nest that moving background shot in your main comp and apply time remapping. You can then do all the speed changes you want without dropping frames or creating jerky moves.

I hope this explanation helps. This kind of VFX shot takes a lot of planning and preparation to pull off. You can't just grab a camera and take a bunch of stills and end up with something that will be believable.