Well, this clearly can't be done the lazy man's way and boils down to good old traditional masking and rotoing, i.e. filling in the areas with a clean plate. Really not much more to say than that. Realyl just boring, tedious work, but perfectly doable, assuming you actualyl have a clean plate/ are able to reconstruct the background either in AE or Photoshop.
The most efficient workflow I can think of for that shot is to start by tracking in Mocha AE. The entire workflow is too complicated to go over in a post but the idea is to motion stabilize the part of the footage that you want to replace, then create your clean plate. Assuming you don't have footage without the people in the scene, you can do that by cloning. Then you do the simplest roto you can to do replace the people in the shot with the clean plate. You wouldn't want to completely rotoscope the actors, you create mattes that expose the fixed parts of the shot. When that is done, you put the motion back in the shot. I think it would talk me about two to three hours to complete the shot.
Here's a tutorial that explains basically how to do a stabilized corner pin workflow. This is not something you can easily do without a clear understanding of Mocha AE, track mattes, animating paths, and layering.
The procedure of getting tracking data to a layer has changed, but the workflow is solid. I use it all the time.