removing a person from a scene

As davidarbor said, shoot twice. The shot you need is a clean plate with no Grandparent or person sitting next to them in the shot. It would even be easier if the chairs were removed. What you need is a clean plate that includes everything behind the person being removed. If you want overlapping action like someone passing the potatoes in front of the grandparent you want to make disappear, you'll need to shoot the action without the grandparent then shoot just the grandparent, then roto everything that passes in front of the grandparent. If the scene is complex putting a green screen in the empty chair will simplify the grandparent.

The key to making this work and keeping post-production simple is to carefully analyze everything that you want in the scene and then make the roto work as simple as possible (green screen) and make sure you get a clean plate. 

You can even do this with a moving camera if you are careful. A slider or a jib arm and shooting at a higher than normal frame rate help with the timing. I've done this kind of thing a lot and the planning and shooting is about 20% of the work involved, but if you don't do that 20% well your post-production time can easily go from a few minutes to several days.

Just for fun, to see what is possible, take a look at this Behind the Scenes video and pay attention to the grips moving the green screen in front of the hero. Great planning, clean plates, and a well-directed actor, and careful detail let the hero of the film walk from the streets of San Francisco to Paris to Wu Zhen in what appears to be one take. They could not have done it without a simple way to separate the hero from the background and clean plates.

Yes, shoot the scene twice with the camera locked off and make sure there is plenty of space around each grandparent, so you can mask easier. If you have someone reaching their arm or moving an object that overlaps with one of the people you want to remove then you're going to cause problems for yourself. If you want to add a camera move or a zoom you should just do it inside of After Effects. Save yourself the grief and keep the camera locked off. 

From moderator: removed duplicate post