Why rotobrushed img sequence shows object in different position than in un-rotobrushed sequence?

Hi, I am noticing something a bit odd that I can't figure out.

I have 25 fps mp4. It is 804 frames. In AE I rotobrushed it and rendered an 804 frame EXR sequence. I also rendered the original mp4 as EXR sequence. 

I am just experimenting, triyng to confirm that I can solve a camera, use that solve, then add a rotobrushed image sequence to then apply compositing later on (adding object in footage to 3d environment).

In blender, if I have the original sequence play as camera background, then you can see that the cat is lined up with the red line. All is well.

But if I then use the rotobrushed img sequence as the camera background, notice how the cat is sliding all over the place, especially at the beginning.

In the attachments I have the playback in AE of the rotobrushed scene.  As well as playback in blender showing the disparity.

I can't figure out how to account for what I am observing and what might be causing it. Ideas?

thanks