Speed/Duration Changing Sub-clip in Premiere

I'm trying to edit a short video essay on slow-motion usage in superhero films for a class project and I'm trying to gradually slow down the end of one subclip, hold it on the last frame, then speed up all the footage in reverse to "reset" and go back to the beginning of the subclip.

I understand how to gradually slow down and speed up subclips using the pen tool and speed/duration, but for some reason on the reverse subclip when I speed up it drags footage from outside the subclip bounds and fills in the beginning of the clip.

For example, if the subclip starts at 5 sec and 15 frames of the master clip, when I speed up the subclip the new beginning of the sublip would be at 4 sec 20 frames of the master clip.

Essentially, speeding up my subclips is working fine where I want it to, but "filling in" (for lack of a better term) the beginning of the subclip with footage from the master clip. You can see how the first frame of the subclip changes after I speed it up in the second image.