Layer Parenting Causing Unnecessary Movement

I'm having a problem where, when I parent my layers, my child symbols move in a completely wrong direction. I want the whole speaker and the words to move as one unit while the words slide out. 

This is my first frame:

And this is my last (the words slide out from behind the speaker): 

I have the words set to Tween < Ease In-Out Quart. When I don't parent the layers, the words slide out along their angle just fine. But, when I parent the layers, this is what one of the tweened slides look like:

I have no idea why it does that but the words are no where near the speaker. This is what the words look like without parenting:

 I'm trying to have both the speaker and words move in tandem without interrupting the movement of the words sliding out.