Is it possible to connect expression to multiple keyframe layers ?

Controlling time can get tricky. You can go forward and backward in time by simply using the linear(t, tMin, tMax, value1, value2) expression where t = the value of the slider generated by the Keyframe Assistant Audio to Keyframes tool, tMin is the minimum value of the slider you want to use to start time moving forward and tMax is the maximum value you want to use for the maximum time of the clip. if you have a 10-second clip that you want to vary the speed then value1 is 0 (zero) and value 2 is 10. That will show you the first frame of the video when the slider value is at the minimum value you set and the last frame of the video when the slider value is at its maximum value. These minimum and maximum values are easy to find if you look at the Graph Editor and check the Value Graph.

If you want to look at some expression solutions that are more sophisticated than that check out http://motionscript.com and look at the great info that Dan Ebberts has posted on audio driving animation. The bottom section of Speed and frequency control will be helpful, and the first section of the Expressioner's Design Guide also has some good stuff.

If you want to use Twixtor instead of Time Remapping you should be able to figure out what values to edit. 

One more hint, when using Twixtor it is always more efficient to apply Twixtor to a solid and then point to the video you want to manipulate. You'll find it much easier to handle if you work that way.

You actually solved my problem. I was trying to do it in a very complicated (dumb) way :D. Output 1 + Output 2 did the trick. Thank you very much. Have a nice day 🙂