What "Drift Over Time" expression? If it is the included animation preset you can easily change the angle by adjusting the direction of each layer. To adjust the other parameters you'll have to modify each expression.
It would really help us help you if we saw a screenshot and had a good description of exactly how you wanted the layers to move. It might be easier to just add a wiggle expression like this to position. Each layer would have a different random seed so the movement would be different for each layer.
That would randomly move a layer up to 800 pixels over 4 seconds and create some nice curved motion paths.
Thank you Rick - here is a screenshot. I trying to get each object to start in a random position and then move in random, linear motion paths while staying within the bounds of the frame the whole time.
You can get the different angles by adjusting the angle in the preset, but the get the edge detection you are going to either going to add some keyframes manually or add about 20 lines of code to the expression that detects the size of the layer including any scale factor, the size of the frame, and calculates a reflection angle for the collision. I've been doing this for a very long time and I would guess that it would take me 3 or 4 hours to come up with an expression or 3 or 4 minutes to keyframe the animation preset. Your choice.