Struggling to implement a curved pan zoom along a specific path

Sometimes it is easier to move the layer than it is to move the camera. Just as in real life, moving your actors around the set is often easier than moving the camera. Sometimes it is easier to move the camera. The hardest shot to pull off is when you move both the actors (layers) and the camera. For this problem, as you described it, I would set up the camera at its default position, open up 2 comp views, top and Active Camera, then move the layer to the starting position by adjusting the X and Z position of the layer and set the first keyframe. Then I would move down the timeline to the end of the move and set the end position by adjusting the X, Y and Z position.

Then I would open up 4 views, make sure show all keyframes is turned on in your preferences, choose the Pen tool, hold down the Alt/Option key to bring up the Convert Point tool and click on the starting and ending keyframes and drag out the bezier handles on the top view and front view until you get the path you like. Putting the CTI in the center of the move makes this a lot easier to judge. Then scrub through the comp a couple of times and check your timing. When you get the motion path to look right it's is time to change your two keyframes to easy ease and if necessary, fine tune the velocity with the graph editor.

Something like this (using a solid for your image layer:

With either technique, the technique will involve having all keyframes visible and multiple views. The pen tool makes manipulating the curves in the motion path much easier and prevents you from changing the position of the hero keyframes. (hero = usually ending position of a layer, but in this case, both starting and ending position are hero poses).

I use the Shift key and a click and drag on position properties to set up my hero position then adjusted the path using the pen tool. Most of the time, if I want to keep the layer's perpendicular to the camera, I move my actors (layers). If I need to move the camera between several actors I move the camera. I seldom move both in the same shot unless the move can't be done any other way.

If you have a bunch of these moves to make and the final video is going to be more than a few seconds I would create a separate comp for each short sequence then edit the final project in Premiere Pro. More than 90% of my AE comps are a single shot and under 7 seconds. Most of my sequences (multiple shots like a slide show) are only as long as a single sentence or musical phrase (4 or 8 bars), and the final editing is done in Premiere Pro. This makes changes easier to make and the final polish much easier and the project takes much less time to complete.

I hope these suggestions help. These techniques are all part of the toolset you need to become familiar with if you intend to work in AE's 3D space.