What are the different modes of selection that shapes get into and how exactly can you get them into these modes? ie. when you single click, double click, etc. In particular, how do you get a shape to get back into the mode where you can grab the bezier handles or drag the size of the shape by grabbing one vertex. For example, in some modes of you grab a vertex and drag it adjusts the scale of the shape, in another mode if you grab a vertex and drag it moves the entire shape's position.
I'm not clear what you're getting at. You simply click outside the object to deselect. the rest is determined by where the various shape entities reside hierarchically, so it may help to actually revela the structures in the timeline rather than relying on hit-based selections in the composition.
Reading my OP I imagine it is not very clear. For clarity's sake, one of the main issues I'm having is; after creating a shape with the pen tool I am not able to put the shape into the mode where you can grab the bezier handles. My workflow concerning this has been reduced to randomly clicking the shape in various combinations to at some point reveal the bezier handles. The shape, in particular that I'm working with is a straight vertical line.
If you have just two points in a straight line hold down the Ctrl/Cmnd key and drag a selection around one of the points to select it, then hold down the Alt/Option key to change the tool to the convert vertex tool (sideways open V) and click on the selected point and drag out a bezier handle. You can then edit each handle separately.
If you want bezier handles on a shape layer that has been created with one of the parametric tools (Rectangle, Polystar, etc.) you have to right-click on the shape (Rectangle 1) in the timeline and select Convert to Bezier Path.
When you want to edit a Bezier path the easiest thing to do is select the pen tool (g) and then use the Alt/Option and Crtl/Cmnd keys to convert the pen tool to selection and convert point tools. Select the path you want to edit in the timeline, select the Pen tool, hold down Ctrl/Cmnd and then observe the cursor as you move over the path. When you get a pointer with a white fill you can drag a selection around one or more points to select them then release the Ctrl/Cmnd key and move the points. Holding down the Alt/Option key will let you adjust the Bezier handles with the Convert Vertex tool.
I explain it all in this short video:
Grabbing a vertex and moving the whole shape happens when all of the vertices are selected. If only one vertex is selected, then it just moves that vertex. Clicking on the Path property of a shape layer in the timeline panel will select all of the vertices. You can drag a selection around one vertex (or any number of vertices) when it's in that mode to select a specific one (or specific set).
Double-clicking a vertex will launch the bounding box that lets you Transform the shape.
I had this same issue today, the way I was able to resolve it was to right click on the layer Path in the timeline, and select the Ungroup Shapes option, and voila!