How do I stop the quick selection tool from jumping all over the place?

I love the quick selection tool - it's my favorite! It also takes a bit to get used to it. Everything that Barb said and:

It's a brush - and the size of your brush also sets the tolerance. Bigger brush = looser selection. Use your bracket keys (next to the P) to make your brush bigger and smaller as needed. Don't let the edge of brush touch the stuff you are trying to not select. It expands on it own - as you are experiencing. Selecting the background is tricky - select the person, then you can always invert your selection after. Practice and patients...

The Quick Selection tool makes selections based on tone and color. If the tones/colors are too close, it's going to leap beyond where you want it to go. Start by lowering the Tolerance value in the Options bar (up top): a lower value is less tolerant of a tone or color change, and keep the cursor completely inside the area you want to select—don't let the edge of the circle move outside, at all.

The Quick Selection Tool - Photoshop Selections

I want to add that while it's great for some situations, it doesn't work for everything. In addition to the basic selection tools (marquees, lasso, quick select and magic wand), you might want to explore Quick Mask (which uses painting tools), Select and Mask and some of the more advanced techniques like using channels and the pen tool.