soft-proofing troubles

[Colinml]

I'm fairly new to Lightroom and printing, so I honestly can't tell if I'm just missing something obvious.  I think I probably am.  Here's the issue:  I've been printing on a Canon Pro-300 with generally acceptable result, meaning the prints matched the screen fairly well in terms of color, until I tried to print a field of fall color bracken ferns.  On screen, the ferns look very red, but the print shows them as more orange.  My monitor is calibrated.  

Here's the part I don't get:  If I use soft-proofing (using an ICC profile from Canon for the paper I'm using, which in this case is Pro Premium semi-gloss), Lightroom shows me (on screen) the effect it will have, including the out of gamut areas, and I can make adjustments to get the side by side comparison between the original and the soft-proof version to be pretty close (on screen).  Then I print, and it looks nothing like the soft-proof version.  I've tried various things (perceptual vs relative, etc).  No matter what I change, the print doesn't look like the soft-proof.

And here's a more confusing twist.  There is a setting for "profiles."  NOT the profiles you install for paper/printer combinations (ICC Canon, etc), but the "profile" listed in the Develop module (Adobe color, Adobe Landscape, Adobe Portrait, Adobe Standard, etc).  For some reason, the default appears to be Adobe color, but if I change this to Adobe Standard, the screen matches my print exactly.  I mean, perfectly.  To be clear, I mean that I turn off soft proofing entirely and, in the Develop module, I switch the "profile" to Adobe Standard, and it looks exactly like my print.

So, why am I complaining?  

Well, it's great that, on this particular print, I can make the screen match the print by simply switching to Adobe Standard profile in the Develop module, but clearly I'm not using the soft-proofing feature correctly, or there is something wrong.  I have made 5 different prints, with all kinds of different adjustments made to to the soft proof version created, BUT all 5 prints look identical to EACH OTHER (the paper prints all look the same), BUT the prints don't look like any of the soft-proof versions.  So, in other words, whatever changes I make to the soft proof versions have no impact on the output of the printer.  I've triple checked printer settings.  Lightroom is managing color (not printer).  

Help?

[TheDigitalDog]

Start here then ask anything missed:

http://digitaldog.net/files/LR4_softproof.mov
http://digitaldog.net/files/LR4_softproof2.mov

Newer, and covers Photoshop but the idea is the same:

http://digitaldog.net/files/SoftProofingInPhotoshopCC.mp4

Soft proofing is using ICC printer profiles. But there are other 'profiles' for cameras which are not for the task of soft proofing, they are input profiles for the camera and raw. Adobe Color, Neutral etc are DCP profiles:

Everything you thought you wanted to know about DNG/DCP camera profiles:
All about In this 30 minute video, we will look into the creation and use of DNG (.dcp) camera profiles in three raw converters. The video covers:
What are DNG camera profiles, how do they differ from ICC camera profiles.
Misconceptions about DNG camera profiles.
Just when, and why do you need to build custom DNG camera profiles?
How to build custom DNG camera profiles using the X-rite Passport software.
The role of various illuminants on camera sensors and DNG camera profiles.
Dual Illuminant DNG camera profiles.
Examples of usage of DNG camera profiles in Lightroom, ACR, and Iridient Developer.
Low Rez (YouTube):
http://youtu.be/_fikTm8XIt4
High Rez (download):
http://www.digitaldog.net/files/DNG%20Camera%20profile%20video.mov

You can't get a match between display and print, there is this:

Why are my prints too dark?
Why doesn’t my display match my prints?
A video update to a written piece on subject from 2013
In this 24 minute video, I'll cover:

Are your prints really too dark?
Display calibration and WYSIWYG
Proper print viewing conditions
Trouble shooting to get a match
Avoiding kludges that don't solve the problem

High resolution: http://digitaldog.net/files/Why_are_my_prints_too_dark.mp4
Low resolution: https://youtu.be/iS6sjZmxjY4

[Colinml]

Thanks so much for this.  I watched everything, and learned a lot.  I turned off gamut warnings.  (I was one of those who were misled into thinking that the object was to manually edit the soft proof to "fix" the out of gamut warnings).  Now I see how this doesn't make sense.  As for my specific issue, I'm delighted and puzzled to say that it's gone.  Now soft proofing seems to be working normally and my print looks like the proof preview.  I think my confusion has something do do with not completely understanding the before/after options when soft proofing is enabled.  I think I need to look further into DNG camera profiles.  Hadn't heard of this at all before.  Thanks again for the help.  -Colin