Short answer: the functionality you're looking for does not currently exist in the After Effects-to-AME workflow.
When AME is encoding an After Effects comp, it does not have the capability to affect how the After Effects comp is rendered. The comp will always be rendered at it's inherent properties. Even when you scale the encoding settings in AME, the full comp is rendered and scaled only during encoding.
Caveat: in AME CS6, we started some work to allow this functionality to exist. When you right-click on a comp in the AME queue and click Source Settings, you can control one of the render settings from After Effects: whether or not Guide Layers are visible. Our intention is to expand upon this functionality, though adding settings that affect the properties of the comp (as opposed to visibility of layers) is much more difficult and remains on our to-do list.
Also, the composition viewer settings do not affect the inherent properties of the comp itself, it only affects how the comp is viewed.
Since you've correctly deduced the differences between rendering and encoding, I expect you've also already thought of the workarounds to get what you want (a faster-rendered comp that is encoded in AME), but let's summarize these ideas anyway:
1) Render a lossless intermediate file from After Effects, and encode that to your delivery format with AME, possibly using Watch Folders to automate the AME side of the process which is what I'd recommend.
2) Nest and scale your comp into a smaller comp that matches your desired settings, and encode that comp in AME.
As always, please send us your thoughts about how you'd like to see the After Effects-to-AME workflow improved: http://adobe.ly/feature_request