The render queue isn't AE's best option. Ideally you should export to Adobe Media Encoder, which has lots of render presets. You can tweak the settings there too.
A bit of history: render queue used to be the only option, but it meant you could work on anything in AE while it was rendering. AME took over and it also works with Premiere Pro in a similar way. I only use the render queue for when I need to render out interim mp3 files and only because it means I don't have to change settings in AME each time I use it.
Check the User Guide. There is an entire section on rendering.
After Effects is an incredibly complex app with hundreds of millions of possible workflows. I've never met anyone that has successfully just opened the app and started to create something original or even usable with it. AE is not a video editing app. It is designed to create shots or short sequences that can be edited into a finished film using an NLE like Premiere Pro. Most of your work in AE should be just one shot and it will probably be less than 10 seconds. Start in the Learn Workspace or use the Learn resources in the Home Screen that pops up by default when you open After Effects for the first time. Spend some time with the User Guide. Do not customize composition settings and don't customize render settings until you have a decent understanding of video standards, formats, codecs, and frame rates.