Here are some things that should improve quality:
Make sure you're animating in an ActionScript 3 FLA and not an HTML5 one.
In the Actions panel for frame 1, add this code:
stage.quality = "16x16";
Leave everything that is vector as vector.
Make all bitmaps in the library be Lossless and Allow Smoothing. You can select them all in one go and click the i button in the library, to change them all.
If you're using sound, only import WAV, 44.1 kHz, 16 bit (mono or stereo). In Publish Settings set event and stream audio to Raw, 44.1 kHz, 16 bit (mono or stereo).
When you do a File/Export/Export Video... you will get an Animation codec MOV. That's better quality than ProRes 422, though it is a different colorspace.
Not everyone has Apple ProRes. Although there's a good chance a competition for video entries would have, you might want to make an H.264 MP4 version, in case they don't have ProRes or don't have QuickTime.
You could use Adobe Media Encoder with the highest quality H.264 setting you can find (if you do it manually, try 20 megabits per second, the video may not need that much, so if it comes in less that's probably ok).
If you're using Mac, try opening the MOV from Animate in QuickTIme Player, and use its File/Export As/1080p. It uses GPU to do the H.264 encoding, and is quite fast. It's likely to come out at around 10 mbps, so do still try a test with media encoder.