Help! Need smoother lines.

All right friends, need help with this.

Here is a detail from an illustration I am working with in photoshop. The problem: the lines are jaggy, and when I zoom out to 100%, they are visibly pixelated. The resolution is set to 300dpi, which--based on the reading I've done so far--should be high enough to create a quality non-pixelated image. However, the lines themselves are not smooth enough. The lines are their own layer.

How do I get smoother lines here, without creating some awkward whitespace between the colors? Is there a way to anti-alias this very complex selection of lines?

The image, as you can see, is very detailed (this is just a 2-inch section of a much larger 22 x 8.5 image). The pen-tool seems too unruly for an image this complex. Illustrator's image-trace wasn't precise enough, because, I think, the linework is so complicated.

Would greatly appreciate any insight you might have.