Thanks guys,
These are good suggestions. It's close, but not perfect.
@Samedog, I have cuts in my captions, so I can't export my captions as one full asset.
@Jeff, thanks for this, but it wasn't quite what I was looking for, but reading the article helped me think outside the box.
Here's what I do now:
Instead of exporting a movie file and sidecar .scc file. I set the settings to export the audio only with the sidecar .scc. It still takes about a minute to export, but at least it won't take an hour to export (this is the time I get for exporting the video file with it)
Thanks all!