Yes, Export options will be coming later. Currently, you can only burn-in Captions for Export and all Caption tab options are disabled.
From the New Captions Workflow post:
Exporting your sequence with captions burned in
The pieces of the workflow that are ready to test right now are for creating really nice looking burned in captions and exporting directly from Premiere Pro. We will certainly be adding the ability to export a subtitle file such as SRT before we release this workflow but that is not functional right now.
Excellent, thank you for the swift response!
Keep checking back with newer Beta builds. We plan on adding more Caption functionality over time.