you can export a sequence with a caption track embedded in the vertical interval of the video (at least I assume that's where it's embedded) so that when you import the file back in to premiere (or another app) you would have the option to view or import the captions. Here's the relevant help url and the relevant text.
Embedded:The option to export as Embedded is available only for pro codecs like MXF OP1a, DNX, and ProRes and requires that the caption track format be set to one of the broadcast standards like CEA-608.
QuickTime embedded export currently only supports embedding CEA-608. MXF Op1a Formats can embed CEA-608, CEA-708, and Australian OP-47 (if 25fps). You will also need those track types in your sequence in order to embed. For example, if you have an SRT in sequence, that cannot be embedded. You must have a compatible format such as 608, 708 or Aus OP-47.