DISCUSS: New Captions Workflow in Premiere Pro

[Francis-Crossman]

UPDATE: January 29, 2021

Since the innitial post back in October of last year we have made a ton of progress and some of the details originally posted here are no longer acurate.  I have updated the post to be more accurate to the state of things today.  

Here are some important changes to be aware of

• The Premiere Pro beta is now version 15 (starting January 26, 2021)
• The captions workflow in on by default - no need to enable it
• The beta menu has been removed (because you don't need it anymore - the advanced beta features are always on)
• Applications for speech to text early access has closed and those who have been accepted should have recieved an email.  The email you used to apply needs to match the email (Adobe ID) you use to log into creative cloud. If you did not recieve an email stating that you have been accepted, or if you are logged into creative cloud using a different email you will contininue to see the message stating that speech to text is coming soon.
• Broadcast closed captions are now supported CEA 608/708, OP47, Teletext, EBU Subtitles.
• Exporting is working (burned-in, sidecar, embedded) and queue to Media Encoder is working too now.

Thanks to everyone for testing and for all your awesome feedback! - Francis

The captioning workflow in Premiere Pro is getting a complete makeover and you can be the first to try it out!  We have a new text panel for editing and viewing your captions as text.  Captions now have their own track on the timeline so they can be edited just like video.  Styling uses the power of the Essential Graphics panel so you can make fantastic looking captions. 

Best of all – we will be adding speech to text and auto captions powered by Adobe Sensei – coming later.

 

Some important things to know before you start testing this feature

1. Just like any beta software, we do not recommend using this for mission critical projects since features are likely to change over time. 
2.  The Premiere Pro beta has been updated to the next major version – 15.  This means your projects will NOT be backwards compatible with the current shipping version of Premiere Pro – 14.x.  All projects created in Premiere Pro Beta, even if you don’t use captions, will be saved as version 15. 
3. Opening existing projects in Premiere Pro Beta will ask you to save a copy of your project in the new project version. Take extra care with Team Projects since you do not get a version update warning and you can mess up the project for your entire team!
4. If you have existing captions in a project that gets upgraded, we will convert from the old style captions to the new style.  We will do our best to match the styling, but do not guarantee a pixel-perfect transition. 

So, if you’re good with all that, read on . . .

Enabling the Feature:

1. Update to the latest Beta version - you're done.  It's on by default now

Quickstart Guide: 

Download this sample project to follow along PR New Captions Workflow quickstart.zip

1. Open the sequence The climber - start here and familiarize yourself with the spoken dialog.
2. Open the Text panel from the window menu and make sure it’s showing the Captions tab.
3. Locate the SRT file in the project panel The Climber - subtitles.srt
4. Drag the SRT file into the sequence and drop it anywhere. 
5. A new caption track is created, and all the caption items are placed at the correct time.
6. Place the playhead over one of the caption items to select it.  This should activate the Essential Graphics panel into editing mode.
7. Change the Style in the Essential Graphics panel by choosing a different preset from the Style dropdown.
8. Notice the font, size, color, and background have been updated.
9. Play the sequence and watch the captions follow along in the Text panel.

Play around with visual styles in the Essential Graphics panel and try editing the text in the Text panel.  Read on for more detailed info on the various pieces of the workflow.

The Text Panel
Open the Text panel from the Window menu.  This is where you interact with text and it has two tabs – Transcript and Captions

 

1. Text panel – open from the Window Menu
2. Transcript tab – This is where you can create automatic speech to text transcriptions, navigate the transcript, and create automatic captions.  This is only available to those who been accepted into the eary access program for speech to text.  All others will continue to see the "coming soon" graphics.
   
3. Captions tab – Navigate and edit caption text.
4. Search – find characters, words and phrases in your captions.
5. Next / Previous search result – use to navigate search results.
6. Replace / Replace All – used to replace search results with new text.
   
7. Add Caption – this will place a blank caption at the playhead in the current sequence
8. Caption number – a sequence number to count your captions
9. Timecode – TC start and stop of the current caption item.  This is not editable.  To change the timing of the caption, edit it in the sequence.
10. Caption text – the text of the caption itself.  Double click to edit.

The Caption track - working with captions in the Sequence

We have completely reimagined the way captions work in Premiere Pro. If you have used captions previously in Premiere Pro, this is totally different – much better we think. Please tell us what YOU think by leaving a public comment below.

 

Create and delete caption tracks

There are a few ways to create a new caption track.  You can drag an SRT file onto the Sequence or use the “Create new captions track” button in the Captions tab of the Text panel. To delete a caption track, right-click on the caption track header and choose “delete track”

Drag an SRT file on the sequence

Dragging an SRT caption file onto the Sequence will automatically create a new caption track at the top of the Timeline.

Keyboard Shortcuts

To see all the keyboard shortcuts available for captions, open the keyboard shortcuts menu from the Premiere Pro menu and search for "caption".  Here are some notable ones.  There are many other which do not have default assignments.

1. Add a captions track - option/alt + command/ctrl + A
2. Add a new caption segment at the playhead- option/alt + command/ctrl + C
3. Go to next caption segment in the timeline - option/alt + command/ctrl + UP
4. Go to next caption segment in the timeline - option/alt + command/ctrl + down

Captions tracks

 

1. Caption track setting – the CC button

This can hide the caption track area or solo just the active caption track if you have more than one.  It is useful if you want to declutter your timeline. It does not turn off the active caption track from rendering in the program monitor, however. 

• Hide all caption tracks
• Show all caption tracks
• Show active caption track only 

2. Active Track/Visibility – the eyeball icon

This will enable/disable Program Monitor viewing of the Caption track selected. Only one Caption track can be active at a time, so when you make one track active, the other tracks will automatically be disabled. You can also choose to disable all Caption tracks. Inactive Caption tracks will dim making it easier to note your active Caption track. 

3. Label

By default, this will show the caption track format (eg Subtitles, CEA-708, CEA-608, etc.) You can right-click the Caption track header and choose Rename to choose a custom name (same functionality as A/V tracks)

4. Right-click the caption track header

Additional caption tracks can be added by right-clicking in the caption track header – choose “Add Track”. You can also delete or rename a caption track in the same way.

5. Caption tracks area

Caption tracks will always be at the top of the sequence. Additional caption tracks are added on top. This will continue until you have reached 25% of the Timeline vertical space, then the tracks will stop “growing” and scroll bars will appear to the right. Caption tracks will never take up more than 25% of the upper space of the Timeline. This is dynamic, so if you increase or decrease the Timeline panel height, the 25% will adjust accordingly.

6. Track Lock

Lock the track to prevent editing.  This functions the same as A/V tracks.

7. Track Targeting

The Caption track header has a ‘Toggle the track targeting for this track’ button with same functionality as A/V tracks.

8. Sync Lock

The Caption track header has a ‘Toggle Sync Lock’ button with same functionality as A/V tracks.

Editing timing of caption items on the track

You can use all the familiar editing tools that you are used just like editing video and audio.

 

• Select (one or many) You can select multiple Caption track items by Shift-selecting each item or marquee/lasso select items or use Select All (which will also include other items in tracks like video, audio, etc.)
• Trim
• Ripple Trim
• Razor/Add Edit: You can add an edit/razor a Caption item as you would with any Timeline item and Track Targeting is supported. The difference is that when you razor a Caption item, both items will have the same text which you can then modify in the Text panel
• Copy/Paste: paste of Caption item(s) follows Track targeting which allows you to copy and paste captions items between tracks.

Styling with the Essential Graphics panel

Open the Essential Graphics panel from the Window menu.  This is where you choose things like font, size, color and position. Make sure you have at least one caption selected. This will activate the Edit tab of the Essential Graphics panel. If you have created and stylized text in Premiere Pro before, the following should be very familiar to you. However, for captions we are adding small enhancements such as zones, and vertical text alignment.

Styles

• Create Master Text Styles for consistent styling across the entire captions track. A style saves all the settings made in the Essential Graphics panels, including font, alignment, color and more. Setting a Master Text Style to one caption applies it to all captions on the track. You can have different styles for different tracks.
• Sync from Master Style and Push to Master Style: When changing the look of a caption you may want to push this change to the entire track for consistency, that’s were Push to Master Style comes into play. Or you might want to revert an edit to a caption back to the Master Style. Then use Sync from Master Style.
•  

Text

• Font: Set font, font style and font size.
• Paragraph Alignment: For horizontal alignment use Left align text, Center align text, Right align text and Justify. We have now also added Vertical alignment with Top align text, Center text vertically and Bottom align text. This defines how a caption grows when adding additional lines. As an example, there is a good chance you will want a caption to be bottom aligned, this way a single line caption and the second line of a two-line caption will always be in the same vertical position.
• Tracking: Loose or tighten the space between characters.
• Leading: Loose or tighten the vertical space between lines.
• Faux Styles: Bold, Italic, All Caps, Small Caps, Superscript, Subscript, Underline.

Align and Transform

• Position captions with zones: You can choose from different zones to position your caption in different areas on screen, e.g. bottom center.
• Fine tune position: Through Set Horizontal and Set Vertical Position you can add an offset to your zone setting.
• Change the text box size: If you want to shrink or expand the text box size you can do this through Set Horizontal Scale and Set Vertical Scale. This will affect text wrapping and paragraph align settings.

Appearance

• Fill: Change the color of your captions.
• Stroke: Add single or multiple strokes. The Graphics Properties menu under the wrench menu gives you more control over stroke styling.
• Background: Add a background box. You can choose the color, add additional padding and change the opacity.
• Shadow: You can add a shadow and fine tune with controls such as opacity, angle, distance and more.

Transcribing the dialog into captions

You have three options for transcribing your audio and creating captions: speech to text, use a third-party service, or do it by hand.

Speech to text

• Text Panel > Transcript tab
• Only available for those who have applied and been accepted to the speech to text early access program.  Other will not have access to this feature.

Use a third-party service

If you already have a workflow that uses a third party transcription service, you can bring in that file.  SRT is the best option.

1. Import the SRT file into your Premiere Pro project just like any piece of media
2. Drag the SRT from the project panel into your sequence and let go anywhere
3. A new caption track is automatically created, and the captions are placed on the track

Do it by hand

For short sequences, doing it by hand may not too much trouble.

1. Open Text panel from the Window menu
2. Go to the Captions tab of the Text Panel
3. Press the “Add Captions Track” button – a new captions track is added to your current sequence
4. Place the playhead at the beginning of your first piece of dialog (hint – use the waveforms in the audio to help align)
5. Press the “+” button near the top of the Captions tab of the Text Panel to add a blank caption
6. Double click on <Type your caption here> to go into edit mode
7. Type out the caption text
8. Trim the end of the caption in the timeline to align with the end of the spoken dialog
9. Repeat for each caption you wish to add
10. See the other sections on Working with captions in the Sequence and Styling with the Essential Graphics panel for more details on editing and styling.

Exporting your sequence with captions

There are three options for exporting captions: burned in, sidecar file, or embedded into the video file.  

1. Make sure the caption track you want exported is visible by toggling the eyeball on (Toggle Active Captions Track).
2. Choose File > Export
3. Open the Captions tab in the export settings and choose burn in, sidecar or embedded
4. Burned in and sidecar are supported with any encoding preset
5. Embedded is limited to pro codecs like MXF OP1a, DNX, and Prores and requires that the caption track format be set to one of the broadcast standards like CEA-608
6. Click Export
7. Sidecar files will be saved next to the video file with the same name
8. Queueing to Media encoder is supported too

Happy captioning!  Once again, we welcome your feedback.  Please leave comments below.

[Nick Lear]

This is good to know. I feel like if it can done like that, it can't be that hard to code it within Premiere so I hope this does get done.

[Nick Lear]

The background box shape has been updated in recent weeks - which is a partly good and partly bad.

It used to be tight around the text, ignoring uppercase & ascenders (h,i etc.) and descenders (g,j,y etc.).

Now it is allowing for those which is great, but it means if you have a run of subtitles and a few sentences happen to not have them , then the box suddenly becomes smaller and that looks very glitchy and catches the eye in a bad way.

It would be much better if the box stayed the same size always, allowing for ascenders and descenders to be there even if they occasionally aren't.

[Pierre Louis B.]

Yes, this needs to be addressed, 100%!  Users need a checkbox option to easily turn constant vertical sizing on/off.

[Liam5FC5]

Okay!  I've been playing with these new tools for a bit now and I finally have my feedback for the tech team.  These are problems I experienced and some changes I hope to be made.  I'm a video editor for content creators mostly on youtube, so be aware that is where I am coming from and who I am representing.

Firstly, there is no function to convert the caption track into essential graphics.  I think this should most definitely be a feature.  I have already found some third party softwares that do this and seeing that I know this is too simple to not have built in.  You should be able to drag down your caption track to duplicate it into a video track, this would seamlessly turn the captions into essential graphics format titles.  The reason this is necessary for a lot of editors is because people like to animate their captions/subtitles and apply effects to them.  This, of course, is not possible with the caption style tracks.

Next, I think the animation in the subtitling panel is a bit excessive.  It seems to slow down my build and also slow down the ability to change text as you have to wait for the page to scroll to your correct text before you can type and change.  I think this should instead snap around, as pretty as it looks, it slows down the working process.

This is something I'm not entirely sure I understand fully, I could be missing something and in that case, my bad.  The speaker identifying function is great, but as of now I can't seem to apply certain text settings to various speakers.  Lots of youtubers style their cast's subtitles differently so the viewer can identify who is talking, this is something that I would have hoped could be driven by the speaker tags.  I can't seem to find a way to do that, although like I said, it's possible I am missing something.  If this is not currently available, this should most definitely be added as it is going to be a very commonly used tool.  Maybe a function where you can apply a text style to an entire speaker instead of just the individual caption piece?

Otherwise, this tool is super helpful and I'm glad it's in Premiere now, thanks to the tech team for all of this hard work, it paid off and is definitely helping a lot of editors now!

[mattyboy1980]

Firstly, there is no function to convert the caption track into essential graphics.  I think this should most definitely be a feature.  I have already found some third party softwares that do this and seeing that I know this is too simple to not have built in.  You should be able to drag down your caption track to duplicate it into a video track, this would seamlessly turn the captions into essential graphics format titles.  The reason this is necessary for a lot of editors is because people like to animate their captions/subtitles and apply effects to them.  This, of course, is not possible with the caption style tracks.


This a 1000% - it would solve a whole host of issues with more freedom to position them and also replicate them for particular styles of different companies

Also being able to have more than one captions tracks visible would be handy for different styles on the same sequence

It looks like the captions in general have been geared toward broadcast, when I'd assume the reality is that the majority of users need it for reversioning a clients video for social media with subs.

[MrSenpai]

Do you know the name or have a link to the 3rd party tool ? 

[mattyboy1980]

Hey John,

Only on windows unfortunately - https://www.subtitletools.net/subtitle2xml

I'm going to try and run an emulator to try test it out....

[MrSenpai]

Thank you 🙏

[mattyboy1980]

No worries, if you test it out let me know how you get on? 

[digitlman]

It looks like the captions in general have been geared toward broadcast, when I'd assume the reality is that the majority of users need it for reversioning a clients video for social media with subs.

By @mattyboy1980

Yes that is true. Closed Captioning was created around 1971 for televisions to decoded those captions sent out by broadcast television stations.  A law was passed that by 1993 all 13" or larger televisions were required by the FCC to decode closed captions. It is a fixed system with very specific capabilities. All television stations are required by law to have these closed captions in all television shows that we broadcast. Premiere creates CEA-608 and CEA-708 Closed captions which can be decoded by every television set in the world to abide by the laws that have been created. They have a very specific criteria for what can be decoded with size, font, color, and location.

Some Premiere users it seem really want to take this Closed Captioning system and turn it into something else to make some sort of animated motion text graphics systems for videos. 

However that is simply not what Closed Captioning was created or designed for. Hopefully Adobe can focus on this system and get it finished and functional for the broadcast requirements. Then maybe they can work on adding other animated graphics non-standard features people are requesting.

[mattyboy1980]

I appreciate that the original purpose of captions are for broadcast requirements. And I hope Adobe, as you say, manage to get them working effectively for this purpose. But you also have to respect the fact the majority of users are not using them for broadcast. 

I'm a purist, I work with VHS, Super 8, and 16mm. I'm an 80's kid & love working with older formats. But my bread and butter is freelance editing for clients (mostly digital strands of TV stations), and subtitles are an integral part of that workflow for FB, YT, Twitter, and IG. 

Personally, I'm not asking for the captions function to be an exclusively all singing and dancing online-geared application. But.... it is completely logical and understandable for editors to request a means in which captions can be adpated for their own work and for greater flexibility to replicate clients branding or style. To have a broadcast and an online option would surely be the best of both worlds. 

If you're arguing otherwise, might be a good idea to start modding up a DeLorean.

[digitlman]

Yes i understand your desires. However the requirements for online streaming broadcast are the same. We export our captions out of premiere and apply them to our youtube videos just the same as we do our television broadcast captions. 

Really i was referencing some previous requests that have been made here that were tantamount to asking for closed captions to allow you have multicolored text fly in from off screen and twirl it around and flash colors and have little animated twinkly stars on it, etc...  essentially wanted CC to be equivilent to text animation styles that are done in after effects and i simply feel its unrealistic to have adobe modify the CC system to do that kind of stuff.

But anyways sorry, i will digress as i dont want to keep going off the subject of fixing bugs and improving editing capabilites. 

[mattyboy1980]

For me all I require is the ability to position the text where I want. Regardless of what you do with them though, the ability should be there to adapt them the same as you can adapt anything else with effect controls and essential graphics. I'm certainly not requesting any magical new features akin to After Effects modifications, and I agree that anything other than that is overkill - but the point is we should have the ability to edit them using the tools we already have within Premiere. Sorry if my position on this was not clear.

And I'm certainly not trying to start an argument, it's just for reversioning to 1x1 or 9x16 the broadcast constraints are a problem. It's not so much 16x9 videos, but anything else and yes it certainly is an issue. Whether mobile viewing is a problem in a wider-context that's another conversation (!), but whether we edit for broadcast or for online delivery, we all pay the same subscription fees. And I don't think it's asking for much to be able to modify them using the tools we have in place already.  

I don't see the big problem why for example there couldn't be a 'convert to video layer' option which would eradicate many of the issues that users appear to be having in regards to positioning and altering of text. It's a great step forward with captions, and I'm sure will keep improving with future updates.

[Wholeo]

Personally, I rely on CC when watching TV or online videos. So I am keenly interested. I know CEA-608 is standard, but I see different formats with my transcribed captions in Premiere, how they appear when I upload to YouTube or Vimeo, and the CC on TV programs. I wonder how my work would look if broadcast? Also, TVs have settings for CC.

On a different note, it would be nice to reply in discussions in email but the mailbox is full. 

[Aflalo]

Good morning friends. This is the first time I'm working with the new captioning workflow so please forgive me if any of this has been previously addressed. If so, I cannot find the answers. 

I have gone ahead and created captions for a short 15 second promo and while the workflow was great, the program monitor is showing me a traditional type of caption text and also a formatted type. When I physically delete the caption block in the timeline, the formatted version then disappears and keeps the traditional format in place. See the below image. 

Is this just a matter of toggling something on or off?

[Stan Jones]

A screenshot of the timeline and the Text (caption) Panel would help....

Stan

[Aflalo]

But of course! 

[Stan Jones]

When you turn off the eyball on the caption track, do both captions disappear? I'm wondering if you have a burned in caption? 

If turning off the caption track does not remove both, turn off each video layer one at a time.

Stan 

[Aflalo]

Yeah, I tried that. When I turn off the caption track it hides the stylized captions but continues to show the other. When I start hiding other tracks that "other" caption remains the entire time.

It's the oddest thing. When I view the exported MXF after rendering with embedded captions, I only see the stylized version.

[Stan Jones]

Let's see what someone thinks. What was your source of the caption stream?

When I emported an embedded mxf, the 608 stream seemed locked. 

Stan

[Aflalo]

My own source files, recorded here. Nothing has burnt in captions for sure.

I wrote the captions myself manually.

[Stan Jones]

Weird for sure.

I'm Win 10, running the release version to test. I'll try to test with the latest Beta in a bit. But Pickleball time is drawing close!

Provide your specs so staff will know...

Stan

[Aflalo]

Thanks.

Running MacPro 2019 Rack
3.2 GHz 16-Core Intel Xeon W
192 GB 2933 MHz DDR4
AMD Radeon Pro W5700X 16GB

Big Sur 11.1

Premiere Pro Version 15.0.0 Build 41

[Stan Jones]

Just FYI, a couple of other users have reported this in the release version. I think also mac users.

One filed a bug report:

https://adobe-video.uservoice.com/forums/911233-premiere-pro/suggestions/42914085-closed-captions-in...

Stan

[MelonShaveIce]

I'm a Japanese who makes Youtube videos as a hobby.
I'm experimenting with new caption workflows and speech recognition features and am excited about this amazing update!

This time, I would like to make two requests from a Japanese perspective.

1.

I need the ability to export the text retrieved from speech recognition on the Transcript tab to a video track instead of a caption track.

Ideally, you should be able to choose a Essential Graphics Template for each speaker that is distinguished by speech recognition.

At least in Japan, it is common to change the color and decoration of the character depending on the speaker.
We also often use character decorations and animations to express our emotions.

Currently, you can only select one track style for the caption track.
Also, I couldn't switch the text color for each speaker because I couldn't enable multiple subtitle tracks at the same time.

If you can export text from speech recognition to a video track, this problem will be solved.

2.

I would like you to add an option that does not output punctuation marks by voice recognition.

In Japanese, there is a punctuation mark "。". This has the same meaning as the English sentence ending symbol ".". This means the end of the sentence.
Captions printed on Japanese videos often omit punctuation at the end of the sentence and must be manually removed.

Even in Japanese, the accuracy of voice recognition itself is higher than expected, and I hope that the work time for creating videos can be significantly reduced.