DISCUSS: New Captions Workflow in Premiere Pro

[Francis-Crossman]

UPDATE: January 29, 2021

Since the innitial post back in October of last year we have made a ton of progress and some of the details originally posted here are no longer acurate.  I have updated the post to be more accurate to the state of things today.  

Here are some important changes to be aware of

• The Premiere Pro beta is now version 15 (starting January 26, 2021)
• The captions workflow in on by default - no need to enable it
• The beta menu has been removed (because you don't need it anymore - the advanced beta features are always on)
• Applications for speech to text early access has closed and those who have been accepted should have recieved an email.  The email you used to apply needs to match the email (Adobe ID) you use to log into creative cloud. If you did not recieve an email stating that you have been accepted, or if you are logged into creative cloud using a different email you will contininue to see the message stating that speech to text is coming soon.
• Broadcast closed captions are now supported CEA 608/708, OP47, Teletext, EBU Subtitles.
• Exporting is working (burned-in, sidecar, embedded) and queue to Media Encoder is working too now.

Thanks to everyone for testing and for all your awesome feedback! - Francis

The captioning workflow in Premiere Pro is getting a complete makeover and you can be the first to try it out!  We have a new text panel for editing and viewing your captions as text.  Captions now have their own track on the timeline so they can be edited just like video.  Styling uses the power of the Essential Graphics panel so you can make fantastic looking captions. 

Best of all – we will be adding speech to text and auto captions powered by Adobe Sensei – coming later.

 

Some important things to know before you start testing this feature

1. Just like any beta software, we do not recommend using this for mission critical projects since features are likely to change over time. 
2.  The Premiere Pro beta has been updated to the next major version – 15.  This means your projects will NOT be backwards compatible with the current shipping version of Premiere Pro – 14.x.  All projects created in Premiere Pro Beta, even if you don’t use captions, will be saved as version 15. 
3. Opening existing projects in Premiere Pro Beta will ask you to save a copy of your project in the new project version. Take extra care with Team Projects since you do not get a version update warning and you can mess up the project for your entire team!
4. If you have existing captions in a project that gets upgraded, we will convert from the old style captions to the new style.  We will do our best to match the styling, but do not guarantee a pixel-perfect transition. 

So, if you’re good with all that, read on . . .

Enabling the Feature:

1. Update to the latest Beta version - you're done.  It's on by default now

Quickstart Guide: 

Download this sample project to follow along PR New Captions Workflow quickstart.zip

1. Open the sequence The climber - start here and familiarize yourself with the spoken dialog.
2. Open the Text panel from the window menu and make sure it’s showing the Captions tab.
3. Locate the SRT file in the project panel The Climber - subtitles.srt
4. Drag the SRT file into the sequence and drop it anywhere. 
5. A new caption track is created, and all the caption items are placed at the correct time.
6. Place the playhead over one of the caption items to select it.  This should activate the Essential Graphics panel into editing mode.
7. Change the Style in the Essential Graphics panel by choosing a different preset from the Style dropdown.
8. Notice the font, size, color, and background have been updated.
9. Play the sequence and watch the captions follow along in the Text panel.

Play around with visual styles in the Essential Graphics panel and try editing the text in the Text panel.  Read on for more detailed info on the various pieces of the workflow.

The Text Panel
Open the Text panel from the Window menu.  This is where you interact with text and it has two tabs – Transcript and Captions

 

1. Text panel – open from the Window Menu
2. Transcript tab – This is where you can create automatic speech to text transcriptions, navigate the transcript, and create automatic captions.  This is only available to those who been accepted into the eary access program for speech to text.  All others will continue to see the "coming soon" graphics.
   
3. Captions tab – Navigate and edit caption text.
4. Search – find characters, words and phrases in your captions.
5. Next / Previous search result – use to navigate search results.
6. Replace / Replace All – used to replace search results with new text.
   
7. Add Caption – this will place a blank caption at the playhead in the current sequence
8. Caption number – a sequence number to count your captions
9. Timecode – TC start and stop of the current caption item.  This is not editable.  To change the timing of the caption, edit it in the sequence.
10. Caption text – the text of the caption itself.  Double click to edit.

The Caption track - working with captions in the Sequence

We have completely reimagined the way captions work in Premiere Pro. If you have used captions previously in Premiere Pro, this is totally different – much better we think. Please tell us what YOU think by leaving a public comment below.

 

Create and delete caption tracks

There are a few ways to create a new caption track.  You can drag an SRT file onto the Sequence or use the “Create new captions track” button in the Captions tab of the Text panel. To delete a caption track, right-click on the caption track header and choose “delete track”

Drag an SRT file on the sequence

Dragging an SRT caption file onto the Sequence will automatically create a new caption track at the top of the Timeline.

Keyboard Shortcuts

To see all the keyboard shortcuts available for captions, open the keyboard shortcuts menu from the Premiere Pro menu and search for "caption".  Here are some notable ones.  There are many other which do not have default assignments.

1. Add a captions track - option/alt + command/ctrl + A
2. Add a new caption segment at the playhead- option/alt + command/ctrl + C
3. Go to next caption segment in the timeline - option/alt + command/ctrl + UP
4. Go to next caption segment in the timeline - option/alt + command/ctrl + down

Captions tracks

 

1. Caption track setting – the CC button

This can hide the caption track area or solo just the active caption track if you have more than one.  It is useful if you want to declutter your timeline. It does not turn off the active caption track from rendering in the program monitor, however. 

• Hide all caption tracks
• Show all caption tracks
• Show active caption track only 

2. Active Track/Visibility – the eyeball icon

This will enable/disable Program Monitor viewing of the Caption track selected. Only one Caption track can be active at a time, so when you make one track active, the other tracks will automatically be disabled. You can also choose to disable all Caption tracks. Inactive Caption tracks will dim making it easier to note your active Caption track. 

3. Label

By default, this will show the caption track format (eg Subtitles, CEA-708, CEA-608, etc.) You can right-click the Caption track header and choose Rename to choose a custom name (same functionality as A/V tracks)

4. Right-click the caption track header

Additional caption tracks can be added by right-clicking in the caption track header – choose “Add Track”. You can also delete or rename a caption track in the same way.

5. Caption tracks area

Caption tracks will always be at the top of the sequence. Additional caption tracks are added on top. This will continue until you have reached 25% of the Timeline vertical space, then the tracks will stop “growing” and scroll bars will appear to the right. Caption tracks will never take up more than 25% of the upper space of the Timeline. This is dynamic, so if you increase or decrease the Timeline panel height, the 25% will adjust accordingly.

6. Track Lock

Lock the track to prevent editing.  This functions the same as A/V tracks.

7. Track Targeting

The Caption track header has a ‘Toggle the track targeting for this track’ button with same functionality as A/V tracks.

8. Sync Lock

The Caption track header has a ‘Toggle Sync Lock’ button with same functionality as A/V tracks.

Editing timing of caption items on the track

You can use all the familiar editing tools that you are used just like editing video and audio.

 

• Select (one or many) You can select multiple Caption track items by Shift-selecting each item or marquee/lasso select items or use Select All (which will also include other items in tracks like video, audio, etc.)
• Trim
• Ripple Trim
• Razor/Add Edit: You can add an edit/razor a Caption item as you would with any Timeline item and Track Targeting is supported. The difference is that when you razor a Caption item, both items will have the same text which you can then modify in the Text panel
• Copy/Paste: paste of Caption item(s) follows Track targeting which allows you to copy and paste captions items between tracks.

Styling with the Essential Graphics panel

Open the Essential Graphics panel from the Window menu.  This is where you choose things like font, size, color and position. Make sure you have at least one caption selected. This will activate the Edit tab of the Essential Graphics panel. If you have created and stylized text in Premiere Pro before, the following should be very familiar to you. However, for captions we are adding small enhancements such as zones, and vertical text alignment.

Styles

• Create Master Text Styles for consistent styling across the entire captions track. A style saves all the settings made in the Essential Graphics panels, including font, alignment, color and more. Setting a Master Text Style to one caption applies it to all captions on the track. You can have different styles for different tracks.
• Sync from Master Style and Push to Master Style: When changing the look of a caption you may want to push this change to the entire track for consistency, that’s were Push to Master Style comes into play. Or you might want to revert an edit to a caption back to the Master Style. Then use Sync from Master Style.
•  

Text

• Font: Set font, font style and font size.
• Paragraph Alignment: For horizontal alignment use Left align text, Center align text, Right align text and Justify. We have now also added Vertical alignment with Top align text, Center text vertically and Bottom align text. This defines how a caption grows when adding additional lines. As an example, there is a good chance you will want a caption to be bottom aligned, this way a single line caption and the second line of a two-line caption will always be in the same vertical position.
• Tracking: Loose or tighten the space between characters.
• Leading: Loose or tighten the vertical space between lines.
• Faux Styles: Bold, Italic, All Caps, Small Caps, Superscript, Subscript, Underline.

Align and Transform

• Position captions with zones: You can choose from different zones to position your caption in different areas on screen, e.g. bottom center.
• Fine tune position: Through Set Horizontal and Set Vertical Position you can add an offset to your zone setting.
• Change the text box size: If you want to shrink or expand the text box size you can do this through Set Horizontal Scale and Set Vertical Scale. This will affect text wrapping and paragraph align settings.

Appearance

• Fill: Change the color of your captions.
• Stroke: Add single or multiple strokes. The Graphics Properties menu under the wrench menu gives you more control over stroke styling.
• Background: Add a background box. You can choose the color, add additional padding and change the opacity.
• Shadow: You can add a shadow and fine tune with controls such as opacity, angle, distance and more.

Transcribing the dialog into captions

You have three options for transcribing your audio and creating captions: speech to text, use a third-party service, or do it by hand.

Speech to text

• Text Panel > Transcript tab
• Only available for those who have applied and been accepted to the speech to text early access program.  Other will not have access to this feature.

Use a third-party service

If you already have a workflow that uses a third party transcription service, you can bring in that file.  SRT is the best option.

1. Import the SRT file into your Premiere Pro project just like any piece of media
2. Drag the SRT from the project panel into your sequence and let go anywhere
3. A new caption track is automatically created, and the captions are placed on the track

Do it by hand

For short sequences, doing it by hand may not too much trouble.

1. Open Text panel from the Window menu
2. Go to the Captions tab of the Text Panel
3. Press the “Add Captions Track” button – a new captions track is added to your current sequence
4. Place the playhead at the beginning of your first piece of dialog (hint – use the waveforms in the audio to help align)
5. Press the “+” button near the top of the Captions tab of the Text Panel to add a blank caption
6. Double click on <Type your caption here> to go into edit mode
7. Type out the caption text
8. Trim the end of the caption in the timeline to align with the end of the spoken dialog
9. Repeat for each caption you wish to add
10. See the other sections on Working with captions in the Sequence and Styling with the Essential Graphics panel for more details on editing and styling.

Exporting your sequence with captions

There are three options for exporting captions: burned in, sidecar file, or embedded into the video file.  

1. Make sure the caption track you want exported is visible by toggling the eyeball on (Toggle Active Captions Track).
2. Choose File > Export
3. Open the Captions tab in the export settings and choose burn in, sidecar or embedded
4. Burned in and sidecar are supported with any encoding preset
5. Embedded is limited to pro codecs like MXF OP1a, DNX, and Prores and requires that the caption track format be set to one of the broadcast standards like CEA-608
6. Click Export
7. Sidecar files will be saved next to the video file with the same name
8. Queueing to Media encoder is supported too

Happy captioning!  Once again, we welcome your feedback.  Please leave comments below.

[SpikeSpiegel]

amazing feature! I have a feature request. How do we "Copy " the automatically transcribed text on the text panel? We produce content for channels worldwide and all our clients want a finals cript. The premiere software's automatic transcription is AMAZINGLY accurate, kudos to you guys. BUT, how do we COPY the auto script that is generated? I'm not looking for a SRT/caption file, i'm looking to simply copy paste what the auto transcription generated.. PLEASE ALLOW a copy-ing feature from the text panel or an export to pdf or word for people that need full transcription of their content!! Unless i'm missing another route any insight into this really really appreciate. You haveNO idea how much time you are saving us, (well, maybe you do :P)

[Pierre Louis B.]

BUG: If I change a caption's properties in the EGP (font, size, tracking, alignment, fill, background, etc) and then cut it in two in the Timeline or with the Split Caption (i.e. the poorly named Split Segment) button in the Captions tab, the second half of the caption reverts back to its original settings.  The only settings that are kept are the on/off settings for faux bold, italic, all caps, etc (see illustration).

Is anyone else seeing this bug?

[Wholeo]

My primary need for CC is to upload my videos to YouTube and Vimeo. I just tested the process for YouTube and it worked!

I transcribed and created captions in the Text window, which nicely disabled previous Caption tracks and placed the new one in my sequence. I edited a couple of changes which seemed easy to do. So I chose Export Media and a .mp4 file to export. Under the captions tab I chose Export Options: Create Sidecar File, then choose File Format: Subrip Subtitle Format (.srt). I did not check the "Include SRT styling", although the format in PrP Beta was incorrect. After uploading the video and the SRT file to YouTube and it finished HD and processing the SRT, I tested the video. YouTube had no problem and displayed the CC Sans Serif font, centered at the bottom of the screen. That works for me. Bravo!!

Now I don't know how to proceed with Vimeo. When I checked last summer it said "If you are using a SRT file, please convert it to a WebVTT file". I'll now go check it again.

By the way, where should I use these? I couldn't find the Facebook group for Beta. #SpeechtoText and be sure to tag @AdobeVideo 

[Wholeo]

Vimeo uses the SRT file with no problem. Captions appear instantly, taking no processing time. I'm impressed and so grateful for this feature. 

[bradpaulp]

First: I love that this feature has been added. Both the caption workflow and the speech to text.

Second: I've been using the speech to text for a few days and it is REALLY accurate.Third: There's a lot of feedback around the new captioning workflow:

1) It's clunky to edit the transcript. If you click in the transcript area, you can see it following the sequence as it goes. But then if you double click to edit, it opens up sort of a text editor for that one "speaker".

- I'd love for it to not have to open a text box to edit. Just edit in the same box if that makes sense? It doesn't relaly open a new dialogue box, but it requires opening and closing things for each line.

2) I wish in the graphic styling there was a way to make the background of the text a "full width".

Right now I am having to create an essential graphic layer with a rectangle to place behind the text.

While the caption graphic panel allows for background it only is the width of the text line itself, with some optional padding.

Thank you.

[andymees@aje]

> 2) I wish in the graphic styling there was a way to make the background of the text a "full width".

Right now I am having to create an essential graphic layer with a rectangle to place behind the text.

While the caption graphic panel allows for background it only is the width of the text line itself, with some optional padding.

This... sadly. The lack of a non-adaptive background option is really the only thing stopping me from jumping all in with the new captioning workflow.

[Pierre Louis B.]

Although I've never needed full width caption backgrounds myself, I agree 100% it should be added as a feature for those that need it!

Implementing this should be easy enough to do.  Simply change the single size option into two parameters:

Horizontal size

Vertical size

and add a lock checkbox to lock or unlock both values (just as users can do with in many programs, including the EGP itself when dealing with shapes or regular text)

Doing this would serve the needs of those who need a full width background to appear behind their captions, simply by setting the Horizontal size to a large enough value.  As a bonus, it would ALSO serve the needs of anyone who wants more side padding or more padding above and below their captions.  Please let the users choose how they want things to look by giving them appropriate styling options.

Thank you!

[kla96966041]

Just to be clear, the background option needs to accomodate non-adaptive needs as well as adaptive needs ... so we would probably need a radio button or similar to choose whether the controls are defining a static or dynamic background ie whether the controls are describing the static Horizontal and Vertical background size or whether they are describing the Horizontal and Vertical pad.

[Pierre Louis B.]

The way the background size currently works is as a padding value. 

I understand the reasoning of your request kla96966041, but implementing what you suggest might take more programming resources than warranted, given how unlikely users are to ever want a black background rectangle that remains the same size no matter how many characters are on the screen.

The option to make a black bar stretch across the entire screen, however, is something I see users wanting.  The background size scale would have to go well beyond the current 500.0 limit for this to work however (in the slider the limit is 100, but users can type up to 500.0 by clicking on the blue size value). 

The danger with setting a full width background using an adaptive scale setting is that while a user might indeed get a full width background by correctly adjusting the horizontal size past the minimum required value to fill the screen with a 10 character long caption, for example, the background might mistakenly be less than full width with shorter captions than the one used when setting the width.  This is isn't a big issue though since users could just crank the setting up to its max, 10,000 for example, that would continue being full width even with a 1 character caption.

Or perhaps a checkbox such as "[x] Full Width", right under to Horizontal size setting, would better accomplish full width backgrounds? 

Any other thoughts on how best to achieve this?

[chrisc98685619]

Do any of you all know why I'm not seeing these controls? 

[kla96966041]

Hi Chris

The controls displayed are dependent on which kind of caption track you're working with ... your screenshot shows the controls for a CEA-708 caption track ... the background controls that some of us have been discussing above are with respect to the more generic 'Subtitle' caption track format (which, in my own use case, is what I would be using for producing content for broadcast with burnt in subtitles).

Cheers

Andy

[chrisc98685619]

Thanks, Andy! That did it 🙂

[kla96966041]

>given how unlikely users are to ever want a black background rectangle that remains the same size no matter how many characters are on the screen.

Don't know about you Pierre, but I'm an enterprise user and our network definielty requires this.

For reference, our network used to do the full width subtitle strap thing, but we switched to a smaller (but still fixed size) subtilte strap during a rebrand/gfx refresh a few years ago... and then we switched again, to a different sized (but still fixed) subtitle strap a little over a year ago.

So whilst I certainly appreciate that most users producing content for distribution online and for broadcast may not need the functionality, many of us who produce content for traditional network broadcast have to adhere to very strict 'house rules' ... my point being that (hopefully) the new captioning workflow can work equaly well for everybody.

Meanwhile, even in this tiny online beta forum microcosm of the Adobe user base, we've already had one other user asking for / needing a static full width option, so perhaps thers a bigger demand than might otherwise be imagined, for static vs dynamic subtitle backgrounds, in the broader user base at large.

Fingers crossed we all get what we need. 🙂

Andy

[defaultbce1k5w7k2dw]

Hello! First off, thank you so, so, SO much for the improved captioning, and the Speech to Text features! Going to save me hours of time. I just tried Speech to Text today and it worked amazingly -- and in French too! Only had to make a couple of tiny edits, it was incredible.

I may have missed the answer in one of the other posts here, but one thing I can't seem to fix is that when I want to add a new caption, it defaults to adding a 2 second caption at the playhead and overwrites any captions that might follow. Is there a way to change the default duration of a caption, or to make it so that an added caption only extends from the playhead to the next caption's start point without overwriting anything?

[SpikeSpiegel]

@Francis-Crossman - How do i copy the raw text that is generated for transcription?? THis seems a pertinent feature left out! I want to provide a raw text file exactly as the 'transcript' tab shows after analyzing my full timeline. HOW do I copy this to a text doc? Or export? please advise!!!

[주연5FFD]

How can I make subtitles from transcription generated by speech to text?

Last time, it worked well but something has disappeared I think

[주연5FFD]

[Stan Jones]

주연5FFD, 

Expand the Text Panel to the right. It will appear.

Stan

[주연5FFD]

Thank you so much, Stan jones! I really appreciate it 🙂

[Stan Jones]

I would also like to be able to copy/paste and export the transcription. I found the lack of a copy paste very odd. But this is a service, sold by many different companies, so I wonder if there is a limitation, licensing or otherwise, to what will be allowed.

Stan 

[Pierre Louis B.]

Here's the missing piece to make the new Captions workflow truly AMAZING:

Ability to copy-paste all text from the Transcript tab

I know others have mentioned this already, but no one has suggested how this could actually work in order to illustrate how truly incredible this feature could be, so here it goes...

BENEFITS:

1) Edit interviews in a word processor: First of all, the ability to copy-paste transcripts would allow users to make edit decisions inside a word processor, based on the transcribed text, before ever editing the footage in Pr.  With interviews, what is said is usually much more important than the image, so this text-based edit approach makes sense.  This would also make it easy for clients/others to participate in the editing process, since they would just need to receive a word processing file and then highlight or strike-out the parts they want or don't want included in the final edit!

2) Make corrections in a word processor: Rather than making corrections inside the awkward Transcript tab, users could make corrections/improvements/abreviations in the much more user friendly interface of their favourite word processor!  If Pr could read back in basic formatting options added in a word processor such as italics, bold and underline, that would truly be icing on the cake!  Even better would be the ability to control where individual captions start and end using simple carriage returns, thus giving users full control of how their captions will look before ever pasting the text back into Pr!

3) Translate to another language: Being able to copy-paste full transcriptions would make it so much easier for users to translate it into other languages!  It would allow users to paste the text into Google Translate, for example, to get the bulk of the translation work done, then copy paste it into their favorite word processor for correcting the inevitable mistakes, but with the benefit of spellcheck. 

For these 3 amazing benefits to work, when copying transcripted text from Pr, users should ideally be given 3 options:

1. Copy with Timecode: When pasting text back into Pr, Timecode markings that match those inside the Transcript tab would make it possible for Pr to automatically place the text back into its original formating, complete with Speaker designations, etc (as long as the Timecode text was not altered by the user). 
2. Copy without Timecode: This would provide users with a clean copy of the text for reference, when the user does not plan on pasting it back into Pr.  In this case all text could appear in continuous paragraphs that only break into a new paragraph when there is a change in Speaker.
   
3. Speaker on/off: Showing Speaker labels or not should also be an option. 

What do you think?!

[James Langley]

I agree i'm in desperate need to be able to export or copy out the transcribed text.

It'sd be a godsend to be able to send it to the PR team or whoever is working with the content as it gives them a detailed view of what has been said but also allows them to check grammer and spelling. Then just copy and paste the changes, back in. Easy peasy 

[johankokken]

Why not integrate a translation button, where the user can choose the destination language?

Once the speech to text is done, let Google (?) translate the text to the preferred language. 

[Pierre Louis B.]

I agree that having integrated translation would be even better!  Here's an example of what is possible: Subtitle Pro plug-in https://aescripts.com/subtitle-pro/.  See the 0:15 in the intro video at the top.

I've never used Subtitle Pro, but from their video, it seems their plug-in + Pr's old captioning tools (pre v15.0) might give Pr's new captioning tools a run for their money.  I hope Subtitle Pro get's updated for use with Pr's new captioning tools.  It just might give me all of the translation tools I ever need!

Third party developers are creating amazing plug-ins for Pr.  I'd love to see more of the functionality they offer provided natively inside Pr! 🙂

[Nate McFadden]