UPDATE: January 29, 2021
Since the innitial post back in October of last year we have made a ton of progress and some of the details originally posted here are no longer acurate. I have updated the post to be more accurate to the state of things today.
Here are some important changes to be aware of
Thanks to everyone for testing and for all your awesome feedback! - Francis
The captioning workflow in Premiere Pro is getting a complete makeover and you can be the first to try it out! We have a new text panel for editing and viewing your captions as text. Captions now have their own track on the timeline so they can be edited just like video. Styling uses the power of the Essential Graphics panel so you can make fantastic looking captions.
Best of all – we will be adding speech to text and auto captions powered by Adobe Sensei – coming later.
Some important things to know before you start testing this feature
So, if you’re good with all that, read on . . .
Enabling the Feature:
Download this sample project to follow along PR New Captions Workflow quickstart.zip
Play around with visual styles in the Essential Graphics panel and try editing the text in the Text panel. Read on for more detailed info on the various pieces of the workflow.
The Text Panel
Open the Text panel from the Window menu. This is where you interact with text and it has two tabs – Transcript and Captions
The Caption track - working with captions in the Sequence
We have completely reimagined the way captions work in Premiere Pro. If you have used captions previously in Premiere Pro, this is totally different – much better we think. Please tell us what YOU think by leaving a public comment below.
Create and delete caption tracks
There are a few ways to create a new caption track. You can drag an SRT file onto the Sequence or use the “Create new captions track” button in the Captions tab of the Text panel. To delete a caption track, right-click on the caption track header and choose “delete track”
Drag an SRT file on the sequence
Dragging an SRT caption file onto the Sequence will automatically create a new caption track at the top of the Timeline.
To see all the keyboard shortcuts available for captions, open the keyboard shortcuts menu from the Premiere Pro menu and search for "caption". Here are some notable ones. There are many other which do not have default assignments.
This can hide the caption track area or solo just the active caption track if you have more than one. It is useful if you want to declutter your timeline. It does not turn off the active caption track from rendering in the program monitor, however.
This will enable/disable Program Monitor viewing of the Caption track selected. Only one Caption track can be active at a time, so when you make one track active, the other tracks will automatically be disabled. You can also choose to disable all Caption tracks. Inactive Caption tracks will dim making it easier to note your active Caption track.
By default, this will show the caption track format (eg Subtitles, CEA-708, CEA-608, etc.) You can right-click the Caption track header and choose Rename to choose a custom name (same functionality as A/V tracks)
Additional caption tracks can be added by right-clicking in the caption track header – choose “Add Track”. You can also delete or rename a caption track in the same way.
Caption tracks will always be at the top of the sequence. Additional caption tracks are added on top. This will continue until you have reached 25% of the Timeline vertical space, then the tracks will stop “growing” and scroll bars will appear to the right. Caption tracks will never take up more than 25% of the upper space of the Timeline. This is dynamic, so if you increase or decrease the Timeline panel height, the 25% will adjust accordingly.
Lock the track to prevent editing. This functions the same as A/V tracks.
The Caption track header has a ‘Toggle the track targeting for this track’ button with same functionality as A/V tracks.
The Caption track header has a ‘Toggle Sync Lock’ button with same functionality as A/V tracks.
Editing timing of caption items on the track
You can use all the familiar editing tools that you are used just like editing video and audio.
Styling with the Essential Graphics panel
Open the Essential Graphics panel from the Window menu. This is where you choose things like font, size, color and position. Make sure you have at least one caption selected. This will activate the Edit tab of the Essential Graphics panel. If you have created and stylized text in Premiere Pro before, the following should be very familiar to you. However, for captions we are adding small enhancements such as zones, and vertical text alignment.
Align and Transform
Transcribing the dialog into captions
You have three options for transcribing your audio and creating captions: speech to text, use a third-party service, or do it by hand.
Speech to text
Use a third-party service
If you already have a workflow that uses a third party transcription service, you can bring in that file. SRT is the best option.
Do it by hand
For short sequences, doing it by hand may not too much trouble.
Exporting your sequence with captions
There are three options for exporting captions: burned in, sidecar file, or embedded into the video file.
Happy captioning! Once again, we welcome your feedback. Please leave comments below.
the same thing is happening to me
Hi im using the project file supplied to create a transcript but PP beta keeps on crashing
I know this one won't be an easy one, but, here's my issue when listening to the transcription to make some corrections.
In the transcript panel, the played word is highlighted, I realize there's an edit to make, so I double click on the word :
What I get is this layout :
As you can see, the edit window is narrower than the play window, and that makes all the texts shift. I had double clicked top right, and I need to edit bottom left.
The other thing is that when I double click on a word in a transcription, all the text will be selected, and I'll have to manually place my cursor at the right position in edit mode. On long texts, it's tedious. And when I play again, I'm back at step one, out of the edit window 🙂
I understand there's a need to be whether in play or edit mode, but so far the jumps between each really breaks the workflow. (And I say "break", where I should mention that I'm sooooo happy that a 2 minutes video got transcripted in less than 2 minutes, which never happens when I type 🙂 )
I posted an enquiry a few days ago after having been accepted to the Speech to Text option in the Beta version of Premiere Pro. It worked amazingly for the first project I tested the option on, but have not been able to do anything since.
I've just been reading Francis Crossman's post from the 29th and noticed one of the points at the beginning. Namely that The email you used to apply needs to match the email (Adobe ID) you use to log into creative cloud. If you did not recieve an email stating that you have been accepted, or if you are logged into creative cloud using a different email you will contininue to see the message stating that speech to text is coming soon.
I'm pretty sure that the email I provided when I applied for the Beta testing version was the one that I use to access Adobe Cfreative Cloud, but the email reply I received seems to have come to the email that I use from the University I work for. If this is the reason I cannot test the Speech to text option, is it possible to have this amended, particularly since I was able to work on one project but have not been able to do anything else.
Yours Nick Geddes
Premiere Pro Beta has to have WiFi on to transcribe. But while transcribing, it has to save and it can't do that. I'm stuck.
My first success transcribing to captions doesn't work the second time.
I click "Transcribe Sequence" and it says "Creating Auto Transcription" "Transcribing sequence" "Rendering audio". While this was happening I was looking at Facebook when a "Save Project" popped up saying "Saving Project ...." with a "Cancel" button. This didn't work. It wouldn't save or cancel and Force Quit said the app was "Not Responding". I closed everything and restarted with no other activity and got the same problem. I have had Premiere Pro automatically save and close several times when I'm using WiFi. I don't know why. It never does that when I'm working offline.
I don't know what to do next to proceed.
The transcript/caption feature update is great! what would make it amazing is a button on both the transcript and caption windows that give you a quick option to download the transcript as a text document and an option to download the captions as an SRT side car file without having to render the video as well. The use case here is that I'd like to use this feature to go back and transcribe/caption a bunch of videos that I previously produced before this feature was available so we can be more ADA compliant. I'd like a quicker workflow that doesn't involve re-rendering the videos as well.
Thank you for the feedback. It's definitely clear that the request for options to export a text file and SRT from the Text panel is a common ask 😉
Based on this feedback, work is under way and we hope to have to these options added soon.
Thank you Trent!
The crucially important thing here is to make sure it's done in a way that allows users to both export and import text back in. This would give users the benefit of being able to make corrections, formating changes (bold, italic, underline), translations, and hopefully even define caption breaks and cariage returns for 2+ line captions, all outside of Pr, using any number of word processing and translation programs!
I've just signed up to the program and installed the beta version of Premiere. Transcription worked beautifully, and I've successfully created a subtitle track in my project. The issue I'm having is with the size of the background layer.
If I add a background layer to the subtitle track, I don't want the background box to resize vertically for each subtitle. For example, the subtitle "There is no problem" will have an auto-scaled background with one height. But the subtitle "The y is the problem" will have a taller background, because of the"y". I want the background box to be the same height throughout. It would AMAZING to have this as a setting in Essential Graphics for subtitles. At the moment, I don't feel I can use subtitling in the beta because of this issue. Everything else is great, though!
Yeah. This has been an issue for some time. I have my team use an auto-resize formula in AE. The extra step isn't too bad but BOY WOULD IT BE GREAT to have this right in Premiere. 🙂
I did two things to restart transcribing. 1. I deleted the media cache. 2. I restarted and ran PP Beta alone.
I like the feature to burn in captions to the video. This is very useful when needing to be offline.
Hi all, As I haven't seen it mentioned here I would like to give a big Kudos to the development team for the Incredible performance improvement they have made with exporting captioned broadcast MXF files.
I tested dropping a MXF show and a MCC caption file into PP14/2020 and exported it with smart rendering to MXF and my 24 core usage cpu sits at 3% and it takes 10 minutes to export.
Now i drop the same show and captions into PP15 Beta and export it with 5% cpu usage in 42 seconds! This is essentially as fast as my 16Gbit Fibrechannel SAN can read and write the files.
This is an amazing improvement! For stations like ours that deal exclusively with MXF files for our broadcast servers this is a tremendous help to our workflow.
As for Adobe Sensei, you guys have knocked it out of the park so far. The accuracy and puncuation is outstanding. Even though it is beta I am already looking to transition our captioning workflow over to it.
Most of the editing and functionality issues have already been addressed by others so i don't have much else to add to that yet.
One curiosity though. We are using Blackmagic-Design I/O hardware in all of our edit suites. HD-SDI output to professional monitors/analyzers. Do you think there would ever be a possibility to have premiere output the embedded closed captioning stream through the SDI? That way we could see the monitor do the real decoding of the captions and ensure they are encoded and working properly? We have the ability to see all 608/708 captions and services in our equipment.
Thanks and keep up the great work!
Is it possible to make a 'word' doc. of the transcribed text for a third party to check?
Not directly, but you can export to a text file and then copy/paste into word.
first of all, congrats! I'm so happy seeing an overhaul of captions and reintroducing transcribing audio into Premiere!
My question regarding captions is, is there any way to link the subtitle clip in the timeline to its video/audio clip? So when I move the video/audio clip o na timeline, the subtitles move with it too, without having to select them separately every time I move something?
And regarding transcribing, what is the recommended workflow with translators/editors/directors, who don't have premiere, but who have to correct the transcribed text? We use a website based tool for that in my work, which works great, and without some other easy and quick solution, not sure if we would use it in premiere at all.
When reloading a project that has been transcribed but not had captions created, the panel jumps to the first caption every time a caption is corrected/amended.
In addition, there is no option to add captions where the AI has missed transcribing, instead, you have to take a note of the time code and the transcription that was missed and create a caption manually after generating all the AI transcribed captions.
Is there an ability to import caption files other than SRT? Specifically .ITT or .CAP files? Thanks!
We currently support SCC, MCC, DXFP XML, and STL. (also embedded 608/708/Teletext/OP-47 Australia).
We do not currently support .ITT or .CAP files and they are on our feature request list.
1. Import from text file...: Thank you for adding the new "Export from text file..." options! Now... what about the ability to bring corrected/formatted text back into Premiere? I understand the programing to make this work is more complex than the exporting part, but it would be incredibly useful to be able to correct, format and even translate captions outside of Pr and then be able to import that information back in.
2. Transcription accuracy: It's quite good, and naturally seems to perform better in English.
1 Minute English video with background music =
1 spelling mistake - Pr wrote "It" instead of "I"
1 formatting mistake - Pr wrote
"I never wavered from that every day.
Veterinarians and their teams..."
"I never wavered from that.
Every day veterinarians and their teams..." even though the italicized part is by another speaker (both female but with clearly different voices)
1:50 Minute Brazilian Portuguese video with background music =
7 bigger mistakes I won't write here since I imagine it wouldn't make any sense to most. 😉
3. Small gaps: Converting my 1 minute English video auto transcription to captions resulted in 31 individual captions with randomly placed gaps between them:
12 1-frame gaps
2 3-frame gaps
1 4-frame gap
These tiny gaps are annoying to fix and they don't reflect actual pauses in the dialogue. 15 tiny gaps appearing amidst just 31 captions is a signal that something is wrong. Please fix this so that gaps don't appear unless a pause is a minimum of X frames long. A new Preferences window option that allows users to specify the threshold at which gaps are removed might do the trick: "Remove any gaps between captions that are up to [ ___insert number here___ ] frames"
4. Large gaps: Converting my 1:50 min Portuguese video auto transcription to captions resulted in 33 captions, with a couple of larger gaps due to captions ending before the speaker actually finishes speaking:
1 29-frame gap
1 39-frame gap
2 6-frame gaps
1 7-frame gap
4. Text display issues in Create Transcript window: When selecting a language in the Create Transcript window, 5 of the options don't display correctly, with one of the language name's characters appearing cut off or improperly drawn. On my system, drawing issues occur with: English (English UK works fine), Hindi, Russian, Italian and Japanese. Oddly, these issues only appear when selecting these languages. The drop down list of options displays all of them perfectly.
5. Portuguese with a 'u': In the Create Transcription window, Portuguese is incorrectly written without a second 'u'.
6. Merge captions button no longer working: The button no longer works no matter what I try. Nothing happens when I click on it, no matter which captions I have selected.
7. Delete text block now works logically: Thank you for changing the way the 'Delete text block' context menu option works so that it makes more sense, completely deleting the caption from the Text window and the Timeline whenever the caption only has 1 text block.
8. Unintuitive access to EGP text formatting settings: Accessing caption text formatting options in the EGP is very unintuitive since the settings only appear if the user selects a caption's text in the Program Monitor. Selecting text in the Captions tab doesn't show any formatting options in the EGP. This is illogical and unintuitive and should be fixed.
9. Shift-select bug fixed: Thanks for fixing the bug that didn't allow shift-selecting a range of captions in the Captions tab whenever the first caption at the top was selected.
10. Button/tooltip renaming: Thank you for renaming the following buttons as I suggested.
This will make it much more understandable to users! While the function of the Add new captions segment is perfectly clear, it's unnecessarily verbose, which makes it more inconvenient for users to reference. It also will bloat to any help files that refer to it with extra, unnecessary words. That's why I continue to suggest the much more concise yet equally (if not more) understandable/relevant name of New Caption.
That's it for now. Hope this new round of feedback helps!
As always, thanks for the excellent feedback.
This most probably has been asked before, but is there a way to tell how many lines of subtitles the transcribing produces? It seems to only output one line at a time and I can't seem to find a way to change this to two lines. Also, I can't seem to merge any two subtitles either.
HOLY HAND TOOL!!!
The Text window finally allows us to do what we've been able to do in After Effects for ages: click-hold a mouse's scroll wheel to temporarily invoke the Hand Tool. Hurray!!!!
This basic functionality ONLY works in the Text window! What???
Francis, are you and the Pr team planning on adding this functionality to other areas of Pr where it makes perfect sense to have it? (i.e. Timeline, Source/Program Monitors, Project window, etc.)
I posted an official FR for this here: https://adobe-video.uservoice.com/forums/911233-premiere-pro/suggestions/42781670-give-the-mouse-scr...
...along with these charts to illustrate the large discrepancy in scroll wheel functionality between Pr and AE.
Although this post veers off the topic of Captioning in Pr (my apologies), I felt it's worth pointing out the fact that the Text window finally provides a feature people have been requesting in other parts of Pr for ages. It would truly be a shame if the Pr team left it at that and failed to provide this basic and much requested feature in all other areas that make sense (and where, quite frankly, it's needed much more than in the Text window)
small bug reports/feature requests :
Pr Pro Captions Beta - transcript ON A vs ON N'A
this one is tricky in french, there's a spelling link between N and a vowel
On a fait quelque chose (we did something)
On n'a PAS fait quelque chose (we didn't do something)
One is positive, the other is negative ("PAS" = not), and many make this spelling mistake. good luck on training your model 🙂
Pr Pro Captions Beta - descending letters
Annoying one I noticed, if you've got descending letters (pjqy…) the box will be calculated from the bottom of that tail.
On the bottom example, red, bad, I had no descending, which led to a few empty pixels under the box.