Text Panel Enhancements (Part 2): Assemble Rough Cuts from Transcripts

Report · Aug 02, 2022

As you saw in my earlier post about text panel enhancements, transcripts are becoming more central in editing workflows. Starting today in Premiere Pro (Beta) you can create rough cut edits from selected Text passages.

To start, you can generate transcripts of your footage in the background when you import your media. Alternatively, you can generate transcripts from the Text panel after your project has been created. Either way, the Sequence transcript panel dynamically syncs with your Timeline: as you make changes in your Sequence transcript, the Timeline will reflect your changes – and vice versa.

My colleagues and I created some starter footage for you, as well as a tutorial so you can try it out. If you tend to do all of your editing in a Sequence, follow the guidelines below and/or watch this tutorial. If you tend to do three-point (Source) editing, follow the guidelines below and/or watch this tutorial.

Sequence Editing Workflow

Download the four videos from the Five Questions video assets folder
Open Premiere Pro (Beta -23.0x006 or higher) and create a new project
In the Import screen

Import "Kerstin interview"
Turn on Automatic Transcription and make sure “Create new sequence” is checked. (Note: You can turn on and off automatic transcription in Preferences -> Transcription -> Automatically transcribe clips on import.)
Make sure language is set to English
Click "Create"

In Edit view, open the Progress Panel (Window -> Progress). This will allow you to see transcription progress.
The clip should be in your new sequence
Click on the Transcript tab in the Text Panel
Click “Transcribe sources.” This setting allows your Timeline and Transcript to stay in sync.
Use the Search box in the Transcript panel for the word "what." This will allow you to see where the questions are in the video
For the first question, highlight everything from "I had a piece of that cake a while ago" until she says "Second question."
Click the In/Out point at the top of the Transcript panel. You should see that section of the video highlighted with In/Out points in the Program Monitor
Delete the highlighted section using the "Extract" button in the Program Monitor.
Go through the rest of the transcript and cut it down to a simple Q&A highlights reel. Have Kerstin state the question and a simple answer, but delete extraneous information. For example, have her say her favorite karaoke song but exclude detail on what makes it a great karaoke song.

If you wish to have Premiere automatically set In/Out points around any selected text, go to the dropdown menu in the text panel and select "Set In/Out around selections."

Play back your sequence. You should have one sequence with five questions and simple answers.
Play back your clip and check for any issues.
Try it with your own projects!

Three Point (Source) Editing Workflow

Download the four interview videos from the Five Questions video assets folder
Open Premiere Pro (Beta - 23.0x006 or higher) and import the four videos to your Desktop or other preferred folder
Create a new project
In the Import screen

Check the boxes next to all four videos
Turn on Automatic transcription (Note: You can turn on and off automatic transcription in Preferences -> Transcription -> Automatically transcribe clips on import.)
Make sure “Create new sequence” is unchecked
Make sure language is set to English
Click "Create"

In Edit view, open the Progress Panel (Window -> Progress). This will allow you to see transcription progress.

Pro Tip: Set up your windows so that Text Panel, Source Monitor, and Program Monitor are side by side.

Create a new sequence (File-> New -> Sequence). Name the sequence "Karaoke"
Double-click on any clip in the Properties Panel
Click on the Transcript tab in the Text Panel
Click on the “Source” tab to keep the source transcript in view
Search the transcript for the word “karaoke.”
Select the question and answer and click the In/Out point at the top of the Transcript panel. You should see that section of the video highlighted in the Source Monitor
Use the Insert button in the Source Monitor (or comma (,) keyboard shortcut) to insert the selection into the timeline.
Do the same thing for the four remaining source clips

If you wish to have Premiere automatically set In/Out points around any selected text, go to the dropdown menu in the text panel and select "Set In/Out around selections."

Play back your sequence. You should have one sequence with five people answering the same question.
Switch to Sequence view in the Transcript tab
Click “Transcribe sources.” This setting allows your Timeline and Transcript to stay in sync.
If there are any extraneous words in your sequence, highlight those words and click the In/Out point at the top of the Transcript panel. You should see in/out points selected in your Timeline panel.
Delete the words and associated subclip with the Extract (apostrophe (‘) keyboard shortcut in the Program Monitor.
Play back your clip and check for any issues.
Try it with your own projects!

Stay tuned, because we have more coming this year

Document style text editing: Use Cmd+X/Cmd+V and Delete to edit your tracks from the Text panel.
Range selection: Select a sub-clip on a single track using text. We know that In/Out points + ripple delete affects all tracks in your sequence and deletion isn’t helpful if you have B-roll, music, etc.
Language detection: Set your language preferences in Preferences -> Transcription for now. The ability to detect language on import will reduce a click, and can be helpful when you are working on a project that includes multiple languages.
Search Source footage for key moments

We would love to hear from you

How is the performance of automatic transcription when you have hours of files? Please share your system specs + the number of hours of source footage you are transcribing.
When working with your own footage, are you mostly editing the Sequence Transcript, Source Transcript, or a mix of both?
Are there any moments where the UI or descriptions feel unintuitive?
How well do the timing of the cuts work?
For non-English languages: Are there any languages or dialect problems, e.g. problems selecting text, editing word boundaries, etc?

Report · Aug 28, 2022

This looks fantastic! I'll report back after testing it out.

Report · Sep 05, 2022

Hi Joseph,

can you please walk me through the steps you followed when running into this bug? Were you editing text in a source clip transcript or in a sequence transcript? In which beta build are you seeing this?

Thanks,

Kerstin

Report · Sep 07, 2022

Hi, Joseph,

I'm sure you're most wondering if @Kerstin Ebert got a look at it!

I've been out of town and unable to experiment further. I do see the problem in your video, but I cannot replicate it on two different machines. Since you indicated it has been present for some time, I tested in 22.5, not the Beta. Both machines are Win 10, however. I created some text layers and edited text - with regular and then aligned right.

I have seen similar glitchs occassionally, but it is momentary, immediately corrects, and does not freeze. I think the last I saw it, I was changing scaling and position on 4K GoPro clips. It was not with text.

Stan

Report · Sep 20, 2022

Will I be able to use the transcript in other application like Prelude?

Our journalists are still using Prelude for watching and organising footage. Or even to do transcription manually. So is the transcipt part of the metadata of a clip?

Report · Jan 19, 2023

@detlefs23288115

I just noticed that no one replied to you.

Prelude is discontinued. (Unfortunately, I'd say. It was really great having something slanted toward reviewing, logging, and transcribing footage without everything else we find in Premiere Pro.)

It's looking like rather than manually transcribe, we'll do Automatic Transcription and then correct the resulting transcript in the Transcript tab of the Text panel.

Report · Oct 05, 2022

@TeresaDemel I finally started playing with this, but have not tested enough to understand the issues.

Here are two threads in the regular forum that you may want to look at regarding users' workflow interests. Does the new workflow work if you use the source-sequences in multicam? Can a transcript-as-srt, designed for captions, be modified as needed for format and imported as a transcript and used with this workflow? Sorry I have not been able to experiment with this more.

https://community.adobe.com/t5/premiere-pro-discussions/trying-to-edit-multicam-clips-with-subtitles...

https://community.adobe.com/t5/premiere-pro-discussions/srt-s-not-editing-from-a-multicam-sequence-i...

Stan

Report · Oct 11, 2022

This really is a great feature! When you say "Search Source footage for key moments" is coming later this year, is this a global search function? It would be very useful to be able to search across all your source footage.

Report · Oct 11, 2022

Also, the new "Whisper" open source ASR from OpenAI gives me significantly better results in Norwegian than the model Adobe uses. Any plans to implement Whisper in your products?

Thanks!

Report · Nov 03, 2022

Hey - first, I just wanted to say thanks SO much for developing this.

As far as feedback, indeed it feels a bit clunky but I assume these are growing pains.

- It should be possible to select text through breaks in the transcript.

- I hope this doesn't slow down editing?

- Really just take a look at Descript's interface, they've been doing this for a while

- For example, um/ah detection/removal, coughs, breaks, spaces

Thanks again!

Report · Nov 03, 2022

What really needs to happen is for Premiere to be able to upload a standard txt file of the text, and then put it into a transcript that syncs with the audio. This is needed if the speaker is using a teleprompter, and the text file with all the spoken words already exists and the speaker is simply reading it. To use the auto-transcription method produces a huge number of errors that have to be corrected manually, and that is the most time consuming part of the transcription process. But if one already has a text file with all the words in it, you shouldn't need to use auto-transcription in the first place. YouTube allows you to do this with uploads. You just upload or cut and paste the raw text as one big block, and it slices and dices and sorts out the time codes. Anyone using a teleprompter needs this capability in Premiere.

Report · Dec 13, 2022

Agreed. I don't like to have to fix captions when I already have a script I read from.

Report · Nov 20, 2022

Hi there. I am trying this workflow with clips that have audio and video recorded separately. If I import the source audio and video with automatic transcription turned on, then merge clip is greyed out. And there's no way to merge the clips unless I create a new project with automatic transcription turned off. If I do that (turn off automatic transcription) and import the clips, I can merge them. But then if I open the text window and hit transcribe source, I get a progress bar in the text panel, but it never turns up in the progress window. And it never completes.

The only work around I've found is to EXPORT the merged clip as a quicktime, and then re-import it in a new project with automatic transcription on. Inefficient, but a workaround.

So I guess what I'm understanding is, this feature ONLY works when audio is recorded in the same file as video? And negates the workflow Premiere has implemented for people using DSLRs and recording audio and video separately, then using merge clip to combine them in post?

Or did I miss something here???

It's a great feature. But I am teaching a course where all our students use DSLRs and send audio to Tascam recorders. I was planning to teach this new feature as a way for them to simplify their workflow, but suddenly it's made their workflow more complicated.

Report · Nov 22, 2022

Hello,

thank you for reaching out and reporting this problem! This is very valuable feedback and we are working on a fix for this so that you will be able to work with source transcripts for merged clips!

Best,

Kerstin

Report · Dec 13, 2022

I keep getting "No Dialog Found" when I try to transcribe for editing.

It seems to be intermittent and I cannot narrow down the cause. Sometimes it works and sometimes not, Even on the same project. I have tried various projects.

I have already tired re-installing and the issue occurs on both the current Beta and the previous one.

Report · Dec 13, 2022

Maybe you have to tag the dialogue as audio using the Essential Sound tool?

were your audio and video recoreded separately and then you merged them in Premiere?

I've also had problems on systems without admin privelages that prevent downloading the language packs. Apparently even English does not come pre-installed and requires admin access to the OS.

Report · Dec 13, 2022

I appreciate the suggestions. Tagging did not help.

The audio was captured with the video track, so no merging.

And the language pack seems to have downloaded fine. I forgot to menton that regular Transcripting and Captioning works, just not the Transcription Mode where you can make live edits.

It feels like it's an issue with the Transcripting tool detecting the audio for multiple clips of an audio track in a composition.

Report · Dec 13, 2022

Aaron,

Yes, the test editing mode has no options to set for where you get the audio, as the sequence transcription does. I don't know how it works, but I think it is working on only one clip. No multiple tracks (unless they're in the file with the clip?).

Stan

Report · Dec 13, 2022

Aaron,

As discussed above, if you try to transcribe for "Text-based editing," and you are using Merged clips or clips that are video only (separate audio), it will fail (and shows the "No dialogue found" message).

I think it works if you are transcribing for captioning.

Are you using merged or audio separate clips?

@TeresaDemel @Kerstin Ebert Do we have an update on fixing this?

Stan

Report · Dec 13, 2022

That's it. Thank you, Stan.

While I recorded the video and audio together, during the edit I separated them so they are no longer linked. I'm sorry I didn't recognize this has already been addressed in the thread.

Looking forward to this bug being fixed because this is a fantastic new feature!

Report · Dec 13, 2022

It's a long thread....!

A possible workaround to test is using the original to create a transcript, then export/import it to the other source.

Stan

Report · Dec 14, 2022

Hi Aaron!

I wanna investigate a little more here because it sounds like this is happening randomly.

Just to double check, you have multiple a/v-clips on your timeline, and during your editing process you unlinked video from audio, correct? Now when you have your timeline in focus and look at the Transcript tab of the Text panel, it sometimes shows you "No dialogue found"?

Have you tried transcribing the source footage direcly (either from the Project panel via right click -> context menu -> "Transcribe...", or by double clicking on the source clip and transcribing it via the Transcript tab (in source view). When you add a source clip that you already transcribed to your sequence, you can enable "text based editing" mode for your sequence, this should reveal the transcripts for all your edited clips in the timeline. Does this work for you or are you still running into the "no dialogue found" state?

Thanks,

Kerstin

Report · Dec 15, 2022

I think it was patially me not undrstanding the workflow. I am now processing the transcript for the video file rather than trying to do it from a sequence within a sequence.

After running the transcription, I open the first sequence and can edit via transcript just fine. Once the edits are complete, I copy and past that into the second sequence. It's at that point that the transcription for editing will no longer recognize the dialog.

Ideally, I'd like to be able to edit via the transcript, even in a sequence within a sequence.

Report · Dec 23, 2022

Hi Kerstin, I have the same issue. My situation. I have two video track. It's an interview, so one video with audio for me and another video with audio for the interviewee. When I create a sequence with both of them, I cannot create transcription for "text-based editing", I do not want to go with the caption option because 1) I can't edit the raw transcript (I can, but it won't sync with the timeline) and 2) I can edit with the final caption, but it loses the speaker information, which is helpful for my editing.

Report · Jan 03, 2023

Is it possible that you send me a screen recording of the whole scenario (adding clips to the timeline, running into the "no dialogue found" state)? I'm trying to reproduce this on my end as well, but so far it's working correctly for me.

You can share the screen recording here or send me a message.

Thanks!

Report · Dec 14, 2022

Loving this feature. I wish there was a way to batch edit the speakers from a sequence after auto transcribe is enabled. In other words, I have a cut, the source clips were batch encoded, now I'm in the sequence transcript and the speakers names aren't editable. I have to do a match frame to each source clip to edit the speaker name there. There should be some way to edit them in the sequence view and have it effect the source clips.