Now in beta: Enhance Speech in Premiere Pro

[David Polk]

With AI-powered Enhance Speech now in Premiere Pro (beta), you can magically remove noise and improve the quality of dialogue with one click, so it sounds as if it were recorded in a professional studio! We need your feedback to optimize this feature further through adding a comment below and completing a short survey.

Here’s how to use it:

 Open the Essential Sound panel and select a clip containing speech in your sequence. Click on “Dialogue."

 

Then, just click the “Enhance” button. When processing is complete, you can adjust the amount of enhancement using the Mix Amount slider. This slider allows you to mix your raw audio with the enhanced audio, allowing natural background noise to come through.

What kind of recordings work well with Enhance Speech?

 Audio that is recorded with a low-quality microphone (phones, online meetings, on-camera mics) tend to be improved with Enhance Speech. It does not always work well with:

• Extremely very poor-quality audio (e.g. a recording from an old VHS tape with heavy artifacting)
• Multiple overlapping speakers
• Whispers
• Loud background music
• Singing

How does it work?

 The version of Enhance Speech within Premiere Pro (beta) is the same feature that has been available online via Adobe Podcast. However, Enhance Speech within Premiere Pro (beta) has been optimized to perform locally on your machine, allowing you to run Enhance Speech on an unlimited number of clips. Unlike Adobe Podcast’s service, no internet connection is required. The model is not trained on your usage.

As indicated in the name, this won’t work with music, sound effects, or ambient audio. And like most AI-powered features, it requires a performant system; processing times may vary. (See Premiere Pro’s recommended hardware specifications.)

Please check out our full FAQs for this feature for more details.

What we’d love to know from you:

AI models are evolving fast and voice recordings have many unique variables we can’t always predict in our testing. We need your feedback to learn more about how you’re using Enhance Speech so we can further optimize it for all types of voices.

How has this been helpful? Have you run into any enhancement issues that prevent you from using the feature? If you’ve run into performance issues, please tell us what kind of computer you’re using.

Complete this survey and leave us feedback below.

[jacobc37011316]

Started testing this feature today on some audio shot with a boom mic in an echoing room. The feature really made the audio so much better, getting rid of background noise as well as the reverb. However, it did a strange thing. My subject said the word "supportive," but the Enchance Speech feature changed the word. Now it sounds like my subject said "supporter". I couldn't figure out how to fix it so that it matched the other enhanced speech, but still used the correct word.

Additionally the speech enhance sometimes left strange noise artifacts in places where it was quiet. 

Overall I'm super impressed with the feature and can't wait to use it more. It really can save some bad audio.

[Amitzi Photography]

>From my experience, this happens when there are issues with the original
audio, like a noise that goes over speech. The only option is to use the
original audio of the botched word and clean it up with some other plugin.

[Bill Sprague]

I use "prosumer gear" including Rode Wireless Go II mics, a small Rode VideoMicro and sometimes even the camera mics.  I never have a controlled studio.  This new feature is a hugely important tool for me.  Thank you.  

[Amitzi Photography]

Thanks! I've been using it for a while and I'm glad it's now part of Premiere Pro. I have to say that the mix amount slider doesn't really work for me. I would recommend for anyone using the Enhance feature to typically not do ANY mixing and record ambience separately. Let's say you're doing an interview at a busy festival. I still would use the best mic you have (like a Rode Go), ask the interviewee to not shout, but talk in a normal loud voice and then add in post some background noise.

[Jason FB]

how do I "assign a tag to enable editing options based on the tag"?

[Stan Jones]

The tag is "dialogue" or "music" etc. Select a clip and assign it to that category.

Edit: Adding this since I'm back home. Here's the help page:

https://helpx.adobe.com/premiere-pro/using/premiere-essential-sound-panel.html

You must select the clip in the timeline. You'll see which clip in the "selection" line. In your screenshot,  you have nothing selected.

Stan

[Michael Muench]

I love the feature and I can use it for many different occasions. However, I don't fully understand the process yet. After activating "enhance" it renders the audio. But as soon as I make another cut to the clip it has to re render everything. That makes it quite unusable before having a picture lock. Is this correct or is something messed up in my set up?

Michael 

[Alejandr01899]

I'm experiencing the same problem, there should be a way to "flatten" or lock the enhancement so that every time a new cut  is made it doesn't re analyze the clip again. It's really annoying and truly a waste of time to do it before having a picture lock, yet again in my opinion, sound corrections are normally done before proceding with the rest of the edit.

[Stan Jones]

@Michael Muench @Alejandr01899,

> as soon as I make another cut to the clip it has to re render everything. That makes it quite unusable before having a picture lock. 

Yes, that is frustrating. And this must be used, in some situations at least, early in the process, because it is expected to help with accurate transcription. I suspect that its logic is applied in a way that changes the results if you use smaller sections. So that would force the need for a new analysis.

The re-render only happens when you initiate playback. But it doesn't show a new analysis process, just that it is "rendering audio files." So I wonder if it is not analyzing, just creating the cache files for audio playback.

In any event, I tried 2 things. I de-selected the Enhance button. Sometimes it did not re-render, other times it did.

I unchecked the effect. I could not get the audio back to the original state. But it did not re-render anything after cuts.

Stan

[Samuel31228286t6dg]

Yep, having this issue too. It makes the tool unusable unless you wait to render the audio after you've done literally everything else, which is not ideal for obvious reasons.

[neumannfx]

Once you have the Enhanced settings where you want, select the audio clips you want to stay that way and (right click) Render and Replace...

This works to your benefit in two ways. First, it allows you to send the clip to Audition with the Enhance effects on it and secondly, you can always select the clip(s) and use Restore Unrendered get back where you started if you later decide it wasn't quite right.

[gianniw27226249]

In my case nothing happens the progress percentage does not increase🤷‍:male_sign:i gavr up after 15 minutes of waiting. Seems like a bug

[Samuel31228286t6dg]

This happened to me, it turns out the download was queued behind other CC updates.

[A. S.]

A cool feature but the artifacting on any noisy audio is wild. When speech drops out, the AI picks up wind, clunks, bird chirps and anything else and turns it into gibberish robot voice. There is no easy way to eliminate these without dipping out the audio altogether, or cutting a clip into segments of enhanced/not enhanced. More fine tuned controls over bringing the AI in and out of a given clip would help.

[Amitzi Photography]

I typically use other filters that I have to clean the audio (like Waves)
before putting it in Enhance, for the best results.

[20606435732620903sygx]

I seem to have updated my premiere pro 23 to latest version but don't see the feature. It has "enhance speech" then high or low tone but no analisys bar etc. Am i missing the feature and can you please help me get it? I did it through the beta website and it worked amazingly well!

[Stan Jones]

This is in the public Beta.

The Beta can be installed independently from your production version(s). You do so from the regular CC Desktop.

Note that the current Beta (24.1.0 Build xx) is at PR project file version 42, while the release PR 2023.x is project file version 41. This means that you cannot open a Beta version file in the release version.

See the Beta FAQs:
https://helpx.adobe.com/x-productkb/global/creative-cloud-beta.html

Stan

[PBMMark]

I find it works quite well on noise and making a shot sound closer. However, I have not ended up using it in any of my final renders because I find it nearly always makes the speaker sound like they have a lisp (turning all S sounds to an SH or TH sound), especially female voices with more prominent sibilance. Keeping the mix low in order to maintain some sibilance means leaving the noise and its associated artifacting higher than I'd like. So I've kept my habit of just sending to Audition and making the edits manually. If the lisp issue were resolved, I'd use this feature in a heartbeat. 

[jtstewartmedia]

I have an issue when trying to export my timeline. I have two audio tracks, one of them is voice over with enhanced speech and the 2nd track is music that I didn't use enhance audio on.. when I go to export the result is the unrendered version of my enhanced voice over. I have used enhance speech on a project where I had 20 separate interviews, and those all exported properly. (I did also experience the strange artifacting during either mumbling or a pause. sounds like a bot). 

I believe I saw / heard somewhere that the beta version doesn't support multiple audio tracks when using enhance speech (which seems weird since I'm only using it on one track). Does anyone have a work around that will let me export my project with enhanced speech and a music track? I originally thought about nesting the tracks together but can't seem to figure that part out. I can nest the tracks individually but not together into one track. 

[jtstewartmedia]

Okay, I finally (after a week of trying different things) figured out a work around. I duplicated the sequence, got rid of everything except for the enchanced speech track, and exported only audio as .wav uncompressed, re-imported into original sequence (also duplicated as to not mess with the original sequence), and after exporting the audio sounded like it did inside premiere. 

I'm not sure if the issue was due to having so many clips on one track with enhanced audio, but basically took an interview and chopped it up and moved the clips around out of order so I had to enhance all the clips separately since I originally edited all this outside of premiere pro beta. 

Hope this helps solve any issues for other users. 

[MisterAdvent]

I don't have time to try this, but how about render and replace, or edit in Audition (which does the render/replace cycle)?

[lopis1]

I used Enhance Speech now, today, online at Adobe Podcasts, the free version, after some months. It is an improvement over some months ago, but the problem remains: I used it with audio in the romanian language, very noisy, and again the voice is sometimes distorted, when is sounds like a synthesized speech, this occurs for a few continuous words every around 2 minutes, the artificials words created are unintelligible. It is an improvement, because months ago up to half of speech was a synthesized speech with a mix of english and romanian words. I understand that the machine learning, the training, for Enhance Speech is made using english speech. I don't know why some romanian words are replaced with english words or with a mix of english-romanian word, all with a synthesized/robot voice, instead to leave the speech alone, and just to reduce what is not recognised as speech. In other software, Voice Isolation works better, without to change the speech, without to synthesize the voice, without to introduce english words and accents. 

[LadyluckVA]

I used Enhance Speech (beta for Premiere for my problem) .... the voice is sometimes distorted, when is sounds like synthesized speech, .... artificial words created are unintelligible. 

By @lopis1

Same issue-- audio overall much improved but synthesized-ish says a one word over the top of the speaker, usually a word said earlier but clipped. It pains me to not Enhanced Speech because overall it sounds much improved but I don't know how to resolve the weird glitches. ~Rookie User

[Highdeny]

Same for me, i tried it with english and it was mindblowing good. Now i needed it for a german voice...and its totally useless. sounds like a robot/alien who tries to make german language sounds like english, but with phantasy words. Adobe should not make a big fuzz about it in the german "whats new" section as long as it works only for english. i´m really unsatisfied at the moment.