The AI Assistant is awesome, but hindered by transcript timestamp/alignment accuracy
This isn’t strictly a bug, but is undesired behavior.
The Good:
I’ll first say, I’ve been using the AI Assistant and it’s genuinely amazing for my use case. I’ve been giving it a prompt similar to the one below, and it is literally able to take an hour of raw talking-head footage, identify which parts should be cut and which should be kept, and it gets like 99% of the way there, compare to what I would have done.
Make a cut of this A-roll in the sequence. Basically remove repeated phrases, false starts, and dead air around them, partial phrase repeats, and just keep the last version stated. Some parts I may have only re-stated part of a sentence, so only that repeated part needs to be removed. If a later take ends up being incomplete, use the last complete one. Jump cuts are fine.
For any parts that you feel you need to flag something, you can use range markers and put the note in the description/comment of it. Do this only when necessary though.
After completing the cut, trim any pauses/silence that are at the start or end of a clip down to about 4 frames (~0.067s).
The Problem:
That being said, the default transcript, specifically the accuracy of the word-level timestamps, really seems to hold it back. Even if the AI accurately identifies which parts should be cut, when it makes that cut, it often cuts well into the start of the word, or too long before it so there’s a big pause between cuts. Same for the end of words at the end of a cut. The transcript also occasionally seems to make the duration of the word last during a huge silence gap until the next word, which further messes things up even if the AI cuts correctly based on the transcript. This means I’d have to spend considerable time adjusting every single cut.
I know this is a general limitation of most AI transcription models, none of them are dead-on accurate. I’ve been able to work around this by creating a whole custom workflow where I use a dedicated “forced alignment” tool to re-align the transcripts (Montreal Forced Aligner). And with word-level timestamps accurate to within 10ms or so (usually 1-2 frames worth), the assistant really shines, and the edit it produces is pretty darn usable and requires much less adjusting.
Suggestion:
I’d suggest (and request) that for whatever transcription model Premiere uses, consider trying to really nail down the alignment accuracy as much as possible. Or even potentially run the text against a dedicated aligner tool afterwards.

