Burn Subtitles to video doesn't work reliable.

I have turned on 'Burn subtitles into video' in the export settings, but it doesn't work. Flicking it on and of sometimes does the trick, and it's not clear why. Checked it multiple times, also updated everything. Seems like a bug, and a pretty annoying one if you have numerous amounts of assets in different formats.