Premiere Pro Captions put all words in lowercase

Yes, to remove all punctuation, simply go to the Text panel .Then in the search bar, type in "." or "," or whatever other punctuation mark you want to remove. Then click on "Replace" and then "Replace all".  Make sure that your "Replace with:" bar is empty or else Premiere will replace what you're trying to remove with the new entry. I hope this makes sense and improves your workflow!