How to create a rough edge like this in Ps?

You already have great and correct answer, hovewer I will add my two cents: depend on situation I am usually using layer mask and Filter > Filter Gallery to apply some of the filters to layer mask. The reason is because I can remove/modify mask and create new one easily while the original lettering/artwork/text etc. will remain untouched. Another option is to convert text/artwork to smart object layer then to apply filter as smart filter which can be modified and changed at any time.