Thanks for the detailed answers! Those were a huge help.
I'd venture the issue is the Roto Brush, although these render times are still quite shocking.
First, are you using the latest model of Roto Brush? If you're just applying it in a new project, then you should be. If for some reason you're not, definitely use the latest one. It's better in every way, including performance.
Second, are you freezing your propagation after you're satisfied with your roto? I have a feeling that you might not be, and this could be the culprit. If you're not familiar with the workflow, go to section 9 here.
Lastly, you might have some settings cranked up so high that they're tanking performance. Refine Edge can definitely slow things down, and higher than necessary settings mixed with 4K clips can definitely be slooooow.
... View more