Can I assign a matte to a specific color?

Here's one for you guys.

Based on the title, I know what youre thinking. Just use a key, but here's my concern.

I want to use a solid color to apply it to video footage underneath it. But I dont want it to affect certain colors (in this case, skin tones)

In one case, I could use "leave color" , but as soon as I apply the solid color to the footage beneath, that wouldn't matter anymore. I thought about maybe applying a mask using secondary color correction, leaving on show mask, and using THAT as a matte, but I am not in front of my computer to test if Premiere even allows you to export that without removing that option ("show mask")

Is this possible? I'd be impressed if someone comes up with this. I feel as if I am well versed in Premiere, and maybe I am exhausting my brain, but I cant determine a good way to do this (and yes, before you ask, its just for aesthetic purposes)