proxy media and full resolution media must have matching audio channels

I feel like you should be able to attach proxies to full resolution media based on video and not just audio - I have hours worth of proxies that a DIT made and the audio channels don't match the source files so I can't connect them.
I can do this the traditional way, working with proxies, until picture lock, unlinking and then re-linking to full resolution media, however I'd prefer to have both on hand in Premiere.