Remove Unwanted Voices in Audio Recording

I have an audio recording that has multiple people talking. However, one person (male) liked to talk over everyone else. I want to remove his voice so I can use the other people's voices (all females). Is there a way to do this in Audition or am I out of luck?

The file is part of multi-track audio in a video file (mp4), all the voices in question are on 1 track.