Issue: I'd like to remove extraneous (unused) colors.
The added colors are still part of the color definitions: View > Color > Definitions.
Is a MIF wash the wrong approach?
"MIF wash" is a completely wrong approach, because saving to MIF is not supposed to remove ANY user defined items from a file. In contrary, the sh#tstorm would be huge, if elements were going lost after saving to MIF. The so-called "MIF wash" is only (sometimes) good for removing unwanted defects or technical file errors, but not to clean up a man-made mess
There are script based solutions for automatically removing unused styles, formats, or colours.
MIF wash, per se, removes exactly zero [properly formed] entries from any document Library, including Color. Unused entries in particular are not removed. MIF wash might remove malformed entries, but that isn't your problem.
Also, if you delete unused colors by hand or with a script or aftermarket tool, and they come back, that means they are being auto-created by some imported content. I found DXF and DWG images to be vexatious in this regard.
If you open the MIF file in a text editor, you'll find you can rapidly delete one or more color definitions.
Save the changes to the MIF file, reopen in Fm, and you're good to go.
Unless, as Bob noted, you have graphics that are adding colors to your doc. These colors are often ambiguous RGB color definitions. To permanently eliminate these colors you'd also need to replace the offending images with formats (like PNG?) that don't stuff your doc with extra colors.