Why does the color shift when making a selection with color range?

I have some color drawings that have been provided to me by a client that were done on white paper.  However, they had to photograph  the drawings using ambient light in conference room so the lighting is uneven.  I decided to use color range, picking the white background and using fuzziness to capture all the paper that might be slightly grey.  When I invert the selection and then try to work with just the colors, adding a new pure white background below that layer, the colors are altered, and the contrast is off as well.  I've having to use hue/saturation adjustments to fix each of the colors, which is rather time consuming Any suggestions of how to make this work better?  I'm working in PS 2022 on a MacBook Pro.