Illustrator is pasting swatches from other documents which are not used

Hi, 

I have been experiencing this issue for a very long time. When I need to copy/paste something from another document (might be a simple one-color object), many times illustrator pastes a lot of other swatches with it. They are not used anywhere, I can delete them with select all unused swatches - delete. But it is very annoying because I work with many different artwork files (for example 50 at once) which need to be swatch-perfect and I need to check a swatches panel all the time if there is not some unwanted swatch added randomly... 

Is there any solution? Thank you