How to disable CMYK colour in the document in illustrator sdk using

I'm not sure what you want to do is possible. The user can switched colour models whenever they want; there's no way to disable CMYK. Best you could do would be to notice when they've picked CMYK and inform the user your plugin doesn't work with CMYK.

Removing CMYK colours from the document would just mean iterating the swatch list and deleting any colour that is a CMYK colour. Look in AISwatchList.h for functions to do that. Nothing stops the user from creating a CMYK colour manually though.

AIColor.h has functions for iterating all the colours in the document, if you need to insure that no art has a CMYK colour anywhere in the document. That's probably the best you can do.