You can't convert color mode without flattening. It's not possible if you have adjustment layers - they won't apply in the other color mode.
You can, however, convert between color spaces within the same color mode - if you accept that the file will change appearance and the colors won't necessarily be the same. If you want the file to remain unchanged, you have to flatten. Again, because adjustment layers won't mean the same thing.