Yes, PNG support for icc profiles is unreliable. Jpeg supports profiles reliably, and it also supports CMYK in the format specification, although not always in many applications' jpeg decoding.
Jpeg is usually fine as final delivery format, provided you don't edit and resave it multiple times afterwards. Jpeg data compression is destructive, irreversible and cumulative, and quality degrades with every resave. Never use jpeg as a working/archive format.
The reason jpeg is still widely used, is that the compression is incredibly effective for reducing file size - shrinking a file to, say, 2-10% of original size with little visible quality loss. This can be important for online delivery.
If image quality and integrity is critical, e.g. for high quality print, jpeg is risky. For this, the standard interchange format is TIFF. Much bigger files, but no quality loss whatsoever.
PSD is the native Photoshop format, and the only one that supports all Photoshop functions.
A raw file - many proprietary formats, CR2, NEF etc - is not something you will ever send away. That's the "naked" sensor data from a camera, and it needs processing in a raw processor to become a useful image. Not to be confused with Photoshop .raw, a very specialized format that is almost never used.