Chinese punctuation following English/Latin characters are not rendered

I'm displaying a mix of English and Chinese in the same paragraph. Whenever chinese punctuation (0xFE00 to 0xFE6F)  is following latin characters (0x0000 to 0x00FF), the chinese punctuation is shown as "empty square". The punctuation displays correctly otherwise when not following latin characters.

It behaves the same on Windows and Mac. I'm using device 'Arial' font.  You can see that in the two attachments. They're from the same <span>, the chinese comma (0xFE50) is not displayed in 'NotWorking.tiff' file, but displayed correctly in 'Working.tiff'.

Any help would be appreciated. Thanks.