CFTEXTAREA, richtext, and black diamond question marks

[WolfShade]

Hello, all,

I have inherited a project that (sadly) uses CFFORM and some of the related elements, including a CFTEXTAREA with richtext="yes" attribute.  This application will take the text from an online news article and insert it into a database to be displayed on a local restricted webpage (with a link to the actual online article), as well as generate an email with all of that days entries and send it to a list of people who are signed up to receive said email.

The datatype for the article text is CLOB (Oracle 12g).  The pages are set to UTF-8 encoding.  I believe the email is also set for type="html" with UTF-8 encoding.

Every so often, text is entered that contains characters that FireFox v52.x.x and IE11 don't like, and are replaced with a black diamond icon that has a question mark inside it.  I believe this is happening with just apostrophe, but I can't swear to it.

What is happening, and how do I correct it?

V/r,

^ _ ^

[haxtbh]

Normally happens because the characters are not UTF-8. Word has its own apostrophes and quote marks that are none standard and can cause this if someone just copies and pastes it. There are a number of resources that talk about this issues though, mainly on stack overflow. i.e. html - Why do symbols like apostrophes and hyphens get replaced with black diamonds on my website? -...

[WolfShade]

Thanks for the link, haxtbh​.  Looking at the code, everything is set for UTF-8.  I'll check with our DBA to see if the database CLOB is set for UTF-8.  That's all I can think of.

V/r,

^ _ ^

[EddieLotter]

I think there is also a setting in the ColdFusion data source for UNICODE strings.

Check the "Advanced Settings" of your data source.

Cheers

Eddie

[WolfShade]

Thanks, Eddie.  I'll check on that.

V/r,

^ _ ^