Special characters coming through messed up (CF7)

Report · Jan 27, 2010

Hi,

I have a form which users need to be able to paste text into the javascript text editor (tinyMCE) from a range of sources. It's then processed by a coldfusion page which stores the data in a MySQL db. When I dump out #FORM# on the processing page, the characters are there how I want them, but when they are inserted into the db, they appear as the little squares with 00XX inside, or as strange non-ascii characters that display all messed up when read out.

I've tried setting the charset to utf-8 but no luck. I had written a function that converts the special characters to their appropriate html codes before inserting into the db but they do not get as far as the function before they are turned into the messed up characters, even though it would appear nothing is touching them - this function was working locally on my windows server, but now doesn't when I upload to the live linux server. By the time they get to the db they are unrecognisable characters - I will try pasting some of the characters the user is pasting in below separated by comma:

£,€,–,—,‘,’,“,”,

I have tried pretty much everything I can think of... thanks to anyone who can help.

T

Report · Jan 27, 2010

There are 3 areas to look at when you have issues.

1) The page. Make sure the page doesn't have another charset specified at the top. many templates and editors will automatically put in something.

2) The datasource. If mysql make sure the connection string specifies UTF-8

3) The database. Make sure the actual DB and tables are set to use UTF-8

If the db or datasource isn't set right then the characters will be messed up forever. Simply changing the datasource will probably not fix what is already saved in the DB. So after making changes do a full test, don't just view the content already entered.

Report · Jan 27, 2010

I agree with Joshua Cyr's assesment. I had a similar problem due to not setting an explicit charset for my web pages. I used CFCONTENT to expliclty output my content as UTF-8. Below is a sample.

Report · Jan 27, 2010

what version of mysql (ie can it actually handle unicode, i think at least 4.x)?

what driver (it cannot be ODBC)? and as joshua points out, make sure the db's

encoding is set to utf-8.