Storing unicode (khmer) woes
Hey all,
I am working on an applicaiton that needs to accept the khmer language in various text inputs and mssql database. I have gotten most of it working, but still have one bug. I can display khmer characters if they are typed in. If I copy and paste khmer text directly in my database, and query for it, it comes out properly. The issue is when I take khmer text from a form field and insert it, then it is transformed into a bunch of ?????.
Here are the steps I've taken so far to enable unicode on my website
- Configured the datasource to accept high ascii values and unicode
- Configured the database table columns to be of type nvarchar
- Added
<cfscript>
SetEncoding("form","utf-8");
SetEncoding("url","utf-8");
</cfscript>
<cfcontent type="text/html; charset=utf-8">
to my application.cfm file.
-Added <META http-equiv="Content-Type" content="text/html; charset=utf-8"> in the head of my pages.
-Added <cfprocessingdirective pageEncoding="utf-8"> on my page that attempts to update the database.
It's weird. If i copy and paste khmer directly in the DB and query for it that works fine. If I hard code some khmer on a page, that displays fine to. If I type in khmer into a form, and dump the form value back out, that works. It's only when a form value is saved to the database and pulled back out is it mangled. You can see an example here of what I'm talking about.
http://www.psasmart.com/test.cfm
And here is the code that makes that page.
<cfscript>
SetEncoding("form","utf-8");
SetEncoding("url","utf-8");
</cfscript>
<cfcontent type="text/html; charset=utf-8">
<META http-equiv="Content-Type" content="text/html; charset=utf-8">
<cfprocessingdirective pageEncoding="utf-8">
If you have the Khmer language pack installed this: <h2>ម៉ោងផ្សាយ-រលកធ</h2> should appear as cambodian text.
<hr />
<form name="submitForm" method="post" accept-charset="utf-8">
Now enter some Khmer text to save to the database: <input name="text" type="text" value="ម៉ោងផ្សាយ-រលកធ">
<br />
<input type="submit" name="submit" value="submit" >
</form>
<cfoutput>
<cfif isdefined("form.submit")>
This is the same text as entered in the form: <h2>#form.text#</h2><br />
<cfquery name="update" datasource="#application.dsn#" >
Update serverSettings
SET khmerReadWriteTest ='#form.text#'
</cfquery>
</cfif>
<Cfquery name="getKhemer" datasource="#application.dsn#">
select khmerReadTest, khmerReadWriteTest
from serverSettings
</Cfquery>
This is the same text as entered in the form but saved to the db and queried for then displayed: <h2>#getKhemer.khmerReadWriteTest#</h2>
This is some Sample Khmer Text Inputed Directly in the database then queried for and displayed: <h2>#getKhemer.khmerReadTest#</h2>
</cfoutput>
