ColdFusion won't display unicode (utf-8) characters

Report · Nov 29, 2011

I have a very basic page with some unicode characters in it. ColdFusion won't properly display them. If I resave the page as .html (so the coldfusion server doesn't process it), the characters display fine.

Below is my page. Notice that the charset is set to utf-8.

I'm using Dreamweaver CS5.5. Running IIS 7 and CF 9.

The text "why isn’t this working" has a smart apostrophe that DOES NOT DISPLAY PROPERLY when run through CF. I've searched around and all I see is that CF has unicode "turned on by default" but no where can I find where to make sure it is set.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<title>Untitled Document</title>

</head>

<body>

why isn’t this working?

</body>

</html>

Report · Nov 29, 2011

Try adding this to the top of the page:

Report · Nov 29, 2011

Ok, that did work...but do I really need to add that tag to EVERY single page in my entire application? Surely CF wouldn't be so terribly clunky.

I also found that by turning on the BOM in dreamweaver it started to display unicode properly.The problem with turning on the BOM for us is that we develop in a team and it would be very easy for someone to forget to turn this one little setting on as it's NOT the dreamweaver default.

I need a global default. Am I the only one in the world using codlfusion and utf-8? It seems rediculous that CF would make you put that tag in every page, especially when it claims to have UTF-8 as the default anyway.

Any other solutions?

Report · Nov 29, 2011

You could try adding it to your App.cfc, perhaps in onRequest or onRequestStart. I have not tested it, so I don't know if that would work, but I don't know why it wouldn't.

Report · Nov 29, 2011

Nevermind, I guess you can't put it in App.cfc, based on what I am reading. I guess you'll have to go with the so-called "clunky" method.

it seems that having cfprocessingdirective in every template is a documented best-practice in the CFWACK (Vol 2). I learned something new today.

Report · Nov 29, 2011

it seems that having cfprocessingdirective in every template is a documented best-practice in the CFWACK (Vol 2). I learned something new today.

God, really? Who came up with that?

One needs to put it in any file that has characters with "ASCII" codes higher than 127, so as to forewarn the compiler to expect them. I also dunno why one can't just tell the compiler "treat all files this way". It's daft not to.

Anyway, most CFM files - which ought to contain just code - shouldn't have a problem because code won't generally have anything outside the 0-127 range; just the odd hard-coded value. So to recommend to put it at the top of every file is just stupid. Put it in the files that need it. That's it.

As to why it needs to go in the file itself rather than somewhere "upstream" in the request - like OnRequestStart() or something is that this tag - unlike most - is a compile time directive (as opposed to runtime). And at compile time, CF is not running the code, it's just compiling it, so it doesn't know anything about the sequence in which files will be called (indeed it's not actually the CFM files themselves being called anyhow).

--

Adam

Report · Nov 29, 2011

Yeah, having to remember to put it at the top of every page seems like a real PITA to me...especially because the charset is already declared at the top of the page anyway. I want every page to be unicode, so it would make a lot of sense if I could put it in something like application.cfm...sigh.

Report · Nov 30, 2011

Hello,

I encountered this problem previously during my web application development efforts, and discovered a workaround, my CFM files were ANSI and I needed to have them display UTF-8 characters that were static content in the CFM files.

The workaround I used was to open the CFM files with notepad, and then resave them as UTF-8 files with the same code. Notepad gives the option to save a file in ANSI format or in UTF-8 format. After I did this with my web application CFM files they then correctly showed the UTF-8 characters.

Probably not the best solution, but at least it works. Would probably work the same for ASP.NET and PHP files, as well as for COLDFUSION files. Hope this helps.

Michael G. Workman

mworkman@usbid.com

http://www.usbid.com

http://ic.locate-ic.com

Report · Nov 30, 2011

Hi Michael,

I believe what Notepad is doing is just adding the BOM to the file when you save it as "UTF-8". Dreamweaver can do this for you by going to Modify > Page Properties, then clicking Title/Encoding and checking on Include Unicode Signature (BOM). This will work to make CF properly display unicode characters.

The reason this does not seem to be a proper solution to me is that:

1) Developing in a team environment it is pretty tough to enforce that everyone has this checked on. Maybe there is an easy way to tell other than opening every file and going into tthat one dialog in Dreamweaver.

2) PHP and HTML files do not need this BOM in order to display unicode characters. I've already set the charset in the <head> of my document, why do I need to set it again?

Ideally there would be a universal setting in CF to just make it act like HTML, PHP, et al and pay attention to the charset instead of garbling my text. Also CF claims to default to unicode anyway....but this doesn't seem to be the case!! For some reason I now have to add a <cfprocessingdirective> tag to every page in my app that may display text to the user. Seems like a pain and extra work to me!

How come no one else seems to be having this problem?

Report · Nov 30, 2011

>> I've already set the charset in the <head> of my document, why do I need to set it again?

To be clear, these are two different things. Set the encoding in the <head> of your document is for the web browser to know how to render the content. Setting the processing directive is to tell CF how to compile the file. They are very different things for different purposes and, for whatever reason, may not always match.

You could try changing the default that Java uses. Try adding this to your Java settings in the ColdFusion admin:

-Dfile.encoding=UTF8

Jason

Report · Nov 30, 2011

Ok, thanks. That worked!

For others, so it's more clear:

Log into the CF Admin
Under Server Settings click Java and JVM.
Under JVM Arguments add: -Dfile.encoding=UTF8
Restart the server and the page will display unicode.

ColdFusion won't display unicode (utf-8) characters

1 Correct answer

Photos