Skip to main content
Known Participant
December 12, 2008
Question

Reading/Parsing a CSV file in UTF-16 ?

  • December 12, 2008
  • 2 replies
  • 469 views
Hello everyone,

I'm in rush to modify my current CSV file parser that works fine for files in UTF-8 , to be able to parse the UTF-16 as well, as far as I checked the sample plugins, didn't find any code,
Also how could have support for both encodings? to do this I need to recognize the encoding by reading the file first then decide how to read from stream, any advice/ snippet will greatly appreciated.

P.S. I'm using this code to read a file

stream = StreamUtils::CreateFileStreamRead()
.
.
stream ->XferByte(aChar) // in a loop till find a eol char

I need to use to read the 2 bytes, i had some experiment with XferInt16 but seems it doesn't do what i want...

Regards,
Kamran
This topic has been closed for replies.

2 replies

_Kamrun_Author
Known Participant
December 12, 2008
I had forgotten to skip the first two bytes in this case, Now I can read the file properly with XferInt16, also you may consider Byte Swapping for BigEndian in parsing process.

-Kamran
Inspiring
December 12, 2008
Haven't really work on that but PMString.h seems to support UTF16 and PMString can also deal with IPMStream...

perhaps you will need to use two classes instead of one?

pete