Problems with encoding while fetching data

[mirza36163514e0ud]

Hi everyone, I'm fetching data from database with curl (system.callSystem), and data encoding is UTF-8. When I fetch data it returns wrong data. For example, correct should be: 'Aïn Beïda' but it returns 'Aïn Beïda'. I tried fetching same exact command in terminal and the encoding is correct, seems like extendscript is doing something weird. Any ideas? Thanks!

[Mylenium]

This appears to be a bug, plain and simple. You may need to convert to CSV and use that.

Mylenium

[mirza36163514e0ud]

I tried in AE 2023, AE 2022 and even CS6, I get exactly the same result. Is it possible that it's a bug in all of these? I know I can get it to work if I use external files, but I wanted to avoid hasle of fs. Thanks for your reply.

[Mylenium]

It seems it was introduced in one of the recent Windows updates and has to do with the system's unicode and font handling. That's why it may affect legacy versions as well.

Mylenium

[mirza36163514e0ud]

I see. That makes sense, because I don't remember having this issue before. Do you have any suggestion maybe for a workaround without writing files with filesystem, because I'm fetching the data from server and I wanted to avoid fs.

[Mylenium]

It would probably make sense to check the system's language & region settings and play around with them and possibly also uninstall the offending update. Since this is a known issue that also affects printing and text editing in general there's bound to be a fix update for it not too far away, presumably in may with the next big Microsoft updates. I honestly can't tell you more than that, though. I read about this stuff, but as a German with plain Western settings I'm not affected by the problem and haven't looked deeper into it.

Mylenium

[mirza36163514e0ud]

I will try and post to this forum if there are changes, so other can benefit. Thanks a lot!