JSONDecoder.decode max string size

[Svagers]

Hi!

I googled the web and didn't find any valid answer.

What's happening is when I try to unserialize JSON string containing more that 70 000 characters, JSONDecoder fails to decode it and says JSON parse error.

What could be wrong? I'm creating JSON string with Java's JSONObject and it should be fine

[kglad]

use the native JSON class (if you're publishing for fp 11 or better). otherwise, use mike chambers' JSON class

[Svagers]

I've used as3corelib (JSONDecoder is from this library), native JSON class  - none work. So the question remains - is there any limits?

[sinious]

None that should matter to you.. I can parse 500,000 objects from a string containing over 10 million characters in under a second:

var startTime:uint = getTimer();

var str:String = '{"data":[{"0":"foo 0"}';

for (var i:int = 1; i < 500000; i++)

{

          str += ',{"data":"foo ' + i + '"}';

}

str += ']}';

var obj:Object = JSON.parse(str);

trace("Total time: " + (getTimer() - startTime) + "ms");

trace(str.length + ": " + obj['data'][499999]['data']);

trace:

Total time: 810ms

10888897: foo 499999

[Svagers]

At length 75690 characters it worked, but yet at  76583 is throwed error

[sinious]

What Flash Player version are you targeting?

edit:

as3corelib is much slower (targeting FP 10.3) but still works fine:

import com.adobe.serialization.json.*;

var startTime:uint = getTimer();

var str:String = '{"data":[{"0":"foo 0"}';

for (var i:int = 1; i < 500000; i++)

{

          str += ',{"data":"foo ' + i + '"}';

}

str += ']}';

var obj:Object = JSON.decode(str);

trace("Total time: " + (getTimer() - startTime) + "ms");

trace(str.length + ": " + obj['data'][499999]['data']);

trace:

Total time: 3410ms

10888897: foo 499999

You must have an error in your JSON somewhere or the Java serialization isn't following strict rules.

[Svagers]

I'm using AIR 3.6.0.6090 for Desktop

Ok, so we take out the maximum length of string that can be parsed.

Can you look at the JSON? http://dev.svagers.lv/json.txt

Maybe some hidden characters are disturbing the decoding?

[Svagers]

I know understood one interesting thing. Sometimes it can and sometimes it cannot decode the exact same json string

[Svagers]

could those spaces affect anything? I just got rid of them and not getting the error. But it is still interesting, that sometimes it gave an error, but sometimes not

[sinious]

The escaping is just wasting a ton of characters as well (\"). You can use a validator to see if your JSON is strict:

http://jsonlint.com/

I get errors on that JSON even after I removed all the excess spaces and the \" (PHP?) escaping.

[Svagers]

I used this function to terminate string from quotes and spaces:

str.replace(/"/g, '').replace(/^\s+|\s+$/g, "");

and look fine for now, tested about 20 times

P.S. i used this escaping method before I put data in the database

And I must say that this could do something with some hidden chars or anything else with the whole string, because it came from a .xls file.

[sinious]

As long as it's not in the parse that's good. Small thing but you can strip both of those situations in the same RegExp because the replacement in both situations is the same thing, a quote:

str = str.replace(/([\s\t]+"|\\")/gm,'"');

Glad you got it resolved, please mark helpful/correct so we can filter unanswered questions. Good luck!

[Svagers]

Thanks alot Would not pass this alone

[Svagers]

I found another thing what's is wrong. Few of my strings has unwanted chars, for example 'Let'`s get married'

Is there any way to sanitize/escape every unwanted char in the string to be valid for json?

But still the interesting thing is that sometimes it decodes this string and sometimes not

[Svagers]

Well, there's a more fun.

Look at this again http://dev.svagers.lv/json.txt

I put it in to the http://jsonlint.com/ and it throwed an error. When I removed the last object in the array of objects, it parsed well. But when I tried to parse only that object, it parsed aswell.

So, I don't get it at all. I could blame those unescaped strings, but this is something interesting and I don't know wether it would explain this.

[sinious]

Can you post your own sanatized, unescaped version of what you're trying to parse? I see all sorts of things that don't make sense, such as quotes around "{objects}". They shouldn't be quoted unless you want it to be considered just a string. The format is explained on http://www.json.org for the proper syntax. The diagram for an object is right at the top. No quotes around objects, just around the key and if the value is a string.

As for sanatizing your data, that's unfortunately up to you. RegExp will do quite a bit to make it easy however. You can specify your valid ranges like [A-Za-z0-9\.\,] for just upper and lower case letters, all digits, commas and periods. If any character doesn't match whatever you put in that list you should handle it, either via removal, finding a HTML entity or unicode sequence.

Be weary of the escape policy of JSON. You're not even allowed to have what you might consider perfectly valid in a string, such as a [tab]. That must be escaped with \t or it's not valid strict JSON. Things like that can trip you up quick.

[kglad]

sinous showed tests with both the native fp 11 JSON class and mike chambers' class work without problem.