Skip to main content
C
cannedRadio
Known Participant
December 2, 2009
Question

xmlParse is fast, reading the array is not so fast.

  • December 2, 2009
  • 2 replies
  • 537 views

I'm trying to work with xml files that can have up to 5000 children.
On larger files reading the xmlText starts to get timely.

Yesterday I spent the good part of a day trying to optimize strings in coldfusion just to find out the concatenation wasn't my bottle neck.

I'm working with xml documents and xmlparse.  xmlparse() works fine and quickly.
I have parsed and output a small 30 child document.  cftimer proved that outputting the data took 0ms.
I have parsed a large 3500 child document but only looped through the first 30.  cftimer proved that to take 1390ms.  getting through the entire document takes about 2:15 seconds.

This leads me to believe the parsed xml is causing some memory issues.

Does anyone know any optimization settings or a solution I'm not seeing to speed up working with the large structures?
The files I need to work with can contain up to 5000 children.

I'm using CF standard 8.

<cfoutout>
<cfloop from = "1" to = "30" index = "i">
<cfif structKeyExists(xml.x.y,"send")>
(#session.id#,'#xml.x.y.id.xmlText#',
#xml..x.y.send.xmlText#,'#xml.x.y.time.xmlText#',
'#trim(xml.x.y.region.xmlText)#','#trim(replace(xml.x.y.description.xmlText, "'", "''"))#',
'#xml.x.y.name.xmlText#','#xml.x.y.type.xmlText#',
'#xml.x.y.account_balance.xmlText#'),
<cfelse>
(#session. id#,'#xml..x.y.id.xmlText#',
#xml..x.y.receive.xmlText*-1#,'#xml.x.y.time.xmlText#',
'#trim(xml.x.y.region.xmlText)#','#trim(replace(xml.x.y.description.xmlText, "'", "''"))#',
'#xml.x.y.name.xmlText#','#xml.x.y.type.xmlText#',
'#xml.x.y.account_balance.xmlText#'),
</cfif>
</cfloop>
</cfoutput>

    This topic has been closed for replies.

    2 replies

    A
    Anonymous
    December 2, 2009

    Looping over 5000 element will take times I guess.

    I don't know if Coldfusion do implicit optimization, but just in case, factorized "xml.x.y" with a <cfset item = "#xml.x.y#">

    or doing some thing like this:

    <cfloop from = "1" to = "30" index = "i">

    <cfset item = "#xml.x.y#">
    <cfoutout>(#session.id#,'#item.id.xmlText#',</cfoutput>
    <cfif structKeyExists(xml.x.y,"send")>
    <cfoutout>#item.send.xmlText#</cfoutout>
    <cfelse>
    <cfoutout>#item.receive.xmlText*-1#</cfoutout>

    </cfif>

    <cfoutput>,'#item.time.xmlText#',
    '#trim(item.region.xmlText)#','#trim(replace(item.description.xmlTex t, "'", "''"))#',
    '#item.name.xmlText#','#item.type.xmlText#',
    '#item.account_balance.xmlText#'),
    </cfoutput>
    </cfloop>

    C
    cannedRadioAuthor
    Known Participant
    December 2, 2009

    I've tried that too and it doesn't help.

    I've also tried

    <cfset s = structNew()>

    <cfset s = x.y>

    <cfoutput>

    <cfloop from = "1" to = "25" index = "i">

      #s.transactions.send.smlText#<br>

    </cfloop>

    and it didn't help.

    It doesn't appear to be the Loop itsself that's taking a long time.

    if I have a loop:
    <cfloop from = "1" to = "25" index = "i">

    it only loops the first 25 indexes despite the size of the array.

    if i upload a 30 child xml file it takes

    0ms (try 1)

    0ms (try 2)

    15ms (try 3)

    if i upload a 3500 child xml file it takes

    969ms (try 1)

    922ms (try 2)

    954ms (try 3)

    I tried another test....

    <cfset b = "1234567890-0987654321">

    <cfloop from = "1" to = "25" index = "i">

    <cfset a = b>

    </cfloop>

    it takes 0ms every time.

    <cfloop from = "1" to = "25" index = "i">

    <cfset a = x.y.transaction.send.xmlText>

    </cfloop>

    with the small 30 child file it takes 0ms every time.

    with the large 3500 child file it takes ~100ms every time just to set a single variable from the array 25 times.

    C
    cannedRadioAuthor
    Known Participant
    December 2, 2009

    ha, figured it out.

    <cfset xmlshort = arrayNew(1)>

    <cfset xmlshort = xml.x.y>

    <cfloop from = "1" to = "#arrayLen(xmlShort)#" index = "i"> (to is ~3500)

    got throught the entire loop in 1.9 seconds!

    Thanks for the effort though.

    C
    cannedRadioAuthor
    Known Participant
    December 2, 2009

    I just ran the same test on a server I have a bit more control over with the same result.

    Xenon quad core processor 4 gigs of ram.  nothing looked out of place as far as system processing and memory.  Anyone have any ideas?

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices