cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception. cfloop is a query loop and each thread calls cfquery. How do I find the error?

[Guest]

Code

Below is the script, the queries in the thread only use the variables on the cfthread call. The process works about 80% of the time and then I receive Java.util.concurrentmodificationexception on one of the threads, occasionally more than one.  Using CF 9,0,0,251028, but have tested with 9,0,2,282541 with the same issue.

Any hekp is appreciated

<cfquery name = "variables.Dts" datasource="#mydsn#">

select 

to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7-2,'mm/dd/yyyy') as thu,

to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7-1,'mm/dd/yyyy') as fri,

to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7,'mm/dd/yyyy') as sat

from dual

connect by  level <  9

</cfquery>

<!--- test --->

<cfset qtr = 0>

<cfset tname = ''>

<cfloop query="variables.dts" >

<cfset qtr += 1>

<cfthread name="t#qtr#" action="run" sat2='#variables.dts.sat#' thu2='#variables.dts.thu#' selarea2='#selArea#' selDistrict2='#seldistrict#' qtr2='#qtr#' dsn="#mydsn#" >

<cfquery name="variables.m#qtr2#" datasource="#dsn#" >

...

</cfquery>

</cfthread>

<cfset tname &= 't#qtr#' & ','>

</cfloop>

<cfloop query="variables.dts" >

<cfset qtr += 1>

<cfthread name="t#qtr#" action="run" sat1='#variables.dts.sat#' fri1='#variables.dts.fri#' selarea1='#selArea#' selDistrict1='#seldistrict#' qtr1='#qtr#' dsn="#mydsn#">

<cfquery name="variables.m#qtr1#" datasource="#dsn#" >

...

</cfquery>

</cfthread>

<cfset tname &= 't#qtr#' & ','>

</cfloop>

<cfset tname = left(tname,len(tname)-1)>

<cfthread action="join" name="#tname#" />

[Carl Von Stetten]

I moved this to the Advanced Techniques subforum because many of the more experienced ColdFusion community members hang out there.  If you don't get an answer, you might try joining the CFML Slack channel, as you can get access to more real-time community assistance.  I'll even put a link there to this thread to see if I can draw some attention.

-Carl V. - Moderator

[BKBK]

What is your motivation for using threads? I cannot see any, yet the threads add a lot of complexity to the code.

The database is an independent system. It should therefore be treated as a black-box which may have its own concurrency strategy, beyond Coldfusion's control. Before you know it, there are worms everywhere.

Aside from that, if you replace <cfset tname &= 't#qtr#' & ','> with <cfset tname = listAppend(tname, 't#qtr#')> you will save yourself one line of code.

[Guest]

Thank you for the reply

The threads are being used to speed up the retrieval of the data. It works well, I can shave minutes off the total query time. Each query returns the data of a single partition and is combined in a dbquery. It provides in a week analysis of selected facilities almost instantly. It is exactly what is needed, if I can resolve the concurrentmodificationexception. Thank you for the listAppend suggestion, much more readable.

[BKBK]

You have omitted some code (marked by ellipsis), so it is difficult to say.  In any case, the hint from the concurrentmodificationexception is that a thread th6, say, attempted to modify a variable variables.somevar while thread th7was still busy with it.

As your threads are asynchronous, execution may already move to the second loop while threads of the first loop are still busy. Is that consistent with your business logic?

[Guest]

I agree with your assessment and appreciate your time, I tried to eliminate any access by the thread to local variables. I have added the complete code (below), if you have time to consider

The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error

T7

struct

ELAPSEDTIME 0

ERROR

struct

Message

StackTrace java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)

TagContext

array

Type java.util.ConcurrentModificationException

NAME T7

OUTPUT

PRIORITY NORMAL

STARTTIME {ts '2015-12-20 19:06:13'}

STATUS TERMINATED

*************

,'mm/dd/yyyy') start_the_clock_date,

sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,

0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc

from V_mstr a

inner join BI_MSTR_SVC_STD b

on b.ORGN_ENTERED = 'N'

and b.ML_CL_CODE= a.ML_CL_CODE

and b.SVC_STD = a.SVC_STD

INNER JOIN

V_facility c

ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID

inner join

(select distinct site_name as area_name,area,district_name,district

from v_hier) ad

on ad.district = c.district

where a.start_the_clock_date = to_date(

*********************

William R Lance

425.408.0855

425.482.0375 - Cell

Pacific Time Zone

The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error

T7

struct

ELAPSEDTIME 0

ERROR

struct

Message

StackTrace java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)

TagContext

array

Type java.util.ConcurrentModificationException

NAME T7

OUTPUT

PRIORITY NORMAL

STARTTIME {ts '2015-12-20 19:06:13'}

STATUS TERMINATED

*************

,'mm/dd/yyyy') start_the_clock_date,

sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,

0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc

from V_mstr a

inner join v_SVC_STD b

on b.ORGN_ENTERED = 'N'

and b.ML_CL_CODE= a.ML_CL_CODE

and b.SVC_STD = a.SVC_STD

INNER JOIN

V_facility c

ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID

inner join

(select distinct site_name as area_name,area,district_name,district

from v_hier) ad

on ad.district = c.district

where a.start_the_clock_date = to_date(

[BKBK]

Something seems to have gone wrong. Your last post is unclear.

[Guest]

I agree with your assessment and appreciate your time, I tried to eliminate any access by the thread to local variables. I have added the complete code (below), if you have time to consider

The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error

T7

struct

ELAPSEDTIME 0

ERROR

struct

Message

StackTrace java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)

TagContext

array

Type java.util.ConcurrentModificationException

NAME T7

OUTPUT

PRIORITY NORMAL

STARTTIME {ts '2015-12-20 19:06:13'}

STATUS TERMINATED

*************

,'mm/dd/yyyy') start_the_clock_date,

sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,

0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc

from V_mstr a

inner join BI_MSTR_SVC_STD b

on b.ORGN_ENTERED = 'N'

and b.ML_CL_CODE= a.ML_CL_CODE

and b.SVC_STD = a.SVC_STD

INNER JOIN

V_facility c

ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID

inner join

(select distinct site_name as area_name,area,district_name,district

from v_hier) ad

on ad.district = c.district

where a.start_the_clock_date = to_date(

[Carl Von Stetten]

I think the forum software is clipping/truncating your code.  Maybe post it to PasteBin and link back here?