cfthread in cfloop, all variables passed in on the thread, intermittent java.util.concurrentmodificationexception. cfloop is a query loop and each thread calls cfquery. How do I find the error?

Copy link to clipboard
Copied
Code
Below is the script, the queries in the thread only use the variables on the cfthread call. The process works about 80% of the time and then I receive Java.util.concurrentmodificationexception on one of the threads, occasionally more than one. Using CF 9,0,0,251028, but have tested with 9,0,2,282541 with the same issue.
Any hekp is appreciated
<cfquery name = "variables.Dts" datasource="#mydsn#">
select
to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7-2,'mm/dd/yyyy') as thu,
to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7-1,'mm/dd/yyyy') as fri,
to_char(next_day(to_date('#selDate#','mm/dd/yyyy')-56,'sat')+(level-1)*7,'mm/dd/yyyy') as sat
from dual
connect by level < 9
</cfquery>
<!--- test --->
<cfset qtr = 0>
<cfset tname = ''>
<cfloop query="variables.dts" >
<cfset qtr += 1>
<cfthread name="t#qtr#" action="run" sat2='#variables.dts.sat#' thu2='#variables.dts.thu#' selarea2='#selArea#' selDistrict2='#seldistrict#' qtr2='#qtr#' dsn="#mydsn#" >
<cfquery name="variables.m#qtr2#" datasource="#dsn#" >
...
</cfquery>
</cfthread>
<cfset tname &= 't#qtr#' & ','>
</cfloop>
<cfloop query="variables.dts" >
<cfset qtr += 1>
<cfthread name="t#qtr#" action="run" sat1='#variables.dts.sat#' fri1='#variables.dts.fri#' selarea1='#selArea#' selDistrict1='#seldistrict#' qtr1='#qtr#' dsn="#mydsn#">
<cfquery name="variables.m#qtr1#" datasource="#dsn#" >
...
</cfquery>
</cfthread>
<cfset tname &= 't#qtr#' & ','>
</cfloop>
<cfset tname = left(tname,len(tname)-1)>
<cfthread action="join" name="#tname#" />
Copy link to clipboard
Copied
I moved this to the Advanced Techniques subforum because many of the more experienced ColdFusion community members hang out there. If you don't get an answer, you might try joining the CFML Slack channel, as you can get access to more real-time community assistance. I'll even put a link there to this thread to see if I can draw some attention.
-Carl V. - Moderator
Copy link to clipboard
Copied
What is your motivation for using threads? I cannot see any, yet the threads add a lot of complexity to the code.
The database is an independent system. It should therefore be treated as a black-box which may have its own concurrency strategy, beyond Coldfusion's control. Before you know it, there are worms everywhere.
Aside from that, if you replace <cfset tname &= 't#qtr#' & ','> with <cfset tname = listAppend(tname, 't#qtr#')> you will save yourself one line of code.

Copy link to clipboard
Copied
Thank you for the reply
The threads are being used to speed up the retrieval of the data. It works well, I can shave minutes off the total query time. Each query returns the data of a single partition and is combined in a dbquery. It provides in a week analysis of selected facilities almost instantly. It is exactly what is needed, if I can resolve the concurrentmodificationexception. Thank you for the listAppend suggestion, much more readable.
Copy link to clipboard
Copied
You have omitted some code (marked by ellipsis), so it is difficult to say. In any case, the hint from the concurrentmodificationexception is that a thread th6, say, attempted to modify a variable variables.somevar while thread th7was still busy with it.
As your threads are asynchronous, execution may already move to the second loop while threads of the first loop are still busy. Is that consistent with your business logic?

Copy link to clipboard
Copied
I agree with your assessment and appreciate your time, I tried to eliminate any access by the thread to local variables. I have added the complete code (below), if you have time to consider
The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error
T7
struct
ELAPSEDTIME 0
ERROR
struct
Message
StackTrace java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)
TagContext
array
Type java.util.ConcurrentModificationException
NAME T7
OUTPUT
PRIORITY NORMAL
STARTTIME {ts '2015-12-20 19:06:13'}
STATUS TERMINATED
*************
,'mm/dd/yyyy') start_the_clock_date,
sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,
0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc
from V_mstr a
inner join BI_MSTR_SVC_STD b
on b.ORGN_ENTERED = 'N'
and b.ML_CL_CODE= a.ML_CL_CODE
and b.SVC_STD = a.SVC_STD
INNER JOIN
V_facility c
ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID
inner join
(select distinct site_name as area_name,area,district_name,district
from v_hier) ad
on ad.district = c.district
where a.start_the_clock_date = to_date(
*********************
William R Lance
425.408.0855
425.482.0375 - Cell
Pacific Time Zone
The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error
T7
struct
ELAPSEDTIME 0
ERROR
struct
Message
StackTrace java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)
TagContext
array
Type java.util.ConcurrentModificationException
NAME T7
OUTPUT
PRIORITY NORMAL
STARTTIME {ts '2015-12-20 19:06:13'}
STATUS TERMINATED
*************
,'mm/dd/yyyy') start_the_clock_date,
sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,
0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc
from V_mstr a
inner join v_SVC_STD b
on b.ORGN_ENTERED = 'N'
and b.ML_CL_CODE= a.ML_CL_CODE
and b.SVC_STD = a.SVC_STD
INNER JOIN
V_facility c
ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID
inner join
(select distinct site_name as area_name,area,district_name,district
from v_hier) ad
on ad.district = c.district
where a.start_the_clock_date = to_date(
Copy link to clipboard
Copied
Something seems to have gone wrong. Your last post is unclear.

Copy link to clipboard
Copied
I agree with your assessment and appreciate your time, I tried to eliminate any access by the thread to local variables. I have added the complete code (below), if you have time to consider
The below code works most of the time , but occasionally a thread ( in this example t7) will reurn this error
T7
struct
ELAPSEDTIME 0
ERROR
struct
Message
StackTrace java.util.ConcurrentModificationException at java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:762) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:801) at java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:799) at java.util.HashMap.putAll(HashMap.java:523) at coldfusion.monitor.memory.MemoryTracker.getAllPrimitiveMap(MemoryTracker.java:92) at coldfusion.monitor.memory.MemoryTracker.getEstimatedMemory(MemoryTracker.java:117) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:100) at coldfusion.monitor.memory.MemoryCalculator.calculateMemory(MemoryCalculator.java:30) at coldfusion.monitor.memory.MemoryTrackerProxy.onAddObject(MemoryTrackerProxy.java:277) at coldfusion.runtime.NeoPageContext.setVariableScope(NeoPageContext.java:1546) at coldfusion.runtime.NeoPageContext.SymTab_initializeForPage(NeoPageContext.java:963) at coldfusion.runtime.NeoPageContext.initializeWith(NeoPageContext.java:293) at coldfusion.thread.Task.run(Task.java:131) at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:201) at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)
TagContext
array
Type java.util.ConcurrentModificationException
NAME T7
OUTPUT
PRIORITY NORMAL
STARTTIME {ts '2015-12-20 19:06:13'}
STATUS TERMINATED
*************
,'mm/dd/yyyy') start_the_clock_date,
sum(piece_cnt) AS BLUE_tot, sum(INCL_SVC_VAR_MINUS4_CNTINCL_SVC_VAR_MINUS3_CNTINCL_SVC_VAR_MINUS2_CNT) as blue,
0 AS ORANGE_tot, 0 as orange,hyb_svc_std_desc
from V_mstr a
inner join BI_MSTR_SVC_STD b
on b.ORGN_ENTERED = 'N'
and b.ML_CL_CODE= a.ML_CL_CODE
and b.SVC_STD = a.SVC_STD
INNER JOIN
V_facility c
ON c.fac_SEQ_ID = ORGN_FAC_SEQ_ID
inner join
(select distinct site_name as area_name,area,district_name,district
from v_hier) ad
on ad.district = c.district
where a.start_the_clock_date = to_date(
Copy link to clipboard
Copied
I think the forum software is clipping/truncating your code. Maybe post it to PasteBin and link back here?

