Thread dump stacktrace stuck

[jimt82977416]

sample stacktrace that shows the threads that are stuck in ‘SocketInputStream.socketRead0’ API. You can notice irrespective of the protocol threads to get stuck on SocketInputStream.socketRead0() API.

"RMI TCP Connection(2)-192.xxx.xx.xx" daemon prio=6 tid=0x000000000a3e8800 nid=0x158e50 runnable [0x000000000adbe000]

java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(Unknown Source)

at java.net.SocketInputStream.read(Unknown Source)

at java.io.BufferedInputStream.fill(Unknown Source)

at java.io.BufferedInputStream.read(Unknown Source)

- locked (0x00000007ad784010) (a java.io.BufferedInputStream)

at java.io.FilterInputStream.read(Unknown Source)

at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

[BKBK]

From experience, the stacktrace seems to me to be

•     from an application that uses replication;
•     the result of a broken connection or of  deadlocked or hanging process.

Could you do a thread dump? That should provide more clues.

For the time being, here's a wild guess: do you see -Djava.net.preferIPv4Stack=true in your JVM settings? If not, add it, and restart ColdFusion.

Do these clues point to a particular process you're using? Also, what's your ColdFusion version? Java version?

[Charlie Arehart]

Jim, can you tell us first why you regard that as a problem? It's an RMI thread, so it's not directly related to processing CF requests. (It's often about java-level monitoring features like JMX or profiling tools).

One thread being in that state is not itself necessarily a problem. Do you have a reason to suspect that it is?

I will note as well that I see one discussion of that specific thread in that specific state being related to some caching tech, and indicating that it's ok to be in that state.  It's not about CF, but it just goes to show that this may not be a problem:

What is rmi tcp connection thread in jira ?

But maybe other threads ARE having an issue, if you are having some problem. Again, what led you to look to this?

[sara001]

If you a thread gets stuck in SocketInputStream.socketRead0 API and doesn’t recover from it for a longer period, then customer whoever originated the transaction will not see any response in his screen. It can puzzle, confuse the user. If multiple threads get stuck in SocketInputStream.socketRead0 API and doesn’t recover for a longer period it can pose serious availability concerns to your application.

Here with we are outlining few potential solutions to address this problem:

1. Instrument timeout settings

      1.1. JVM Network settings

      1.2. setSoTimeout

      1.3. JDBC

      1.4. Oracle JDBC

      1.5. Websphere

      1.6. Axis2

2. Validate Network connectivity

3. Work with remote application

4. Non-blocking HTTP client

Universal Thread dump analysis tool

fastThread, a free online universal tool that will parse and analyze thread dumps written in any language that runs on the JVM. It will convert your Java, Scala, Jython, JRuby thread dumps to useful information for root cause analysis (RCA).

[Charlie Arehart]

jimt82977416, can you let us know if the replies from myself and/or BKBK have helped you with the problem you raised?

Also, as for sara001's subsequent reply, I feel it's appropriate to point out to readers that the text she offered in is in fact some boilerplate text which appears on many other sites.  As such, it's not a personal response by her to the question. Indeed, at first I didn't understand why she was offering these thoughts, which led me to look to around and see if it was content offered elsewhere, when I found it.