On 10/02/2016 09:20, Mark Thomas wrote: > On 09/02/2016 16:19, Jeroen van Ooststroom wrote: > >>> Hello Mark, >>> >>> I have been trying to get an isolated test case reproducing the issue, >>> but so far without luck. I'll try to reproduce it again with the >>> original application and get a thread dump for you at the time it got >>> stuck. Is there any Tomcat-level logging you want to see as well >>> leading up to the stuck thread issue? >>> >>> Thanks, >>> Jeroen... >> Hello Mark, >> >> See the thread dump attached. > > Thanks. Is it edited? There are some "..." sections in the thread dump > that don't look normal. > > A further piece of information that would be helpful is the version of > IceFaces you are using.
I've been looking at the source code for IcePush 4.1.0 and I have a theory. Keep in mind I don't know the IcePush code at all. 1. Container Thread A ends up in AsyncAdaptingServlet.service() This calls request.startAsync() followed by pushServer.service(). This moves the async state to STARTING which will only change to STARTED once the container thread exists the AsyncAdaptingServlet.service() 2. Assuming (as it appears from the stack traces) that pushServer is an instance of BlockingConnectionServer, this calls activeServer.service 3. activeServer is an instance of RunningServer. The serviceMethod calls pendingRequest.put(pushRequest). Call this push request A. 4. Meanwhile, non-container thread B is running the "Notification Queue Consumer". It holds a lock on the BlockingConnectionServer instance. It tries to process push request A and calls complete(). This blocks because the AsyncContext is still in state STARTING for that request. 5. Container thread A continues and reaches sendNotifications(). This blocks because that method is synchronized and non-container thread B currently holds the lock. At this point deadlock occurs. The async request is in state STARTING. That state can't change until container thread A exits the service() method. Container Thread A is blocked in the service() method waiting for a lock on BlockingConnectionServer. Container Thread B holds the lock on BlockingConnectionServer and won't release it until the the async request changes state to STARTED. My guess is that the deadlock only occurs if the "Notification Queue Consumer" happens to try processing pending requests in the short period of time between container thread A calling pendingRequest.put(pushRequest) and sendNotifications(). The Servlet spec is clear that any call to AsyncContext.complete() will not complete until the associated container thread has completed processing the original request. On that basis it is my current view that this is an ICEpush bug. You should be able to confirm this fairly easily is you use a debugge HTH, Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
