Hi Christopher, Thanks for your comments.
Regarding the behavior of the non-container thread when an async request gets cancelled, I don't really care exactly how it's handled. Currently, my strategy is to let it finish if it had already started processing when the request got cancelled. Or abort processing if the request got cancelled before processing began. It's the first case that's giving me headaches. Using a shared object that handles coordination between the two worlds certainly makes things clearer from an architectural perspective. However, I still feel like there's a fundamental issue here, no matter how the coordination is designed. When the onComplete signal is received, I feel compelled to NOT return from the method until I can be certain that my application will no longer use HTTP request/response objects. I can forward that signal to the coordinator, but I must still wait for the "all clear" response. Otherwise, as soon as the onComplete method returns, the container tomcat can (and will!) recycle the HTTP request/response objects that the non-container thread may still be referencing/using. If the non-container thread isn't trying to write the response out at that exact moment, then it's a trivial case and I can just discard the HTTP request/response objects and return immediately. But if the non-container thread is actively writing the response out, then I should wait for the non-container thread to complete processing. The non-container thread may already be executing tomcat/container code, so aborting it at the application level is pretty much impossible to do reliably. Thanks again for your valuable insight, Regards, François On Tue, Mar 4, 2025 at 2:43 PM Christopher Schultz <ch...@christopherschultz.net> wrote: > > François, > > On 3/4/25 10:32 AM, François Rajotte wrote: > > Hi, > > > > I'm looking for advice on how to properly synchronize asynchronous > > servlets that use the Java servlet 3.0 async APIs. > > > > Especially, I'm trying to avoid having the servlet experience > > IllegalStateExceptions when accessing HttpServletRequest and > > HttpServletResponse objects that tomcat has recycled. > > > > The theory I'm working with is that: > > The servlet is accessing the HttpServletRequest and > > HttpServletResponse objects from non-container threads. For example, > > we can assume that an asynchronous operation has completed on some > > thread and we want to send the response. > > At any time, tomcat may decide to "complete" the request and recycle > > the HttpRequest and HttpResponse objects. For example, the socket > > could be disconnected or timed out. Tomcat notifies the servlet that > > this is happening by calling the AsyncListener.onComplete method. > > > > The non-container thread may be accessing the HttpServletRequest and > > HttpServletResponse objects while Tomcat is about to recycle the > > objects. So it seems that some sort of synchronization is required at > > the servlet level to avoid accessing recycled objects. > > The strategy I have implemented is to acquire a lock at the servlet > > level when processing the AsyncListener.onComplete callback and also > > when accessing the HttpServletRequest and HttpServletResponse objects > > from non-container threads. That way, we can be sure that the > > non-container thread will never access recycled objects. > > > > However, I've noticed that implementing synchronization at the servlet > > level introduces deadlocks between application locks and internal > > tomcat locks. > > This is exactly what I'd expect, given the explanation above. > > Be advised, I'm not an expert at servlet async. I'm only looking at this > from an architectural perspective, here. > > First, if Tomcat were still "allowed" to terminate the async request, > what would you like the non-container thread to do? Continue? Stop? > Something else? > > > Here are two stack traces taken from tomcat 9.0.97: > > > > Non-container thread calling > > HttpServletResponse.getOutputStream().close(), while holding an > > application-level lock: > > java.lang.Thread.State: BLOCKED (on object monitor) > > at > > org.apache.coyote.AsyncStateMachine.asyncError(AsyncStateMachine.java:421) > > - waiting to lock <0x00000007fc5feec0> (a > > org.apache.coyote.AsyncStateMachine) > > at > > org.apache.coyote.AbstractProcessor.setErrorState(AbstractProcessor.java:121) > > at > > org.apache.coyote.AbstractProcessor.handleIOException(AbstractProcessor.java:665) > > at > > org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:388) > > at org.apache.coyote.Response.action(Response.java:207) > > at > > org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:258) > > at > > org.apache.catalina.connector.CoyoteOutputStream.close(CoyoteOutputStream.java:176) > > > > Container thread calling AsyncListener.onComplete(), AsyncListener > > implementation trying to acquire the application-level lock: > > <application code, stuck trying to acquire the lock held by the > > non-container thread> > > at > > org.apache.catalina.core.AsyncListenerWrapper.fireOnComplete(AsyncListenerWrapper.java:39) > > at > > org.apache.catalina.core.AsyncContextImpl.fireOnComplete(AsyncContextImpl.java:106) > > at > > org.apache.coyote.AsyncStateMachine.asyncPostProcess(AsyncStateMachine.java:284) > > - locked <0x00000007fc5feec0> (a > > org.apache.coyote.AsyncStateMachine) > > at > > org.apache.coyote.AbstractProcessor.asyncPostProcess(AbstractProcessor.java:196) > > at > > org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:83) > > at > > org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:937) > > at > > org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791) > > at > > org.apache.tomcat.util.net.BwSocketProcessor.doRun(BwSocketProcessor.java:24) > > at > > org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) > > > > There is a deadlock between the application-level lock and the > > AsyncStateMachine object monitor. They are acquired in different order > > by two different threads. > > > > What would be the recommendation to avoid this deadlock? > > - Don't synchronize at the application/servlet level, and accept that > > sometimes an IllegalStateException will be thrown on the non-container > > thread? But then, isn't the IllegalStateException a symptom that a > > recycled object is being accessed? What if the recycled object is > > already being reused for another purpose? Doesn't that risk cross-talk > > between requests? > > > > In general, how can we ensure consistency when accessing the > > HttpServletRequest and HttpServletResponse from a non-container thread > > while running in async mode? > > I would say that you should manage "synchronization" (maybe > "coordination" since synchronization might imply the use of the > synchronized keyword in Java?) by using some kind of helper object that > handles events coming from both the Servlet world (e.g. onComplete) and > your non-container-thread world (e.g. sendDataBackToClient). > > If you have a class that can be shared between the two worlds, an object > of that class can be notified that Tomcat is completing the request, and > even notify the non-container-thread that its work will not be able to > be returned to the client, and it can stop if it can. > > That same object will also be able to veto and/or ignore any data sent > to it from the non-container thread in a hypothetical > sendDataBackToClient method. > > Does that make any sense? > > -chris > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
