-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Julien,
Warning: this is long. Like, André-or-Mark-Eggers long. On 12/11/12 7:30 AM, Julien Martin wrote: > I am in reference to the following blog entry: Blog > entry<http://blog.springsource.org/2012/05/06/spring-mvc-3-2-preview-introducing-servlet-3-async-support> > > about Spring MVC 3.2 asynchronous support. > > I understand Tomcat uses a thread pool in order to serve http/web > requests. Furthermore, the above article seems to indicate that > Spring MVC asynchronous support relieves Tomcat's thread pool and > allows for better concurrency in the webapp by using background > threads for "heavy-lift" operations. I believe you are misinterpreting what that post has to say. It's not that a "background" thread itself is more efficient, it's that processing that does not need to communicate with the client can be de-coupled from the request-processing thread-pool that exists for that purpose. An example - right from the blog post - will make much more sense than what I wrote above. Let's take the example of sending an email message. First, some assumptions: 1. Sending an email takes a long time (say, 5 seconds) 2. The client does not need confirmation that the email has been sent If your were to write a "classic" servlet, it would look something like this: doPost() { validateOrder(); queueOrder(); sendOrderConfirmation(); // This is the email response.sendRedirect("/order_complete.jsp"); } Let's say that validation takes 500ms, queuing takes 800ms, and emailing (as above) takes 5000ms. That means that the request, from the client perspective, takes 6300ms (6.3 sec). That's a noticeable delay. Also, during that whole time, a single request-processing thread (from Tomcat's thread pool) is tied-up, meaning that no other requests can be processed by that same thread. If you have a thread pool of size=1 (foolish, yet instructive), it means you can only process a single transaction every 6.3 seconds. Lets re-write the servlet with a background thread -- no "asynchronous" stuff from the servlet API, but just with a simple background thread: doPost() { validateOrder(); queueOrder(); (new Thread() { public void run() { sendOrderConfirmation(); } }).start(); response.sendRedirect("order_complete.jsp"); } So, now the email is being sent by a background thread: the response returns to the client after 1.3 seconds which is a significant improvement. Now, we can handle a request once every 1.3 seconds with a request-processing thread-pool of size=1. Note that a better implementation of the above would be to use a thread pool for this sort of thing instead of creating a new thread for every request. This is what Spring provides. It's not that Spring can do a better job of thread management, it's that Tomcat's thread pool is special: it's the only one that can actually dispatch client requests. Off-loading onto another thread pool for background processing means more client requests can be handled with a smaller (or same-sized) pool. Looking above, you might notice that the validateOrder() and queueOrder() processes still take some time (1.3 seconds) to complete, and there is no interaction with the client during that time -- the client is just sitting there waiting for a response. Work is still getting done on the server, of course, but there's no real reason that the request-processing thread has to be the one doing that work: we can delegate the entire thing to a background thread so the request-processor thread can get back to dispatching new requests. This is where servlet 3.0 async comes into play. Let's re-write the servlet as an asynchronous one. I've never actually written one, so I'm sure the details are going to be wrong, but the idea is the same. This time, we'll do everything asynchronously. doPost() { final AsyncContext ctx = request.startAsync(); (new Thread() { public void run() { validateOrder(); queueOrder(); sendOrderConfirmation(); ctx.getResponse().sendRedirect("/order_complete.jsp"); ctx.complete(); } }).start(); } So, how what happens? When startAsync is called, an AsyncContext is created and basically the request and response are packaged-up for later use. The doPost method creates a new thread and starts it (or it may start a few ms later), then returns from doPost. At this point, the request-processor thread has only spent a few ms (let's say 5ms) setting up the request and then it goes back into Tomcat's thread-pool and can accept another request. Meanwhile, the "background" thread will process the actual transaction. Let's assume that nothing in the run() method above interacts in any way with the client. In the first example (no async), the client waits the whole time for a response from the server, and the request-processing thread does all the work. So, the client waits 6.3 seconds and the request-processing thread is "working" for 6.3 seconds. In the async example, the client will probably still wait 6.3 seconds, but the request-processing thread is back and ready for more client requests after a tiny amount of time. Of course, the transaction is not complete, yet. The background thread will run and process the transaction, including the 5-second email process. Once the email confirmation has been sent, the background thread "sends" a redirect and completes the async request. I'm not sure of the exact details, here, but either the background thread itself (via getRequest().sendRedirect()) pushes the response back to the client, or Tomcat fetches a request-processing thread from the pool and uses that to do the same thing. I can't see why the background-thread wouldn't do that itself, but it's up to the container to determine who does what. The point is that, when using asynchronous requests, fewer request-processing threads can handle a whole lot of load. In the async example, still with a thread-pool of size=1 and an async-setup time of 5ms, that means that you can handle one client transaction every 5ms. That's much better than every 6.3 seconds, don't you think? (Note that this isn't magic: if your background threads are either limited or your system can't handle the number of transactions you are trying to asynchronously-process, eventually you'll still have everyone waiting 6.3 seconds no matter what). So, a recap of throughput (req/sec) of the above 3 implementations: Standard: .15873 Background: .76923 Async: 200.00000 Using asynchronous dispatching can improve our throughput a huge number of times. It's worth repeating what I said earlier: if your server can't actually handle this much load (200 emails per second, let's say), then using asych isn't going to change anything. Honestly, this trick only works when you have a lot of heterogeneous requests. For example, maybe 10% of your traffic is handling orders as implemented above, while the rest of your traffic is for much smaller sub-500ms-requests. There's probably no reason to convert those short-running requests into asynchronous operations. Only long-running processes with no client interaction make any sense for this. If only 10% of your requests are orders, that means that maybe you can process 20 orders and 190 "small" requests per second. That's much better than, say, waiting 6.3 seconds for a single order and then processing a single short request, then another order and so on. Just remember that once all request-processing threads are tied-up doing something, everyone else waits in line. Asynchronous request dispatching aims to run through the line as quickly as possible. It does *not* improve the processing time of any one transaction. - -chris -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDHTdkACgkQ9CaO5/Lv0PDrNACgsaeHmBzr9RMSFuZX9ksX3g9d bKYAniJzbqRjGBAjwxIYihvcyJYV5rIl =l+ie -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
