On 05/10/2015 10:09, Yogesh Patel wrote: > Hi Thomas, > > Connector configuration is like : > <Connector port="10004" protocol="AJP/1.3" redirectPort="8443" />
That means you are using the BIO AJP connector. You don't have a problem with long running requests. You have a problem with thread starvation. AJP uses persistent connections. BIO requires one thread per connection, regardless of whether or nor that connection is processing a request or waiting for the next one. httpd will (eventually, assuming mostly default config) create one AJP connection for each httpd thread. If you have more httpd threads than Tomcat threads you will eventually reach the point where httpd can't create a Tomcat thread it requires and at that point it will appear to hang. There are several possible fixes: a) disable connection re-use in httpd for AJP connections b) use the NIO AJP connector in Tomcat c) increase maxThreads in Tomcat so it is > max threads in httpd For more explanation read this: http://people.apache.org/~markt/presentations/2015-04-15-Tomcat-clustering-part-1-reverse-proxies.pdf from slide 29 to 33 Mark > > On 5 October 2015 at 14:17, Mark Thomas <ma...@apache.org> wrote: > >> On 05/10/2015 09:07, Yogesh Patel wrote: >>> Thanks Mark Thomas , >>> >>> Our application is access by Apache TO Tomcat using AJP Connector. >> >> OK. That answers one of the questions I asked. However, you haven't >> provided the connector configuration. >> >>> Problem : >>> Our application was mostly hanged,after looking at tomcat manager it >>> shown there are so many long running threads shown. >> >> The Tomcat Manager app does not show long running threads. It does show >> you how many requests are busy but again, a busy thread is not >> necessarily processing a request. >> >>> We want to recognize why so many threads are running since long time. >> >> Threads are always "running". The key question is are those threads >> 'idle' or 'busy'. An idle thread is waiting to be allocated work. A busy >> thread has been allocated work but the manager application can't >> distinguish if that work is 'wait for a request to process' or >> 'processing a request'. >> >>> We want to detect such thread and want to kill these stucked thread. >> >> You continue to make the (increasingly unlikely) assumption that 200 >> busy threads mean you have 200 threads processing long running requests. >> Had you provided the connector configuration I asked for (by that I mean >> the <Connector ... /> elements in server.xml) then I'd be in a better >> position to tell you what is wrong. >> >> If you do have 200 long running requests then the >> StuckThreadDetectionValve is how you detect them. That fact that that >> valve is not reporting any long running requests should be a big clue >> that your assumptions about what is going on are not correct. >> >> If, and it is a big if, you did have 200 concurrent long running >> requests then there is no guaranteed way to stop them. If your >> application checks for interrupts (most don't) then the >> StuckThreadDetectionValve can interrupt them which should terminate the >> processing of that request but the time it would take to code the >> application to handle that properly would normally be better spent >> fixing the root cause of the long running request. >> >> Mark >> >>> >>> >>> >>> >>> >>> On 5 October 2015 at 13:11, Mark Thomas <ma...@apache.org> wrote: >>> >>>> On 05/10/2015 07:54, Yogesh Patel wrote: >>>>> We are facing issues with long running thread in tomcat . we >> are >>>>> using Tomcat-7.0.47.Tomcat manager shows Current busy threads : 200, >>>>> application gets stucked due to these long running threads. >>>> >>>> What makes you think you have issues with long running threads? A busy >>>> thread isn't necessarily processing a request. Configuration errors >>>> leading to thread starvation are a more typical cause of the symptom you >>>> describe rather than long running threads in the application. >>>> >>>>> We implemented StuckThreadDetectionValve in server.xml( <Valve >>>>> >>>>> className="org.apache.catalina.valves.StuckThreadDetectionValve" >>>>> >>>>> threshold="60" />), but it could not help out. >>>> >>>> Why not? Again, this suggests that long running threads aren't the >> problem. >>>> >>>>> So we implemented custom StuckThreadDetectionValve by extending >>>>> StuckThreadDetectionValve from >>>>> catalina, it only goes to "constructor","initInternal",and in >>>> "invoke"(when >>>>> action fires), it does not go to function "getStuckThreadNames()" even >>>>> after threshold time.How to achieve the same.Is there any way to detect >>>>> stucked thread and kill them? >>>> >>>> First of all your invoke() method fails to call super.invoke() so the >>>> Valve is never going to do anything. >>>> >>>> Secondly, if the original valve didn't work adding a bunch of >>>> System.out.println() lines isn't going to magically make it work. >>>> >>>> Thirdly, getStuckThreadNames() is never called by any Tomcat code so it >>>> is no surprise that you never see a call to that method. That method is >>>> intended for use via JMX. >>>> >>>> I suggest you start again and tell us more about the problem you are >>>> trying to solve (lack of response) and your configuration. In particular >>>> we need to know your connector configuration and how users access the >>>> application. Do they connect directly to Tomcat or do they go via a >>>> reverse proxy? >>>> >>>> Mark >>>> >>>> >>>>> >>>>> My custom Valve is like following : >>>>> >>>>> public class StuckThreadDetection extends StuckThreadDetectionValve >>>>> { >>>>> StuckThreadDetection stuckThreadDetection; >>>>> >>>>> public StuckThreadDetection() >>>>> { >>>>> System.out.println("in StuckThreadDetection constructor"); >>>>> >>>>> } >>>>> >>>>> public void invoke(Request request, Response response) throws >>>> IOException, >>>>> ServletException >>>>> { >>>>> System.out.println("in invoke..."); >>>>> >>>>> getNext().invoke(request, response); >>>>> } >>>>> >>>>> @Override >>>>> protected void initInternal() throws LifecycleException >>>>> { >>>>> System.out.println("In initInternal"); >>>>> super.initInternal(); >>>>> } >>>>> >>>>> @Override >>>>> public String[] getStuckThreadNames() >>>>> { >>>>> System.out.println("in getStuckThreadNames..."); >>>>> String[] listStuckedThread = this.getStuckThreadNames(); >>>>> >>>>> for (String threadName : listStuckedThread) >>>>> { >>>>> System.out.println(threadName); >>>>> } >>>>> >>>>> return listStuckedThread; >>>>> >>>>> } >>>>> >>>>> @Override >>>>> public String getInfo() >>>>> { >>>>> System.out.println("In getInfo"); >>>>> return super.getInfo(); >>>>> } >>>>> } >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>> >>>> >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
