Well, if you have 4 cores and all cores are looping tomcat definitely will not
respond any more...
Von meinem iPad gesendet
Am 30.09.2012 um 12:42 schrieb Jeff MAURY <jeffma...@jeffmaury.com>:
> I don't think a cpu loop will make tomcat stopping responding to requests
> I will make it very slow to respond
> But a shortage on memory is hard to recover
>
> Jeff
>
> Le vendredi 28 septembre 2012, mailingl...@j-b-s.de a écrit :
>
>> Maybe an infinite loop? We observed something similar due to a bug in the
>> java regex impl and certain user input causes this regex looping behaviour.
>> As this locked one core but to the user it simply looked like server was
>> not responding, guess what happend? Right: they press refresh page and next
>> core gone :-)
>> So running state might be evil, too.
>> Switch on GC logging, maybe its just related to a full gc. In case this
>> happens again take a thread dump and search similarities. Take multiple
>> dumps in a 5 sec interval. Try to find long running threads (in our case we
>> noticed the regex bug)
>>
>> Jens
>>
>> Sent from my iPhone
>>
>> On 27.09.2012, at 22:05, Kirill Kireyev <kir...@instagrok.com> wrote:
>>
>>> Thanks for all the advice everyone! There is a possibility that the CPU
>> is caused by an app thread - I am looking into that possibility. Will let
>> you know when I find out more.
>>>
>>> Thanks,
>>> Kirill
>>>
>>> On 9/27/12 12:17 PM, Shanti Suresh wrote:
>>>> Hi Kirill,
>>>>
>>>> Like Mark, Bill and Jeff said, those threads are normal
>> request-processing
>>>> threads. I have included a script that might help with isolating high
>> CPU
>>>> issues with Tomcat.
>>>>
>>>> Also, I think it might be helpful to see how the Java heap is
>> performing as
>>>> well.
>>>> Please bring up Jconsole and let it run over the week. Inspect the
>> graphs
>>>> for Memory, CPU and threads. Since you say that high CPU occurs
>>>> intermittently several times during the week and clears itself, I
>> wonder if
>>>> it is somehow related with the garbage collection options you are using
>> for
>>>> the server. Or it may be a code-related problem.
>>>>
>>>> Things to look at may include:
>>>>
>>>> (1) Are high CPU times related to Java heap reductions happening at the
>>>> same time? ==> GC possibly needs tuning
>>>> (2) Are high CPU times related to increase in thread usage? ==>
>> possible
>>>> livelock in looping code?
>>>> (3) how many network connections come into the Tomcat server during
>>>> high-CPU times? Possible overload-related?
>>>>
>>>> Here is the script. I made a couple of small changes, for e.g.,
>> changing
>>>> the username. But didn't test it after the change. During high-CPU
>> times,
>>>> invoke the script a few times, say 30 seconds apart. And then compare
>> the
>>>> thread-dumps. I like to use TDA for thread-dump analysis of Tomcat
>>>> thread-dumps.
>>>>
>>>> Mark, et al, please feel free to help me refine this script. I would
>> like
>>>> to have a script to catch STUCK threads too :-) Let me know if anyone
>> has
>>>> a script already. Thanks.
>>>>
>>>> --------------high_cpu_diagnostics.pl:-----
>>>> #!/usr/bin/perl
>>>> #
>>>>
>>>> use Cwd;
>>>>
>>>> # Make a dated directory for capturing current diagnostics
>>>> my ($sec,$min,$hour,$mday,$mon,$year,
>>>> $wday,$yday,$isdst) = localtime time;
>>>> $year += 1900;
>>>> $mon += 1;
>>>> my $pwd = cwd();
>>>> my $preview_diag_dir =
>> "/tmp/Preview_Diag.$year-$mon-$mday-$hour:$min:$sec";
>>>> print "$preview_diag_dir\n";
>>>> mkdir $preview_diag_dir, 0755;
>>>> chdir($preview_diag_dir) or die "Can't chdir into $preview_diag_dir
>> $!\n";
>>>>
>>>> # Capture Preview thread dump
>>>> my $process_pattern = "preview";
>>>> my $preview_pid = `/usr/bin/pgrep -f $process_pattern`;
>>>> my $login = getpwuid($<) ;
>>>> if (kill 0, $preview_pid){
>>>> #Possible to send a signal to the Preview Tomcat - either
>> "webinf"
>>>> or "root"
>>>> my $count = kill 3, $preview_pid;
>>>> }else {
>>>> # Not possible to send a signal to the VCM - use "sudo"
>>>> system ("/usr/bin/sudo /bin/kill -3 $preview_pid");
>>>> }
>>>>
>>>> # Capture Preview thread dump
>>>> system ("/usr/bin/jmap
>>>> -dump:format=b,file=$preview_diag_dir/preview_heapdump.hprof
>> $preview_pid");
>>>>
>>>> # Gather the top threads; keep around for reference on what other
>> threads
>>>> are running
>>>> @top_cmd = ("/usr/bin/top", "-H", "-n1", "-b");
>>>> @sort_cmd = ("/bin/sort", "-r", "-n", "-k", "9,9");
>>>> @sed_cmd = ("/bin/sed", "-n", "'8,$p'");
>>>> system("@top_cmd 1> top_all_threads.log");
>>>>
>>>> # Get your tomcat user's threads, i.e. threads of user, "webinf"
>>>> system('/usr/bin/tail -n+6 top_all_threads.log | /bin/sort -r -n -k
>> "9,9" |
>>>> /bin/grep webinf top_all_threads.log 1> top_user_webinf_threads.log');
>>>>
>>>> # Get the thread dump> <instaGrok_sml.png>
>>
>
>
> --
> Jeff MAURY
>
>
> "Legacy code" often differs from its suggested alternative by actually
> working and scaling.
> - Bjarne Stroustrup
>
> http://www.jeffmaury.com
> http://riadiscuss.jeffmaury.com
> http://www.twitter.com/jeffmaury
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
