Thanks for the advice. The LOG stmt is just a method that filters all our
logging then calls into log4j. There is nothing complex about it, it just
filters out some of our logging based on system settings.
I will look into some of your suggestions, thanks for the tips.
Christopher Schultz-2 wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Neil,
>
> On 8/21/2009 11:10 AM, neilgoldsmith wrote:
>> periodically they will experience a big slowdown on the app server
>> which does eventually recover (slows down for maybe an hour). We had
>> them increase their maxThreads from 150 to 300 and increase the heap
>> size
>
> If the heap size were a problem, you would be getting OutOfMemoryErrors.
> I wouldn't bother increasing the heap size.
>
> If periodic slowdowns are occurring on one box, and you can observe them
> using code like this:
>
>> LOG("ICConnectorManager.doGet: Wait time to return response:" +
>> (System.currentTimeMillis() - waitStart) + "ms for request ID:" + reqID);
>> LOG("ICConnectorManager.doGet: Looking up " + reqID + " and returning
>> response back to caller:" + sResponse);
>
> ...then I would guess that increasing the number of worker threads would
> exacerbate the problem rather than resolving it: your webapp is more
> likely to be thrashing than anything else, and adding more threads isn't
> likely to help.
>
> You didn't say anything that would lead me down another path, so I'll
> tell you what my gut tells me: you have a bad hard disk or NFS link
> where your log files are being written. If that's the case, you'll see
> multi-second hiccups and stuff like that while the log statements
> attempt to flush to the disk (which may be masked at times by OS-level
> buffering, etc.).
>
> Thrashing and disk problems are the only things I can think of, and you
> say that the box recovers (correlated to a drop in traffic?) and that
> nothing else is running on the box, so I'd bet on #2.
>
> Peter's suggestion that GC could be interfering is a good one, except
> that you'd only expect to see several hiccups in performance maybe over
> a 5-to-10 second period, depending on your heap settings (and 1GiB isn't
> a very big heap). It shouldn't take very long to recover from a heap
> shuffle. Knowing your JVM version and GC settings would be good, too.
>
> Peter's question about app-level synchro locks is a good one, too. What
> does the LOG method actually do? That's not a log4j method, so you've
> probably wired log4j into your application using a log-agnostic API. Is
> it possible you've synchronized that and created a problem for yourself?
>
> Finally, what else can you tell us about these slowdowns? Do they always
> occur at the same time of day? With the same amount of (high) volume? If
> it happens at exactly the same time every day, consider blaming a backup
> or similar process as has been suggested.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkqO258ACgkQ9CaO5/Lv0PDY4wCdGK5ioG0gGP+dOAmtriEKnp2T
> coMAoKLSfFV2IpriuRwpUkUlLRWxoAmG
> =PbvQ
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
>
--
View this message in context:
http://www.nabble.com/Tracking-down-a-Tomcat-slowdown-tp25081707p25085023.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
