Hi! I wonder if this is the same problem we have, but we are using IIS6 and isapi_redirect.dll and if I'm not totally wrong the same codebase is used for apache and iis.
We have come to the conclusion that there is a error in the "free memory" causing a crash when the pool workers of the iis is unloaded, unfortunaly we have not managed to see what is causing the error. Am I wrong or is this a Win 2003 Server error which can be helped with the suggested patch. I wrote a longer mail about our problems: http://marc.theaimsgroup.com/?l=tomcat-user&m=117092232007880&w=2 /Per Jonsson -----Original Message----- From: RChartier [mailto:[EMAIL PROTECTED] Sent: den 9 februari 2007 16:00 To: users@tomcat.apache.org Subject: Re: [OT] Apache mod_jk memory leak? Hi, I am using Apache 2.0.59 and Tomcat 5.0.28 with mod_jk 1.2.20 on Windows 2003 SP1 but the system have some issue on the socket layer. In fact, after some time, some users are able to crash our architecture: apache answers to request such as *.html but mod_jk says it cannot connect to tomcat. Even a restart of services doesn't change anythink. Tomcat is running and answer for simple "hello world" jsp pages but cannot answer to /manage/html. I have exacltly the same error logs and thread dumps as you are. I patch windows with KB 931311 and it seems to work properly. JNeuhoff wrote: > > I have just managed to repeat the error. 2 of us, from 2 different > workstations, hammered our website for a minute, by rapidly clicking > on links within the same site before it ended up always responding > with a standard Error 503 (Service unavailable) coming from the Apache frontend. > I checked all the log files, and again the only errors recorded were > found in the mod_jk.log: > > .... > [Fri Jan 19 11:34:42 2007] [0404:2392] [info] mod_jk.c (2142): > Service error=0 for worker=ajp13 [Fri Jan 19 11:34:42 2007] > [0404:2388] [error] jk_ajp_common.c (948): > (ajp13) can't receive the response message from tomcat, network > problems or tomcat is down (127.0.0.1:8009), err=-53 [Fri Jan 19 > 11:34:42 2007] [0404:2388] [error] jk_ajp_common.c (1566): > (ajp13) Tomcat is down or refused connection. No response has been > sent to the client (yet) [Fri Jan 19 11:34:42 2007] [0404:2388] [info] > jk_ajp_common.c (1877): > (ajp13) receiving from tomcat failed, recoverable operation attempt=1 > [Fri Jan 19 11:34:42 2007] [0404:2388] [info] jk_ajp_common.c (1916): > (ajp13) sending request to tomcat failed, recoverable operation > attempt=2 [Fri Jan 19 11:34:42 2007] [0404:2388] [error] jk_ajp_common.c (1928): > (ajp13) Connecting to tomcat failed. Tomcat is probably not started or > is listening on the wrong port .... > > > The 2 connections between Apache and Tomcat according the the netstat > -a command just after the web server started the 503 (Service > unavilable) responses (hightlighted in bold): > > Proto Local Address Foreign Address State > TCP DAOHPRW2:epmap DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:microsoft-ds DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:1078 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:3389 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:8009 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:8080 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:10000 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:2093 DAOHPRW2.essex.ac.uk:8009 TIME_WAIT > TCP DAOHPRW2:2110 DAOHPRW2.essex.ac.uk:8009 TIME_WAIT > TCP DAOHPRW2:8005 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:netbios-ssn DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:2076 daohprp2.essex.ac.uk:3306 ESTABLISHED > TCP DAOHPRW2:2077 daohprp2.essex.ac.uk:microsoft-ds > TIME_WAIT > TCP DAOHPRW2:2176 sernt29.essex.ac.uk:epmap ESTABLISHED > TCP DAOHPRW2:2177 sernt29.essex.ac.uk:1025 ESTABLISHED > TCP DAOHPRW2:2181 sernt29.essex.ac.uk:microsoft-ds > TIME_WAIT > TCP DAOHPRW2:2194 sernt50.essex.ac.uk:epmap ESTABLISHED > TCP DAOHPRW2:2204 sernt5.essex.ac.uk:epmap ESTABLISHED > TCP DAOHPRW2:2215 sernt25.essex.ac.uk:epmap ESTABLISHED > TCP DAOHPRW2:2230 dalinux6a.essex.ac.uk:netbios-ssn > ESTABLISHED > TCP DAOHPRW2:2250 sernt2.essex.ac.uk:netbios-ssn TIME_WAIT > TCP DAOHPRW2:2253 bodger.essex.ac.uk:microsoft-ds TIME_WAIT > TCP DAOHPRW2:http DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:http daohpr1.essex.ac.uk:1528 TIME_WAIT > TCP DAOHPRW2:http daohpr2.essex.ac.uk:1936 TIME_WAIT > TCP DAOHPRW2:netbios-ssn DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:2232 sernt23.essex.ac.uk:netbios-ssn TIME_WAIT > TCP DAOHPRW2:2237 sernt9.essex.ac.uk:netbios-ssn TIME_WAIT > TCP DAOHPRW2:2245 serlx08.essex.ac.uk:netbios-ssn TIME_WAIT > TCP DAOHPRW2:2249 sernt2.essex.ac.uk:netbios-ssn TIME_WAIT > TCP DAOHPRW2:2261 sernt23.essex.ac.uk:netbios-ssn TIME_WAIT > TCP DAOHPRW2:3389 daohpr1.essex.ac.uk:1533 ESTABLISHED > > > Half another later, long after the connection_pool_timeout, > connect_timeout and prepost_timeout, the netstat -a command still > shows > this: > > Proto Local Address Foreign Address State > TCP DAOHPRW2:epmap DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:microsoft-ds DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:1078 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:3389 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:8009 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:8080 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:10000 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:8005 DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:netbios-ssn DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:2076 daohprp2.essex.ac.uk:3306 ESTABLISHED > TCP DAOHPRW2:2754 sernt29.essex.ac.uk:microsoft-ds > TIME_WAIT > TCP DAOHPRW2:2782 sernt29.essex.ac.uk:epmap ESTABLISHED > TCP DAOHPRW2:2783 sernt29.essex.ac.uk:1025 ESTABLISHED > TCP DAOHPRW2:http DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:netbios-ssn DAOHPRW2.essex.ac.uk:0 LISTENING > TCP DAOHPRW2:microsoft-ds daohpr1.essex.ac.uk:1541 ESTABLISHED > TCP DAOHPRW2:3389 daohpr1.essex.ac.uk:1533 ESTABLISHED > > > Yet if I then try again to access the web service running on Tomcat, > mod_jk fails to re-establish a connection, according to the mod_jk.log: > > [Fri Jan 19 12:50:51 2007] [0404:2404] [error] jk_ajp_common.c (948): > (ajp13) can't receive the response message from tomcat, network > problems or tomcat is down (127.0.0.1:8009), err=-53 [Fri Jan 19 > 12:50:51 2007] [0404:2404] [error] jk_ajp_common.c (1566): > (ajp13) Tomcat is down or refused connection. No response has been > sent to the client (yet) [Fri Jan 19 12:50:51 2007] [0404:2404] [info] > jk_ajp_common.c (1877): > (ajp13) receiving from tomcat failed, recoverable operation attempt=0 > [Fri Jan 19 12:50:51 2007] [0404:2404] [info] jk_ajp_common.c (1916): > (ajp13) sending request to tomcat failed, recoverable operation > attempt=1 [Fri Jan 19 12:50:52 2007] [0404:2404] [error] jk_ajp_common.c (948): > (ajp13) can't receive the response message from tomcat, network > problems or tomcat is down (127.0.0.1:8009), err=-53 [Fri Jan 19 > 12:50:52 2007] [0404:2404] [error] jk_ajp_common.c (1566): > (ajp13) Tomcat is down or refused connection. No response has been > sent to the client (yet) [Fri Jan 19 12:50:52 2007] [0404:2404] [info] > jk_ajp_common.c (1877): > (ajp13) receiving from tomcat failed, recoverable operation attempt=1 > [Fri Jan 19 12:50:52 2007] [0404:2404] [info] jk_ajp_common.c (1916): > (ajp13) sending request to tomcat failed, recoverable operation > attempt=2 [Fri Jan 19 12:50:52 2007] [0404:2404] [error] jk_ajp_common.c (1928): > (ajp13) Connecting to tomcat failed. Tomcat is probably not started or > is listening on the wrong port [Fri Jan 19 12:50:52 2007] ajp13 > daohprw2-a 0.281333 GET /ohpr/servlet/ > HTTP/1.1 200 > [Fri Jan 19 12:50:52 2007] [0404:2404] [info] mod_jk.c (2142): > Service error=0 for worker=ajp13 > > Only Apache continues to respond with a standard 503 error according > to its access.log: > > daohpr1.essex.ac.uk - - [19/Jan/2007:12:50:51 +0000] "GET > /ohpr/servlet/ HTTP/1.1" 503 1101 "-" "Mozilla/5.0 (Windows; U; > Windows NT 5.1; en-US; > rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9" > > In short: There were only 2 established connections, Tomcat was > running all the time, and, as expected, only saw 2 active > HTTPSessions, yet the rapid hammering on weblinks within the website > somehow blocked the connections between Apache and Tomcat. And now > Tomcat keeps just listening on port 8009, but mod_jk is unable to > establish any TCP connections between Apache and Tomcat. Only a > complete server machine re-boot gets things back to normal. > > Any ideas what might have gone wrong? I am still using this > workers.properties: > > # Define 1 real worker using ajp13 > worker.list=ajp13 > # Set properties for worker1 (ajp13) > worker.ajp13.type=ajp13 > worker.ajp13.host=localhost > worker.ajp13.port=8009 > worker.ajp13.connection_pool_timeout=600 > worker.ajp13.connection_pool_minsize=10 > worker.ajp13.connect_timeout=15000 > worker.ajp13.prepost_timeout=10000 > > > Regards > > Juergen Neuhoff > > -- View this message in context: http://www.nabble.com/Apache-mod_jk-memory-leak--tf3023318.html#a8887144 Sent from the Tomcat - User mailing list archive at Nabble.com. --------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ******************************************************************************** This e-mail and the information it contains may be privileged and/or confidential. It is for the intended addressee(s) only. The unauthorised use, disclosure or copying of this e-mail, or any information it contains, is prohibited. If you are not an intended recipient, please contact the sender and delete the material from your computer. ******************************************************************************** --------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]