Hi All,
At our company we run JBoss for several applications, and have recently
setup a new box that is giving some rather odd errors.
The box runs:
Windows 2003 Advanced Server
Apache 2.0.54
JBoss 4.0.2
mod_jk 1.2.15 (precompiled binary)
We were previously running jk2, but since that's now sort of deprecated
we've set up this box on jk 1.2.15
Basically, the box is running fine for some time between a couple of
days to a week, and then suddenly we start getting Apache reporting an
error (503 I think, if I remember correctly from last time it happened).
Looking in the mod_jk log file we get these errors constantly:
[Mon Jan 16 09:53:46 2006] [error] jk_ajp_common.c (1026): ERROR: can't
receive the response message from tomcat, network problems or tomcat
(127.0.0.1:8990) is down -53
[Mon Jan 16 09:53:46 2006] [error] jk_ajp_common.c (1528): Tomcat is
down or network problems. Part of the response has already been sent to
the client
[Mon Jan 16 09:53:46 2006] [info] jk_ajp_common.c (1721): Receiving
from tomcat failed, recoverable operation attempt=0
[Mon Jan 16 09:53:46 2006] [info] jk_ajp_common.c (1749): Sending
request to tomcat failed, recoverable operation attempt=1
[Mon Jan 16 09:53:46 2006] [error] jk_ajp_common.c (1026): ERROR: can't
receive the response message from tomcat, network problems or tomcat
(127.0.0.1:8990) is down -53
It tries a couple more times, with similar results, on the second
attempt saying "Tomcat is down or refused connection" instead of "Tomcat
is down or network problems", and on the third attempt saying "Error
connecting to tomcat. Tomcat is probably not started or is listening on
the wrong port".
I checked the source and found that the error code "-53" is derived
from the windows socket error codes - according to MSDN this one is
"Software caused connection abort. An established connection was aborted
by the software in your host computer, possibly due to a data
transmission time-out or protocol error".
JBoss starts having the following message in its logs repeatedly:
2006-01-16 09:56:39,968 WARN [org.apache.jk.common.ChannelSocket]
processCallbacks status 2
It looks like the sockets are getting screwed up and past this point it
seems mod_jk and JBoss can't communicate properly. Unfortunately,
stopping JBoss and Apache and restarting them does no good - it seems
like the sockets are all occupied and hanging around in a bad state. We
have to actually reboot the box to get it up and running again.
Anybody had anything like this, or know what might cause it? We were
wondering if we've got pools of workers / sockets configured badly
(mis-matched).
Tomcat connectors in servers.xml:
(there are several appservers this box connects to, but they all look
like this, just with different ports)
=====================
<Connector port="8080" address="${jboss.bind.address}"
maxThreads="250" strategy="ms" maxHttpHeaderSize="8192"
emptySessionPath="true"
enableLookups="false" redirectPort="8443" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="true"/>
<!-- A AJP 1.3 Connector on port 8009 -->
<Connector port="8009" address="${jboss.bind.address}"
maxThreads="1000" minSpareThreads="30" maxSpareThreads="60"
emptySessionPath="true" enableLookups="false"
redirectPort="8443"
protocol="AJP/1.3"/>
==========================================================
Contents of workers.properties:
====================
worker.list=eOrig,eOrigUAT, jkstatus
# Status worker
worker.jkstatus.type=status
# eOriginations
worker.eOrig.port=8991
# eOriginations UAT
worker.eOrigUAT.host=uklxfcocr
worker.eOrigUAT.port=8991
==========================================================
Contents of httpd.conf:
(stripped out all the standard, unchanged, and basic stuff)
====================================
#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
ServerRoot "Z:/Apache Group/Apache2"
# PidFile: The file in which the server should record its process
# identification number when it starts.
PidFile logs/httpd.pid
# Timeout: The number of seconds before receives and sends time out.
Timeout 300
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
KeepAlive On
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited
amount.
# We recommend you leave this number high, for maximum performance.
MaxKeepAliveRequests 10000
# KeepAliveTimeout: Number of seconds to wait for the next request from
the
# same client on the same connection.
KeepAliveTimeout 15
##
## Server-Pool Size Regulation (MPM specific)
##
# WinNT MPM
# ThreadsPerChild: constant number of worker threads in the server
process
# MaxRequestsPerChild: maximum number of requests a server process
serves
<IfModule mpm_winnt.c>
ThreadsPerChild 500
MaxRequestsPerChild 0
</IfModule>
#
# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, instead of the default. See also the <VirtualHost>
# directive.
Listen 80
#
# Dynamic Shared Object (DSO) Support
#
LoadModule access_module modules/mod_access.so
LoadModule actions_module modules/mod_actions.so
LoadModule alias_module modules/mod_alias.so
LoadModule asis_module modules/mod_asis.so
LoadModule auth_module modules/mod_auth.so
LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule imap_module modules/mod_imap.so
LoadModule include_module modules/mod_include.so
LoadModule isapi_module modules/mod_isapi.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
...
...
### Reverse Proxy Config - added 2005-11-28 for forwarding bug tracker
requests
# disable forward proxy
ProxyRequests Off
# pass /tracker requests to box for purpose
ProxyPass /tracker http://136.140.111.22/tracker
# rewrite headers from backend box to make it appear that request was
serviced by this box
ProxyPassReverse /tracker http://136.140.111.22/tracker
...
...
###
#
# JBoss connector
#
#LoadModule jk2_module modules/mod_jk2.0.49.so
LoadModule jk_module modules/mod_jk-apache-2.0.55.so
JkWorkersFile "Z:/Apache Group/Apache2/conf/workers.properties"
JkLogFile "Z:/apacheLogs/jk.log"
JkLogLevel info
JkMount /eorig/* eOrig
JkMount /uat/eorig/* eOrigUAT
JkMount /jkmanager/* jkstatus
<Location /jkmanager/>
JkMount jkstatus
Order deny,allow
Deny from all
Allow from 136.140.111
</Location>
This mail has been checked at MSXI for all known virus'.
You open this at your own risk. Please make sure all replies are virus free.
Also, we do not accept or send attachments of the type exe, vbs, scr or bat due
to the increased virus risk they contain.
These types of attachments will be stripped from the message.
MSXI
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
