Find attached the modified AJP14 proposal which include
the remarks and request from the list.
Nota :
The launch of Tomcat JVM by the mod_jk/jakarta-connector
is not covered since we speak here only about protocol.
That feature is something asked by many JServ users and
may be added later in the native part, at least in
Apache 1.3/2.0 mode. I'm not sure how it can be done
when using IIS or NES.
The automatic context update was an interesting subject
and how we can have these kind of admin informations raised
more questions than answers.
Opening alternate sockets (control socket) we'll load more
the web-server with more opened sockets :
on forked environnement (ie Apache 1.3), 2 x forked (1 data + 1 admin)
on threaded environnement (ie Apache 2.0), 1 x forked (x threaded data + 1
admin)
A solution could be to use the URGENT DATA mode of TCP/IP socket but I'm not
sure it's supported now in JDK 1.1/1.2/1.3....
Another solution is to delay the automatic update mode.
AJP14 have provision on NEGOCIATION phase to disactivate this feature.
But we still need to know when a context is DOWN and later UP.
* A lazy solution could be to have the servlet engine drop all
AJP connections at each update of context (UP/DOWN).
The web servlet will have then to re-open the connection and
will received the UPDATED context list.
* Another solution will be having the servlet engine sending
a 'Context XXX Down' reply when the user send a request against
a DOWN context. The servlet engine could then route the
request to another engine (in LB mode), or to indicate
the failure (in simple servlet engine mode).
And when a context is marked DOWN, we need to know when it's UP again.
Brute mode :
the web-server continue to forward the requests to the servlet engine,
and if it receive the 'Context XXX Down', it try another route.
Clean mode :
the web-server send a 'Context XXX Status' request, check if the reply is
'Context XXX Up', and if still down, try another route. This mode could be
tuned. ie for a down context, ask for status each 10 or 50 requests.
The context UP/DOWN is really a rare case, and we mustn't spend to much time
in handling this type of event. The protocol must keep its speed.
Another feature I'd like to have in mod_jk/jakarta-connector and present in
mod_jserv
is the backup mode :
- In standard mode, a request is routed to only one servlet-engine (using a
know worker).
- In load balancing mode, a request is sent to a pool of servlet-engine,
using a load
factor.
- We miss a backup mode, where all requests goes allways to the same servlet
engine
until the connection is broken (maybe the remote engine is down or network
link
closed). We define then one (or more) alternate engines to handle
requests.
In that mode, the web-servlet will try to detect later if the principal
servlet
engine is up each N requests (or after M seconds...)
-
Henri Gomez ___[_]____
EMAIL : [EMAIL PROTECTED] (. .)
PGP KEY : 697ECEDD ...oOOo..(_)..oOOo...
PGP Fingerprint : 9DF8 1EA8 ED53 2F39 DC9B 904A 364F 80E6
Proposal for Apache JServ 1.4 - Second Pass
This document is a proposal of evolution of the current
Apache JServ Protocol version 1.3, also known as ajp13.
I'll not cover here the full protocol but only the add-on from ajp13.
This second pass include comments from the tomca-dev list
Missing features in AJP13
-------------------------
ajp13 is a good protocol to link a servlet engine like tomcat to a web server like
Apache:
* use persistants connections to avoid reconnect time at each request
* encode many http commands to reduce stream size
* send to servlet engine many info from web server (like SSL certs)
But ajp13 lacks support for :
* security between web server and servlet engine.
Anybody can connect to an ajp13 port (no login mecanism used)
You could connect, for example with telnet, and keep the remote thread
up by not sending any data (no timeout in connection)
* context information passed from servlet engine to web server.
Part of the configuration of mod_jk, the web server connector, is to
indicate to the web server which URI to handle.
The mod_jk JkMount directive, told to web server which URI must be
forwarded to servlet engine.
A servlet engine allready knows which URI it handle and TC 3.3 is
allready capable to generate a config file for mod_jk from the list
of available contexts.
* state update of contexts from servlet engine to web server.
Big site with farm of Tomcat, like ISP and virtuals hosters,
may need to stop a context for admin purposes. In that case the front
web server must know that the context is currently down, to eventually
relay the request to another Tomcat
* verify state of connection before sending request.
Actually mod_jk send the request to the servlet engine and next wait
for the answer. But one of the beauty of the socket API, is you that
you could write() to a closed connection without any error reporting,
but a read() to a closed connection return you the error code.
AJP14 add-ons to AJP13
----------------------
Let's descrive here the features and add-on that will be added to AJP13,
which will became AJP14. Since this document is a proposal, a resonable level
of chaos must be expected at start.
Be sure that discussion on tomcat list will help clarify points, add
features but the current list seems to be a 'minimun vital'
* Advanced login features at connect time
* Basic authorisation system, where a shared secret key is
present in web server and servlet engine.
* Basic protocol negociation, just to be sure that if functionnalities are added
to AJP14 in the future, current implementations will still works.
* Clean handling of 'Unknown packets'
* Extended env vars passed from web-server to servlet engine.
Advanced login
--------------
1) WEB-SERVER send LOGIN INIT CMD + NEGOCIATION DATA + WEB SERVER INFO
2) TOMCAT respond with LOGIN SEED CMD + RANDOM DATA
3) WEB-SERVER calculted the MD5 of RANDOM DATA+SECRET DATA
4) WEB-SERVER send LOGIN COMP CMD + MD5 (SECRET DATA + RANDOM DATA)
5) TOMCAT respond with LOGIN STATUS CMD + NEGOCIED DATA + SERVLET ENGINE INFO
To prevent DOS attack, the servlet engine will wait
the LOGIN CMD only 15/30 seconds and reports the
timeout exception for admins investigation.
The login command will contains basic protocol
negociation information like compressing ability,
crypto, context info (at start up), context update at
run-time (up/down), level of SSL env vars, AJP protocol
supported (AJP14/AJP15/AJP16...)
The Web server info will contain web server info and
connector name (ie Apache 1.3.19 + mod_ssl 2.8.2 + mod_jk 3.3 + mod_perl 1.25).
The servlet engine will mask the negociation mask with it's own
mask (what it can do) and return it when loggin is accepted.
This will help having a basic ajp14 implementation
on a web-server working with a more advanced ajp14 on
the servlet engine side or vice-versa.
AJP13 was designed to be small and fast and so many
SSL informations present in the web-server are not
forwarded to the servlet engine.
We add here four negociations flags to provide more
informations on client SSL data (certs), server SSL datas
, crypto used, and misc datas (timeout...).
- Messages Stream -
+-------------------------+---------------------------+---------------------------+
| LOGIN INIT CMD (1 byte) | NEGOCIATION DATA (32bits) | WEB SERVER INFO (CString) |
+-------------------------+---------------------------+---------------------------+
+-------------------------+---------------------------+
| LOGIN SEED CMD (1 byte) | MD5 of entropy (32 chars) |
+-------------------------+---------------------------+
+-------------------------+---------------------------------------+
| LOGIN COMP CMD (1 byte) | MD5 of RANDOM + SECRET KEY (32 chars) |
+-------------------------+---------------------------------------+
+--------------------+------------------------+-------------------------------+
| LOGOK CMD (1 byte) | NEGOCIED DATA (32bits) | SERVLET ENGINE INFO (CString) |
+--------------------+------------------------+-------------------------------+
+---------------------+-----------------------+
| LOGNOK CMD (1 byte) | FAILURE CODE (32bits) |
+---------------------+-----------------------+
The secret key will be set by a new JkSecretKey
ie: JkSecretKey myworker1 myverysecurekey
Shutdown feature
----------------
AJP13 miss a functionnality of AJP12, which is shutdown command.
A logout will tell servlet engine to shutdown itself.
+-----------------------+---------------------------------------+
| SHUTDOWN CMD (1 byte) | MD5 of RANDOM + SECRET KEY (32 chars) |
+-----------------------+---------------------------------------+
+---------------------+
| SHUTOK CMD (1 byte) |
+---------------------+
+----------------------+-----------------------+
| SHUTNOK CMD (1 byte) | FAILURE CODE (32bits) |
+----------------------+-----------------------+
Extended Env Vars feature
-------------------------
Many users will want to see some of their web-server env vars
passed to their servlet engine.
To reduce the network traffic, the web-servlet will send a
table to describing the external vars in a shorter fashion.
We'll use there a functionnality allready present in AJP13,
attributes list :
In the AJP13, we've got :
AJP13_FORWARD_REQUEST :=
prefix_code 2
method (byte)
protocol (string)
req_uri (string)
remote_addr (string)
remote_host (string)
server_name (string)
server_port (integer)
is_ssl (boolean)
num_headers (integer)
request_headers *(req_header_name req_header_value)
?context (byte string)
?servlet_path (byte string)
?remote_user (byte string)
?auth_type (byte string)
?query_string (byte string)
?jvm_route (byte string)
?ssl_cert (byte string)
?ssl_cipher (byte string)
?ssl_session (byte string)
?attributes *(attribute_name attribute_value)
request_terminator (byte)
Using short 'web server attribute name' will reduce the
network traffic.
+----------------------------+-------------------------------------+-----------------------------------------+
| EXTENDED VARS CMD (1 byte) | WEB SERVER ATTRIBUTE NAME (CString) | SERVLET ENGINE
|ATTRIBUTE NAME (CString) |...
+----------------------------+-------------------------------------+-----------------------------------------+
ie :
JkExtVars S1 SSL_CLIENT_V_START javax.servlet.request.ssl_start_cert_date
JkExtVars S2 SSL_CLIENT_V_END javax.servlet.request.ssl_end_cert_date
JkExtVars S3 SSL_SESSION_ID javax.servlet.request.ssl_session_id
+-------------------+----+-------------------------------------------+
| EXTENDED VARS CMD | S1 | javax.servlet.request.ssl_start_cert_date |
+-------------------+----+-------------------------------------------+
+----+-----------------------------------------+
| S2 | javax.servlet.request.ssl_end_cert_date |
+----+-----------------------------------------+
+----+-----------------------------------------+
| S3 | javax.servlet.request.ssl_end_cert_date |
+----+-----------------------------------------+
During transmission in AJP14 we'll see attributes name
containing S1, S2, S3 and attributes values of
2001/01/03, 2002/01/03, 0123AFE56.
This example showed the use of extended SSL vars but
any 'personnal' web-server vars like custom authentification
vars could be reused in the servlet engine.
The cost will be only some more bytes in the AJP traffic.
Context informations forwarding for Servlet engine to Web Server
----------------------------------------------------------------
Just after the LOGON PHASE, the web server will receive (if AJP14_CONTEXT_INFO_NEG is
set)
the list of contexts and URLs handled by the servlet engine. It will ease installation
in many sites, reduce questions about configuration on tomcat-user list, and be ready
for servlet API 2.3.
This mode will be activated by a new directive JkAutoMount
ie: JkAutoMount myworker1
A servlet engine could have many contexts, /examples, /admin, /test.
We may want to use only some contexts for a given worker. It was
done previously, in apache HTTP server for example, by setting by
hand the JkMount accordingly in each <virtual> area of Apache.
The new JkAutoMount is adapted for that purpose. We add a third parameter
in JkAutoMount to meet these requirement
ie: JkAutoMount myworker1 www.myvirtualserver.com
In that case the servlet engine will only return the URL/URI matching
these particular virtual server (defined in server.xml).
This feature will help ISP and big sites which mutualize large farm
of Tomcat in load-balancing configuration.
- Messages Stream -
+--------------------------+---------------------------------+
| CONTEXT QRY CMD (1 byte) | VIRTUAL HOST NAME (CString (*)) |
+--------------------------+---------------------------------+
+---------------------------+---------------------------------+-------------------------------+
| CONTEXT INFO CMD (1 byte) | VIRTUAL HOST NAME (CString (*)) | URL1 [\n] URL2 [\n]
|URL3 [\n] |
+---------------------------+---------------------------------+-------------------------------+
*) The CString is a C string, ie an array of chars terminated by a null byte (/0).
En empty string is just a null byte (/0).
*) When VirtualMode is not to be used, the VIRTUAL HOST NAME is an empty string,
the servlet engine will then send the whole context present.
Context informations updates from Servlet engine to Web Server
---------------------------------------------------------------
Context update are messages caming from the servlet engine each time a context
is desactivated/reactivated. The update will be in use when the directive
JkUpdateMount.
This directive will set the AJP14_CONTEXT_UPDATE_NEG flag.
ie: JkUpdateMount myworker1
Also in this mode we may have a third parameter to handle the virtual
server to be used.
ie: JkUpdateMount myworker1 www.myvirtualserver.com
+-----------------------------+---------------------------------+-------------------------+
| CONTEXT UPDATE CMD (1 byte) | VIRTUAL HOST NAME (CString (*)) | STATUS UP/DOWN (1
|byte) |
+-----------------------------+---------------------------------+-------------------------+
*) When VirtualMode is not in use, the VIRTUAL HOST NAME is an empty string.
Context status query to Servlet engine
--------------------------------------
This query will be used by the web-server to determine if a given
context is UP or DOWN.
+----------------------------+----------------------------------+----------------------------+
| CONTEXT STATE CMD (1 byte) | VIRTUAL HOST NAME (CString (*)) | CONTEXT NAME
|(CString (*)) |
+----------------------------+----------------------------------+----------------------------+
+----------------------------------+---------------------------------+----------------------------+------------------+
| CONTEXT STATE REPLY CMD (1 byte) | VIRTUAL HOST NAME (CString (*)) | CONTEXT NAME
|(CString (*)) | UP/DOWN (1 byte) |
+----------------------------------+---------------------------------+----------------------------+------------------+
*) When VirtualMode is not in use, the VIRTUAL HOST NAME is an empty string.
Handling of unknown packets
---------------------------
Sometimes even with a well negocied protocol, we may be in a situation
where one end (web server or servlet engine), will receive a message it
couldn't understand. In that case the receiver will send an
'UNKNOW PACKET CMD' with attached the unhandled message.
+-----------------------------+------------------------------+
| UNKNOWN PACKET CMD (1 byte) | UNHANDLED MESSAGE (bytes...) |
+-----------------------------+------------------------------+
Depending on the message, the sender will report an error and if
possible will try to forward the message to another endpoint.
Verification of connection before sending request
-------------------------------------------------
One of the beauty of socket APIs, is that you could write on a half closed socket.
When servlet engine close the socket, the web server will discover it only at the
next read() to the socket.
Basically, in the AJP13 protocol, the web server send the HTTP HEADER and HTTP BODY
(POST by chunk of 8K) to the servlet engine and then try to receive the reply.
If the connection was broken the web server will learn it only at receive time.
We could use a buffering scheme but what happen when you use the servlet engine
for upload operations with more than 8ko of datas ?
The hack in the AJP13 protocol is to add some bytes to read after the end of the
service :
EXAMPLE OF DISCUSSION BETWEEN WEB SERVER AND SERVLET ENGINE
AJP HTTP-HEADER (+ HTTP-POST) (WEB->SERVLET)
AJP HTTP-REPLY (SERVLET->WEB)
AJP END OF DISCUSSION (SERVLET->WEB)
---> AJP STATUS (SERVLET->WEB AJP14)
The AJP STATUS will not be read by the servlet engine at the end of
the request/response #N but at the begining of the next session.
More at that time the web server could also use OS dependants functions
(or better APR functions) to determine if there is also more data
to read. And that datas could be CONTEXT Updates.
This will avoid the web server sending a request to a
desactivated context. In that case, if the load-balancing is used,
it will search for another servlet engine to handle the request.
And that feature will help ISP and big sites with farm of tomcat,
to updates their servlet engine without any service interruption.
+---------------------+----------------------+
| STATUS CMD (1 byte) | STATUS DATA (1 byte) |
+---------------------+----------------------+
Conclusion
----------
The goal of the AJP14 protocol is to overcome some of the AJP13 limitation.
An easier configuration, a better support for large site and farm of Tomcat,
a simple authentification system and provision for protocol updates.
Using the stable ajp13 implementation in mod_jk (native) and in servlet
engine (java), it's a reasonable evolution of the well known ajp13.
Commands and IDs in AJP14
-------------------------
- Commands IDs -
AJP14_LOGINIT_CMD 0x10
AJP14_LOGSEED_CMD 0x11
AJP14_LOGCOMP_CMD 0x12
AJP14_LOGOK_CMD 0x13
AJP14_LOGNOK_CMD 0x14
AJP14_CONTEXT_QRY_CMD 0x15
AJP14_CONTEXT_INFO_CMD 0x16
AJP14_CONTEXT_UPDATE_CMD 0x17
AJP14_STATUS_CMD 0x18
AJP14_SHUTDOWN_CMD 0x19
AJP14_SHUTOK_CMD 0x1A
AJP14_SHUTNOK_CMD 0x1B
AJP14_CONTEXT_STATE_CMD 0x1C
AJP14_CONTEXT_STATE_REP_CMD 0x1D
AJP14_UNKNOW_PACKET_CMD 0x1E
- Negociations Flags -
AJP14_CONTEXT_INFO_NEG 0x80000000 /* web-server want context info after login
*/
AJP14_CONTEXT_UPDATE_NEG 0x40000000 /* web-server want context updates */
AJP14_GZIP_STREAM_NEG 0x20000000 /* web-server want compressed stream */
AJP14_DES56_STREAM_NEG 0x10000000 /* web-server want crypted DES56 stream
with secret key */
AJP14_SSL_VSERVER_NEG 0x08000000 /* Extended info on server SSL vars */
AJP14_SSL_VCLIENT_NEG 0x04000000 /* Extended info on client SSL vars */
AJP14_SSL_VCRYPTO_NEG 0x02000000 /* Extended info on crypto SSL vars */
AJP14_SSL_VMISC_NEG 0x01000000 /* Extended info on misc SSL vars */
AJP14_PROTO_SUPPORT_AJPXX_NEG 0x00FF0000 /* mask of protocol supported */
AJP14_PROTO_SUPPORT_AJP14_NEG 0x00010000 /* communication could use AJP14 */
AJP14_PROTO_SUPPORT_AJP15_NEG 0x00020000 /* communication could use AJP15 */
AJP14_PROTO_SUPPORT_AJP16_NEG 0x00040000 /* communication could use AJP16 */
...
All others flags must be set to 0 since they are reserved for future use.
- Failure IDs -
AJP14_BAD_KEY_ERR 0xFFFFFFFF
AJP14_ENGINE_DOWN_ERR 0xFFFFFFFE
AJP14_RETRY_LATER_ERR 0xFFFFFFFD
AJP14_SHUT_AUTHOR_FAILED_ERR 0xFFFFFFFC
- Status -
AJP14_CONTEXT_DOWN 0x01
AJP14_CONTEXT_UP 0x02
AJP14_CONTEXT_OK 0x03
