Ok I'm going to share some ideas I've been mulling on this topic
since it's been coming up a lot lately. Other people seem to be
interested in having Tomcat natively do load balancing itself, by
redirecting requests between instances in a server farm. What
I'm actually more interested in getting Tomcat to work with an
external load balancing mechanism, sitting in front of a farm of
Tomcat instances.
What I'm thinking about is a session distribution system which
meets the following design objectives:
DESIGN OBJECTIVES
- Compliant with servlet 2.3 specification
- Implement on Catalina (Tomcat 4.0)
- Work with different request distribution mechanisms
- Allow the implementation of different session sharing schemes
It should also be noted that the objective of using a distribution
scheme (a Tomcat farm) is one or both of improving performance
by distributing load, and improving fault tolerance by redundancy.
ASSUMPTIONS ABOUT REQUEST DISTRIBUTION MECHANISM:
Working with different request distribution mechanisms means
that some arbitrary system is used to distribute requests between
different instances of Tomcat running on different JVMs, and potentially
different systems. This may be a connector in the HTTP server, or it
may be a router which is directing requests to Tomcat's HTTP port,
or it may be some service built into Tomcat which forces redirects
between instances. The distributed session mechanism should
support any and all of these without being aware of what it is or
how it works.
The request distributor will not necessarily distribute all requests
from a particular client to a particular Tomcat instance. If it does so
consistently, session distribution isn't really needed, the standard
session handling behavior of Catalina, or any servlet container, is
sufficient. However, some request distributors will direct each request
to an arbitrary (from our point of view) instance of Tomcat, and even
those which attempt to consistently direct a particular client to the
same instance may need to switch it to a different instance if the
first one becomes unavailable.
Therefore, the distributed session mechanism should assume that
different requests in a session will be directed to arbitrary instances
of Tomcat. Any instance of Tomcat should be able to handle a
request itself, whether or not a session object has been created
on the same or a different instance.
ASSUMPTIONS ABOUT SESSION DISTRIBUTION
I'm proposing a modification to Catalina to enable the implementation
of different session distribution mechanisms. These session distributors
may work in radically different ways. The main situations they will have
to deal with are the following use cases:
1) New Session: Application requests a session, no session has been
created for the current client on any Catalina instance in the farm.
2) Unknown Session: Application requests a session, a session has
been created for the current client on a Catalina instance other than
the one handling the current request.
3) Fresh Known Session: Application requests a session, a session
has been created for the current client, and the current Catalina
instance handled the previous request.
4) Stale Known Session: Application requests a session, a session
has been created and handled by the current Catalina instance,
but another Catalina instance has handled a request from the
client in the meantime.
5) Session Invalidated: Application code calls the HttpSession.invalidate()
method
6) Session Times Out: The inactivity timeout period has elapsed without
a request from the client to any Catalina instance.
What others are there? I haven't though fully on the consequences of
calling various methods of HttpSession, such as setMaxInactiveInterval().
SESSION DISTRIBUTION IMPLEMENTATIONS
Catalina should allow a variety of implementations of session distribution.
This allows the implementation of different mechanisms to suit different
user requirements, and also makes it easy to experiment with new ideas.
I've come up with some examples of how session distribution might be
implemented, to get an idea of what Catalina's session distribution
interface should be able to support:
- When a session is created or updated it is stored to a central repository -
e.g. a shared file system or database. When an existing session is requested,
it is retrieved from this repository.
- When a session is created or updated, it is replicated to all other Catalina
instances.
- When a session is created or updated, it is replicated to a selected "partner"
Catalina instance.
- When a session is requested, the Catalina instance obtains the session
data from whichever instance has the most current copy.
SESSION LOCKING
The servlet 2.3 specification says that only one instance of the servlet
container may handle a request from a particular client at a time. This
makes implementing a session distributor much easier, since it can
assume it has a monoply on the session while it's being used by an
application. Questions this raises for me include:
- What should happen if a request comes in to server B while a
request for the same session is concurrently being handled by server A?
- Should it be possible for application code to access (and even modify?)
session data outside of handling a request, e.g. for diagnostic purposes?
What requirements does this impose to ensure the integrity of the data?
DETECTING SESSION MODIFICATIONS
In short, this can't be done. While it's possible to monitor changes in an
HttpSession object's state by changing the implementation. It's also
possible to detect when attributes are added, removed, and replaced
using an HttpSessionAttributesListener. However, there is no efficient
mechanism for the session distribution to detect when data within an
arbitrary attribute object is modified. Therefore the general session
distribution scheme should always assume a session has been
modified after handling a request in which the session was requested
by application code.
CATALINA SUPPORT FOR SESSION REPLICATION
The following are changes I would suggest need to be made to Catalina
to support session replication:
DistributedSessionManager interface:
Define an interface for a Manager class to manage distributed sessions.
Implementors of session distribution will implement this interface, and all
operations within Catalina which may affect session distribution will make
appropriate calls to the Manager implementation.
This is going to be closely tied in with the existing Manager interface.
Maybe it will be a replacement for the StandardManager class. Quite
likely an abstract DistributedSessionManagerBase class, or something
similar, can be provided in Catalina to cover functionality which will be
common to all implementations.
Catalina Session handling:
Various places in Catalina's code relating to the creation, storage, and
retrieval of sessions will need to be modified. They will check whether
the current application is distributable, and if so they will obtain a reference
to the DistributedSessionManager and make appropriate calls.
Session ID:
It may be necessary to provide more control over the session ID. I haven't looked
closely enough to see if Catalina currently provides user-configurable domain
names for the Cookie ID, but this may be necessary to ensure that session
cookies are shared across servers (e.g. www1.me.com, www2.me.com).
Session distribution mechanisms may also want to use the session ID cookie
value to indicate important information, such as which Catalina instance created
the session. So the Manager should be given the option to override the session
ID value at creation.
NEXT STEPS
What do people think of this, both from a big picture perspective (is this the
right approach), and from a detailed perspective (what am I missing)? I can
whip up some more details of what the Manager class should look like, and
where it will integrate with Catalina.
However I'm not sure how much time I'll have to devote to implementing it,
especially to create sample distribution implementations which will be crucial
to making sure it really works, since I'm between gigs.
Kief
