Bip, thanks for kick-starting this discussion, sorry I've taken a while to look
at it.
>> One thing I haven't figured out is this. Say I replicate a
>> Session to another machine and succefully install it in the manger, if
>> I connect to that machine and tries to resume that session it doesn't
>> seem to relate that session to me instead it creates a new session.
[Craig says:]
>That's probably because of the way sessions work -- the browser sends
>session ids back to the host they came from (for either cookies or URL
>rewriting). I suspect we might have to create some sort of proxy
>front-end (sort of a Local Redirector) that modifies the session cookies
>returned by the back-end Tomcats to be the name of the proxy front-end.
>It would then need to keep track of which host is really hosting this
>session. (In an Apache-connected environment, this would probably be done
>in the connector. For stand-alone, we could create a proxy webapp that
>would run in a front-end Tomcat.)
This is one possibility, but if this technique is used, I'm not sure there's
a real need to distribute the sessions at all - the redirector can simply
send the client to the Tomcat instance which holds the session locally.
The problem I have with this approach is the front-end Tomcat becomes
a single point of failure. A big advantage distributing sessions is that any
Tomcat instance is able to handle requests for any client session, even
in the event of failure of other instances. We should also allow Tomcat's
distribution to work if a router-type solution is used to distribute incoming
requests in a non-sticky manner.
I would make the (controversial, I know) suggestion that the domain
for the cookie be configurable, so that organization foo.com can
set cookies at .foo.com, which would be sent to Tomcat instances
at www1.foo.com, www2.foo.com, etc.
>The front-end could also later be the basis for dynamic load balancing --
>although we need to remember to support the servlet spec restriction that,
>at any specific point in time, a session lives in one and only one
>JVM. In other words, if I have an active request for session 123 being
>executed on host A, I have to direct any other simultaneous requests for
>session 123 to host A as well.
In order to cope with load balancing mechanisms which aren't aware
of these issues, we could implement locking - flag when a session is
in use by a particular instance - and do "something" if a request comes
in on another instance which doesn't hold the lock -send an error, redirect,
or just wait until the lock frees (ugh).
[Bip says:]
>>I started looking at how to implement a DistributedManager and as I see it the
>>best way to do this is to use MulticastSocket.
There have been a variety of suggestions for how to implement, including
JavaSpaces, JMS, and JCache. Also JNDI and JDBC and the ugly old file
system (if a networked file system is used) could work. MulticastSocket
is a good idea I hadn't thought of - from looking at your code it's obviously
workable and pretty straightforward. But I think we ought to make the
mechanism pluggable the same way Store is pluggable for PersistentManager.
The DistributedManager should manage the manipulation of sessions for
the local instance, for instance overriding getSession() to make a call
to the ClusterStore if a) the application is marked as distributable, and
b) the Store is an instance of ClusterStore.
We also need to answer the question of the request life cycle: the
DistributedManager needs to know when a request begins and ends.
At the beginning, it must lock the session to prevent other Catalina
instances from using it in requests. This can probably just be done
in Manager.findSession(). At the end, it must tell the ClusterStore to
update the session to other members of the Cluster, and unlock it.
So what would the ClusterStore interface look like? The following
are some methods it may need:
/** Returns an up-to-date copy of the Session for the given ID,
and locks the session. Throws IllegalStateException if the
session is in use by another instance. */
public Session findSession(String id) throws IllegalStateException;
/** Adds or updates the session in the cluster, releasing the lock if
applicable. */
public void sendSession(Session session);
/** Receive a new or updated session from another instance of the
cluster. May be called by another method of the same class, but
is useful to have in the interface - a ClusterStoreBase will probably
have the canonical implementation of this.
*/
public void receiveSession(Session session);
/** Gets a unique identifier for this Context/JVM instance, used
by the locking mechanism to indicate the owner of the lock.
*/
public int getInstanceID();
/** Sets a unique identifier for this Context/JVM instance.
*/
public void setInstanceID(int id);
public Manager getManager();
public void setManager(Manager manager);
No doubt there will need to be more to it than this. Any comments?
I'll code it up and commit it in a day or two if nobody has major
problems with the concept.
A MulticastStoreCluster (or whatever) should be pretty straightforward
to do with Bip's code. Some comments (intended as a "to think about/do",
rather than as a criticism of Bip's code, which is meant as a starting point
for discussion)
- The findSession() would probably just return null, since receiveSession()
would update the Manager's session list for it.
- Rather than publishing the session after creation, it should be
published at the end of each request, so any changes will be shared.
This will take a bit of digging into request handling.
- Rather than receiving updates when the normal background thread
gets around to it periodically, this Store should probably have a
separate thread which handles incoming session updates from
other instances. Otherwise a request may come in before the
session has been absorbed from another instance.
Regards,
Kief
