DCM. Since my own app only accesses the database through DCM, this works for me;
I think it would be inconvenient but not impossible if you are using Elephant directly.
DCM is a "director-based" system that is in the contrib directory, that aims more toward
a prevalence based system.
There are/were at least three problems that have to be dealt with:
1) The serializer is definitely not thread-safe. In the next release, we need to improve this,
but for the time being I have added mutexes (these are not checked in yet.)
2) CLSQL is not threadsafe unless you use a separate connection in each thread.
Elephant's basic use of CLSQL reuses the connections, and so you can have
big problems even at the CLSQL level in using Elephant in that way.
So, based on what you have described, I can suggest a few solutions, neither of
which are great:
1) If your access to Elephant is in some relatively narrow place, that is, you can find
a few pieces of code where all of the elephant requests occur, you can put mutexes around
those. (Code to define and use these mutexes is included at the end of this message. It
defines a macro, "defemthodex", which is just like defmethod but takes an extra argument
used to define the mutex. However, this is HIGHLY SBCL specific, and would not
be trivial to adapt to anything other than DCM --- this is (in additio to being busy) why
I have not put the thread-safe version of DCM in the contrib directory yet. It is, however,
an example of how to use mutexes in SBCL, which you can easily copy.)
2) Depending on the demands of your application, you might be able to put in a smaller
number of higher-level mutexes (that is, that operate around larger bodies of code), and
still serve pages sufficiently rapidly.
3) If you use BDB, it will seem better, but the current version of Elephant is still not
thread-safe, because of the serializer, although it will take you a lot longer to hit a
problem than under CLSQL.
I think really Ian and I, and whoever else is interested, need to plan on solving this
problem in the next major release. To my knoledge, this would mean:
1) Writing a small connection-pool manager that is threadsafe in Elephant,
2) Making a reentrant serializer. (I know Ian is planning this already - AMEN!)
3) I'd be happy to publish my current version of DCM to anyone who wants it,
but it is SBCL specific, and I personally don't know how to do mutexes under
any other LISP.
* * *
I know that may not be terribly helpful ---- but let's continue this discussion until
we work out a solution. I am extraordinarily busy launching my business, but am
committed to at least managing patches provided by others or getting out of the way
if someone else (such as Ian) wants to be the official maintainer.
(defvar *dcm-mutexes* (make-hash-table :test 'equal))
(defvar *a-mutex* (sb-thread::make-mutex :name "my lock"))
(defun insure-mutex (name)
(let ((mtx (gethash name *dcm-mutexes*))
)
(or mtx (setf (gethash name *dcm-mutexes*) (sb-thread:make-mutex :name name)))
)
)
;; This assumes that the the variable "dir" is being defined and that we can can
;; create
(defmacro defmethodex (mname dir args &body body)
`(defmethod ,mname ,(cons dir args)
;; (format t "Thread ~A running ~%" sb-thread::*current-thread*)
(sb-thread:with-mutex ((insure-mutex (format nil "mutex-~A" ,(car dir))))
;; (format t "Thread ~A got the lock~%" sb-thread::*current-thread*)
(let ((ret
,@body))
;; (format t "Thread ~A dropping lock~%" sb-thread::*current-thread*)
ret
)
)
)
)
On Tue, 2006-10-31 at 14:43 +0100, Pierre THIERRY wrote:
I may have a problem with concurrency and elephant. I'm developing a web
application using SBCL and Araneida (with it's threaded listener).
Whereas araneida by irself doesn't seem to have problems dealing with
many requests at a time (I tested it with apache's benckmark tool, with
thousands of request, with from 1 to 16 concurrent requests to see how
it reacts), my application just collapses when 3 requests come in a very
short time.
I had an error I did not copied, which was about a persistent slot of an
object being unbound (but I know it should be bound when accessed at
this time), and when trying to access one more time my app, I end up
with this in the debugger:
While accessing database #<POSTGRESQL-DATABASE localhost/efco-immo/pierre OPEN {A80FB99}>
with _expression_ "SELECT value FROM keyvalue WHERE ((clctn_id = 0) and (key = 'FSAAAAAyAAAANAAAACAAAABUAAAASQAAAFQAAABSAAAARQAAAA=='))":
Error NIL /
has occurred.
[Condition of type CLSQL-SYS:SQL-DATABASE-DATA-ERROR]
The database itself doesn't seem corrupted, as restarting the
application from scratch, with the same store, works well. For the
moment, I had to shield my app behind an HTTP cache so that requests are
basicaly served to the user by the cache instead of my app...
Concurrently,
Nowhere man
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
_______________________________________________ elephant-devel site list elephant-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel
