Method open-controller (in bdb-controller.lisp) is not getting the register parameter I'm specifying. I'm investigating why.
Yarek On Wed, Mar 11, 2009 at 12:48 PM, Yarek Kowalik <yarek.kowa...@gmail.com>wrote: > Hi Ian, > > I did some digging. Reading in the DB docs: > > *DB_REGISTER* > > - Check to see if recovery needs to be performed before opening the > database environment. (For this check to be accurate, all processes using > the environment must specify DB_REGISTER when opening the environment.) If > recovery needs to be performed for any reason (including the initial use of > the DB_REGISTER flag), and DB_RECOVER is also specified, recovery will be > performed and the open will proceed normally. If recovery needs to be > performed and DB_RECOVER is not specified, > DB_RUNRECOVERY<http://www.oracle.com/technology/documentation/berkeley-db/db/ref/program/errorret.html#DB_RUNRECOVERY>will > be returned. If recovery does not need to be performed, the DB_RECOVER > flag will be ignored. See Architecting Transactional Data Store > > applications<http://www.oracle.com/technology/documentation/berkeley-db/db/ref/transapp/app.html>for > more information. > > I'm getting the DB_RUNRECOVERY error, which is strange since the settings > are :recover T and :register T. So, either I'm not setting the parameters > correctly, elephant is not setting the corresponding DB_* flags correctly, > or BerkleyDB is somehow broken. > > The last one is less likely since my Jan build of elephant worked, and you > indicated that the :recovery was being set to T by default then. > > The first one is potentially true, but digging in whith the SLIME inspector > I can see that the controller's value of SPEC slot set as follows: > > #<CONS {1007C4BCF7}> > -------------------- > A proper list: > 0: :BDB > 1: "/home/yarek/lisp/projects/fashion-origami/data/store/" > 2: :REGISTER > 3: T > 4: :RECOVER > 5: T > > I'm digging into elephant code now to see if I can see the DB_* flags being > set. Let me know if you have any more insight. > > Thanks, > > Yarek > > > > > > On Wed, Mar 11, 2009 at 11:42 AM, Yarek Kowalik > <yarek.kowa...@gmail.com>wrote: > >> - I did a fresh pull from darcs: >> >> darcs get >> http://www.common-lisp.net/project/elephant/darcs/elephant-1.0 >> >> - I rebuilt elephant, and weblocks/elephant >> >> - I set up my weblocks store with recover and register flags both set to >> T: >> >> (defstore *elephant-store* :elephant >> :spec `(:BDB ,(namestring (merge-pathnames (make-pathname :directory >> '(:relative "data/store")) >> (asdf-system-directory :fashion-origami))) >> :register t :recover t)) >> >> - Run db_recovery in the data store directory >> >> - I started the two processes (one serving port 80 the other 443). >> >> - Went to 443 first. All is well. >> >> - Went to 80 next. Webapp application dies (trace below). >> >> *Note:* The trace does not look much different from before. >> >> *Note2:* SLIME died when compiling elephant - got stuck on >> >> gcc -L/usr/local/BerkeleyDB.4.7/lib/ -I/usr/local/BerkeleyDB.4.7/include/ >> -shared -march=x86-64 -fPIC -Wall -g -O2 -g >> /home/yarek/.sbcl/site/elephant-1.0/src/db-bdb/libberkeley-db.c -o >> /home/yarek/.sbcl/site/elephant-1.0/src/db-bdb/libberkeley-db.so -lm >> >> I recompiled from command line and got these warnings: >> >> /home/yarek/.sbcl/site/elephant-1.0/src/db-bdb/libberkeley-db.c: In >> function ‘lisp_compare2’: >> /home/yarek/.sbcl/site/elephant-1.0/src/db-bdb/libberkeley-db.c:1053: >> warning: unused variable ‘i’ >> /home/yarek/.sbcl/site/elephant-1.0/src/db-bdb/libberkeley-db.c: In >> function ‘lisp_compare_key2’: >> /home/yarek/.sbcl/site/elephant-1.0/src/db-bdb/libberkeley-db.c:1147: >> warning: unused variable ‘i’ >> >> >> >> Not sure how to proceed. In order to ensure that I'm doing the right >> steps, can you please confirm: >> >> 1. Are my weblocks store settings correct? >> 2. Did I get the right repository for darcs >> 3. is manual compilation OK for libberkeley-db.so? >> >> Yarek >> >> Here is what I get as debug trace: >> >> Berkeley DB error #-30974: DB_RUNRECOVERY: Fatal error, run database >> recovery >> [Condition of type ELEPHANT:BDB-DB-ERROR] >> >> Restarts: >> 0: [TERMINATE-THREAD] Terminate this thread (#<THREAD >> "hunchentoot-worker-2" RUNNING {1003705211}>) >> >> Backtrace: >> 0: ((LAMBDA (SWANK-BACKEND::DEBUGGER-LOOP-FN)) #<FUNCTION (LAMBDA #) >> {100296C8F9}>) >> 1: (SWANK::DEBUG-IN-EMACS #<ELEPHANT:BDB-DB-ERROR {1003081261}>) >> 2: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) #<FUNCTION >> SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {1003089A39}>) >> 3: (SWANK::CALL-WITH-REDIRECTED-IO #<SWANK::CONNECTION {10036CB5E1}> >> #<CLOSURE (LAMBDA #) {1003089A59}>) >> 4: (SWANK::CALL-WITH-CONNECTION #<SWANK::CONNECTION {10036CB5E1}> >> #<CLOSURE (LAMBDA #) {1003089A39}>) >> 5: (SWANK:INVOKE-SLIME-DEBUGGER #<ELEPHANT:BDB-DB-ERROR {1003081261}>) >> 6: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) #<FUNCTION >> SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {10030899F9}>) >> 7: (INVOKE-DEBUGGER #<ELEPHANT:BDB-DB-ERROR {1003081261}>) >> 8: (INVOKE-DEBUGGER #<ELEPHANT:BDB-DB-ERROR {1003081261}>)[:EXTERNAL] >> 9: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) >> #<unavailable argument> #<unavailable argument> #<ELEPHANT:BDB-DB-ERROR >> {1003081261}>) >> 10: (SIGNAL #<ELEPHANT:BDB-DB-ERROR {1003081261}>)[:EXTERNAL] >> 11: (ERROR #<ELEPHANT:BDB-DB-ERROR {1003081261}>)[:EXTERNAL] >> 12: ((FLET #:LAMBDA43) #<ELEPHANT:BDB-DB-ERROR {1003081261}>) >> 13: ((FLET #:LAMBDA43) #<ELEPHANT:BDB-DB-ERROR {1003081261}>)[:EXTERNAL] >> 14: (SIGNAL #<ELEPHANT:BDB-DB-ERROR {1003081261}>)[:EXTERNAL] >> 15: (ERROR ELEPHANT:BDB-DB-ERROR)[:EXTERNAL] >> Locals: >> SB-DEBUG::ARG-0 = 3 >> SB-DEBUG::ARG-1 = ELEPHANT:BDB-DB-ERROR >> 16: ((SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION >> (DB-BDB::BDB-STORE-CONTROLLER T)) #<unavailable argument> #<unavailable >> argument> #<unavailable argument> #<unavailable argument>)[:EXTERNAL] >> Locals: >> SB-DEBUG::ARG-0 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-1 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-2 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-3 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-4 = :<NOT-AVAILABLE> >> 17: (ELEPHANT::MAP-BTREE-VALUES #<unavailable lambda list>) >> [No Locals] >> 18: (ELEPHANT::GET-DB-SCHEMAS #<unavailable lambda list>) >> [No Locals] >> 19: (ELEPHANT:MAP-CLASS #<unavailable argument> #<unavailable >> argument>)[:EXTERNAL] >> Locals: >> SB-DEBUG::ARG-0 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-1 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-2 = :<NOT-AVAILABLE> >> 20: ((SB-PCL::FAST-METHOD WEBLOCKS:COUNT-PERSISTENT-OBJECTS >> (WEBLOCKS-ELEPHANT:ELEPHANT-STORE T)) ..)[:EXTERNAL] >> Locals: >> SB-DEBUG::ARG-0 = 4 >> SB-DEBUG::ARG-1 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-2 = :<NOT-AVAILABLE> >> SB-DEBUG::ARG-3 = #<WEBLOCKS-ELEPHANT:ELEPHANT-STORE {1002D92011}> >> SB-DEBUG::ARG-4 = FASHION-ORIGAMI::PRODUCT-SET >> >> >> >> On Wed, Mar 11, 2009 at 10:38 AM, Ian Eslick <esl...@media.mit.edu>wrote: >> >>> BDB docs state that :register t :recover t solve the multi-process >>> recover problem. If you have already registered a process, it will inhibit >>> recovery when a second process connects. >>> >>> I have another user with a deserialization error problem. Try updating >>> to the latest elephant-1.0 and you should see an updated >>> deserialization-error report in the backtrace that tells you more about why >>> there was an error. >>> >>> Ian >>> >>> >>> On Mar 11, 2009, at 1:28 PM, Yarek Kowalik wrote: >>> >>> Ok so that did not work. >>>> >>>> I remember that last time I had to remove :recover t from OPEN-STORE >>>> method in weblocks/src/store/elephant/elephant.lisp since that was the >>>> source of problems with opening a store with two processes. I think it >>>> forces the recovery mode at connection startup. >>>> >>>> I wonder if I should try :recover nil :register t -- doing that now. >>>> >>>> Yarek >>>> >>>> On Wed, Mar 11, 2009 at 10:06 AM, Yarek Kowalik < >>>> yarek.kowa...@gmail.com> wrote: >>>> Info: >>>> >>>> - BDB 4.7. >>>> - Ubuntu 8.04 on both 32 bit (Intel and whatever Amazon EC2 is using) >>>> and 64 bit (AMD X2 64) versions. >>>> - got the ELEPHANT-1-0-A2 via darcs >>>> >>>> I'm trying out the :register t and :recover t options now. >>>> >>>> Yarek >>>> >>>> >>>> On Tue, Mar 10, 2009 at 4:20 PM, Ian Eslick <esl...@media.mit.edu> >>>> wrote: >>>> You're using BDB 4.7, right? What machine, os, word-width, etc? >>>> >>>> Add :register t and :recover t to the open-store keyword list. >>>> >>>> You can also try downloading from the elephant-1.0 repo but using: >>>> >>>> darcs get --tag=ELEPHANT-1-0-A2 >>>> http://www.common-lisp.net/project/elephant/darcs/elephant-1.0 >>>> >>>> Ian >>>> >>>> >>>> >>>> >>>> On Mar 10, 2009, at 6:57 PM, Yarek Kowalik wrote: >>>> >>>> > Hi Ian, >>>> > >>>> > Thanks for replying. >>>> > >>>> > The only difference on my end between Jan version and now is the >>>> > version of Elephant. I can switch back to the unstable version and >>>> > the current app works fine. >>>> > >>>> > Re: slots on the controler: all are set to some value - none are >>>> > unbound, but some are (see below). >>>> > >>>> > THe only reason I was upgrading was do to some other erros seen with >>>> > map-inverted-index that returned nils (though there is a way to do a >>>> > cleanup of those). >>>> > >>>> > I hope that Leslie can shed more light. Is there a way to set the >>>> > 'regsiter' flag? >>>> > >>>> > Yarek >>>> > >>>> > >>>> > #<DB-BDB::BDB-STORE-CONTROLLER {10024ADF61}> >>>> > -------------------- >>>> > Class: #<STANDARD-CLASS DB-BDB::BDB-STORE-CONTROLLER> >>>> > -------------------- >>>> > All Slots: >>>> > BTREES = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X006417A0 :TYPE (* T)> [set value] [make unbound] >>>> > CID-SEQ = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X006439A0 :TYPE (* T)> [set value] [make unbound] >>>> > DB = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X00641060 :TYPE (* T)> [set value] [make unbound] >>>> > DB-VERSION = 100 [set value] [make unbound] >>>> > DEADLOCK-DETECT-THREAD = NIL [set value] [make unbound] >>>> > DEADLOCK-PID = NIL [set value] [make unbound] >>>> > DESERIALIZE = ELEPHANT-SERIALIZER2::DESERIALIZE [set >>>> > value] [make unbound] >>>> > DESERIALIZE-FN = #<FUNCTION (SB-C::&OPTIONAL-DISPATCH ..)> >>>> > [set value] [make unbound] >>>> > DUP-BTREES = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X00641EE0 :TYPE (* T)> [set value] [make unbound] >>>> > ENVIRONMENT = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X0063F320 :TYPE (* T)> [set value] [make unbound] >>>> > GC-MARK-LIST = NIL [set value] [make unbound] >>>> > GC-MARK-TABLE = NIL [set value] [make unbound] >>>> > GC-MARKING-P = NIL [set value] [make unbound] >>>> > GC-MAX-OID = NIL [set value] [make unbound] >>>> > INDEX-TABLE = #<BDB-BTREE oid:-2> [set value] [make >>>> > unbound] >>>> > INDICES = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X00642620 :TYPE (* T)> [set value] [make unbound] >>>> > INDICES-ASSOC = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X00642D60 :TYPE (* T)> [set value] [make unbound] >>>> > INSTANCE-CACHE = #<HASH-TABLE :TEST EQL :COUNT 8 >>>> > {100271C671}> [set value] [make unbound] >>>> > INSTANCE-CACHE-LOCK = #S(SB-THREAD:MUTEX :NAME NIL :%OWNER >>>> > NIL :STATE 0) [set value] [make unbound] >>>> > INSTANCE-CLASS-INDEX = #<BDB-BTREE-INDEX oid:1> [set value] [make >>>> > unbound] >>>> > INSTANCE-TABLE = #<BDB-INDEXED-BTREE oid:-3> [set value] >>>> > [make unbound] >>>> > METADATA = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X006409B0 :TYPE (* T)> [set value] [make unbound] >>>> > OID-DB = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X006434E0 :TYPE (* T)> [set value] [make unbound] >>>> > OID-SEQ = #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP >>>> > #X00643B70 :TYPE (* T)> [set value] [make unbound] >>>> > ROOT = #<BDB-BTREE oid:-1> [set value] [make >>>> > unbound] >>>> > SCHEMA-CACHE = #<HASH-TABLE :TEST EQ :COUNT 0 >>>> > {100271C561}> [set value] [make unbound] >>>> > SCHEMA-CACHE-LOCK = #S(SB-THREAD:MUTEX :NAME NIL :%OWNER >>>> > NIL :STATE 0) [set value] [make unbound] >>>> > SCHEMA-CLASSES = NIL [set value] [make unbound] >>>> > SCHEMA-NAME-INDEX = #<BDB-BTREE-INDEX oid:0> [set value] [make >>>> > unbound] >>>> > SCHEMA-TABLE = #<BDB-INDEXED-BTREE oid:-4> [set value] >>>> > [make unbound] >>>> > SERIALIZE = ELEPHANT-SERIALIZER2::SERIALIZE [set value] >>>> > [make unbound] >>>> > SERIALIZE-FN = #<FUNCTION ELEPHANT-SERIALIZER2::SERIALIZE> >>>> > [set value] [make unbound] >>>> > SERIALIZER-VERSION = 2 [set value] [make unbound] >>>> > SPEC = (:BDB "/home/yarek/lisp/projects/zzz/data/ >>>> > store/" :RECOVER NIL) [set value] [make unbound] >>>> > >>>> > >>>> > >>>> > >>>> > On Tue, Mar 10, 2009 at 2:50 PM, Ian Eslick <esl...@media.mit.edu> >>>> > wrote: >>>> > Unfortunately that's not a highly informative backtrace. Did you >>>> > upgrade to the latest, and this caused it, or did something suddenly >>>> > change that caused the January '09 version to work? >>>> > >>>> > Some possible sources of these problems: >>>> > >>>> > 1) Somehow the 'register' flag that helps support multiple processes >>>> > is causing problems; it is no longer set by default I believe. >>>> > (Leslie may know more) >>>> > >>>> > 2) The store-controller is not being opened properly. Are all the >>>> > slots set in the controller after the second process is opened? >>>> > >>>> > Can you be more specific about what changed between January and now? >>>> > >>>> > Ian >>>> > >>>> > On Mar 10, 2009, at 5:31 PM, Yarek Kowalik wrote: >>>> > >>>> > > Hi folks, >>>> > > >>>> > > I have two processes accessing the same BDB. One process manages >>>> > > weblocks requests on port 80, the other on port 443. In the elephant >>>> > > from last January, I was able to start, connect and use BDB from >>>> > > both processes. Now, when I the user is redirected to port 443 and >>>> > > the process tries to retrieve data from the BDB, I get a >>>> > > DB_RUNRECOVERY error (see trace below). This happens when the port >>>> > > 443 porcess connects for the very first time to BDB. >>>> > > >>>> > > I think I have seen this before, and I think it had to do with some >>>> > > default configuration on the controler, some argument that forced >>>> > > the BDB into the recovery mode when process first starts up. >>>> > > >>>> > > Any idea how to resolve this? It's killing my secure connection on >>>> > > my web app - it's urgent. >>>> > > >>>> > > Thanks, >>>> > > >>>> > > Yarek >>>> > > >>>> > > >>>> > > >>>> > > Berkeley DB error #-30974: DB_RUNRECOVERY: Fatal error, run database >>>> > > recovery >>>> > > [Condition of type ELEPHANT:BDB-DB-ERROR] >>>> > > >>>> > > Restarts: >>>> > > 0: [TERMINATE-THREAD] Terminate this thread (#<THREAD "hunchentoot- >>>> > > worker-6" RUNNING {B6CD101}>) >>>> > > >>>> > > Backtrace: >>>> > > 0: ((LAMBDA (SWANK-BACKEND::DEBUGGER-LOOP-FN)) #<FUNCTION (LAMBDA >>>> > > #) {AD4FBA5}>) >>>> > > 1: (SWANK::DEBUG-IN-EMACS #<ELEPHANT:BDB-DB-ERROR {B729BF9}>) >>>> > > 2: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) #<FUNCTION >>>> > > SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {B729FED}>) >>>> > > 3: (SWANK::CALL-WITH-REDIRECTED-IO #<SWANK::CONNECTION {AF23819}> >>>> > > #<CLOSURE (LAMBDA #) {B729FFD}>) >>>> > > 4: (SWANK::CALL-WITH-CONNECTION #<SWANK::CONNECTION {AF23819}> >>>> > > #<CLOSURE (LAMBDA #) {B729FED}>) >>>> > > 5: (SWANK:INVOKE-SLIME-DEBUGGER #<ELEPHANT:BDB-DB-ERROR >>>> > {B729BF9}>) >>>> > > 6: ((LAMBDA (SWANK-BACKEND::HOOK SWANK-BACKEND::FUN)) #<FUNCTION >>>> > > SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {B729FCD}>) >>>> > > 7: (INVOKE-DEBUGGER #<ELEPHANT:BDB-DB-ERROR {B729BF9}>) >>>> > > 8: (INVOKE-DEBUGGER #<ELEPHANT:BDB-DB-ERROR {B729BF9}>)[:EXTERNAL] >>>> > > 9: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) >>>> > > #<unavailable argument> #<unavailable argument> #<ELEPHANT:BDB-DB- >>>> > > ERROR {B729BF9}>) >>>> > > 10: (SIGNAL #<ELEPHANT:BDB-DB-ERROR {B729BF9}>)[:EXTERNAL] >>>> > > 11: (ERROR #<ELEPHANT:BDB-DB-ERROR {B729BF9}>)[:EXTERNAL] >>>> > > 12: ((FLET #:LAMBDA43) #<ELEPHANT:BDB-DB-ERROR {B729BF9}>) >>>> > > 13: ((FLET #:LAMBDA43) #<ELEPHANT:BDB-DB-ERROR {B729BF9}>) >>>> > [:EXTERNAL] >>>> > > 14: (SIGNAL #<ELEPHANT:BDB-DB-ERROR {B729BF9}>)[:EXTERNAL] >>>> > > 15: (ERROR ELEPHANT:BDB-DB-ERROR)[:EXTERNAL] >>>> > > Locals: >>>> > > SB-DEBUG::ARG-0 = 3 >>>> > > SB-DEBUG::ARG-1 = ELEPHANT:BDB-DB-ERROR >>>> > > 16: ((SB-PCL::FAST-METHOD ELEPHANT:GET-VALUE (T DB-BDB::BDB-BTREE)) >>>> > > #<unavailable lambda list>) >>>> > > [No Locals] >>>> > > 17: (ELEPHANT::ENSURE-SLOT-DEF-INDEX #<unavailable argument> >>>> > > #<unavailable argument>) >>>> > > Locals: >>>> > > SB-DEBUG::ARG-0 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-1 = :<NOT-AVAILABLE> >>>> > > 18: ((SB-PCL::FAST-METHOD ELEPHANT:FIND-INVERTED-INDEX >>>> > > (ELEPHANT:PERSISTENT-METACLASS T)) #<unavailable argument> >>>> > > #<unavailable argument> #<unavailable argument> #<unavailable >>>> > > argument>)[:EXTERNAL] >>>> > > Locals: >>>> > > SB-DEBUG::ARG-0 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-1 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-2 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-3 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-4 = :<NOT-AVAILABLE> >>>> > > 19: (ELEPHANT:MAP-INVERTED-INDEX #<unavailable argument> >>>> > > #<unavailable argument> #<unavailable argument>)[:EXTERNAL] >>>> > > Locals: >>>> > > SB-DEBUG::ARG-0 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-1 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-2 = :<NOT-AVAILABLE> >>>> > > SB-DEBUG::ARG-3 = :<NOT-AVAILABLE> >>>> > > >>>> > > >>>> > > >>>> > > _______________________________________________ >>>> > > elephant-devel site list >>>> > > elephant-devel@common-lisp.net >>>> > > http://common-lisp.net/mailman/listinfo/elephant-devel >>>> > >>>> > >>>> > _______________________________________________ >>>> > elephant-devel site list >>>> > elephant-devel@common-lisp.net >>>> > http://common-lisp.net/mailman/listinfo/elephant-devel >>>> > >>>> > _______________________________________________ >>>> > elephant-devel site list >>>> > elephant-devel@common-lisp.net >>>> > http://common-lisp.net/mailman/listinfo/elephant-devel >>>> >>>> >>>> _______________________________________________ >>>> elephant-devel site list >>>> elephant-devel@common-lisp.net >>>> http://common-lisp.net/mailman/listinfo/elephant-devel >>>> >>>> >>>> >>> >> >
_______________________________________________ elephant-devel site list elephant-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel
