I have started running the bdb backendtests on latest sbcl, amd64 linux.
I also have got a beta snapshot of openmcl-linux-64 (2006-12-31) running
sometimes. I don't know how stable this beta is, but at least it's good
to run several lisps to find bugs.
When running the backend tests for bdb on sbcl, the first failure was
the FIXNUM test. It serializes/deserializes most-positive-fixnum. Since
most-positive-fixnum was larger than 32bit integers, I somehow suspected
64 bit problems. Which was not the main problem. Instead it was related
to the type checking in the serializer. I have attached a solution in a
diff-file.
(A parenthesis: My 64-bit adventures gave some code as a sideffect. I
have added 64 bit integers to memutils.c and refactored/obscured it with
a macro that makes the reader and writer functions. I have changed all
references to int to be explict int32 in memutils.lisp and
bdb-controller. I have a half-finished version of bdb-controller with
macros making the cursor movement methods. I'll keep it in the closet if
it's of any use later on. End of parenthesis)
Now almost all backendtests run.
2 out of 115 total tests failed: INDEXING-WIPE-INDEX, INDEXING-TIMING.
Index-timing fails with this error. This is something I have noted
before with my recent experiments with elephant version 6. Suddently
after deleting instances, elephant would sometimes say that the class
was no longer indexed. I think it showed up with both the bdb and the
clsql backend. Has no one else noticed this? Below is a backtrace, you
can see the drop-instances call which I am suspecting.
About multithreading/testing. I volunteer to add some kind of
stress-test with bourdeaux threads later, but now the outstanding bug is
a blocker.
Speaking of multithreading: since few Lisp:s use native threads, and
writing thread-safe code can be difficult, in my opinion taking
advantage of multi-core processors is most easily done with spreading
processes. what about multiprocessing? I mean parallel Lisp instances,
on the same or different computers, writing and reading to the same
database? Is elephant supposed to work in that kind of environment. I
suppose the clsql backend does, but what about bdb?
That's all for now, it's in the middle of the night for me, got to count
some sheep.
/Henrik Hjelte
Class #<PERSISTENT-METACLASS STRESS-INDEX> is not an indexed class
[Condition of type SIMPLE-ERROR]
Restarts:
0: [ABORT] Return to SLIME's top level.
1: [TERMINATE-THREAD] Terminate this thread (#<THREAD
"worker" {1004E89801}>)
Backtrace:
0: ((SB-PCL::FAST-METHOD FIND-CLASS-INDEX (PERSISTENT-METACLASS))
#<unused argument>
#<unused argument>
#<PERSISTENT-METACLASS STRESS-INDEX>
:SC
NIL
:ERRORP
T)
1: ((LAMBDA ()))
2: ((SB-PCL::FAST-METHOD ELEPHANT::EXECUTE-TRANSACTION
(DB-BDB::BDB-STORE-CONTROLLER #1="#<...>" . #1#))
#<unused argument>
#<unused argument>
#<DB-BDB::BDB-STORE-CONTROLLER {1004E9A2D1}>
#<CLOSURE (LAMBDA #) {10049CE119}>
:TRANSACTION
NIL
:ENVIRONMENT
NIL ..)
3: (DROP-INSTANCES
(#<STRESS-INDEX {10049CBB51}> #<STRESS-INDEX {10049CA331}>
Tue Jan 23 23:14:09 CET 2007 Henrik Hjelte <[EMAIL PROTECTED]>
* Make most-negative and positive fixnum work on 64 bit sbcl
Looks like a bigger change than it is because of indentation change.
Tue Jan 23 12:39:21 CET 2007 Henrik Hjelte <[EMAIL PROTECTED]>
* openmcl complained about importing something that doesn't exist
Tue Jan 23 11:57:22 CET 2007 Henrik Hjelte <[EMAIL PROTECTED]>
* openmcl detected that spec is a list starting with :bdb rather than a
string or pathname
Mon Jan 22 22:23:37 CET 2007 Henrik Hjelte <[EMAIL PROTECTED]>
* two small compiler warnings
diff -rN -u old-elephant/elephant.asd new-elephant/elephant.asd
--- old-elephant/elephant.asd 2007-01-24 00:26:46.318133014 +0100
+++ new-elephant/elephant.asd 2007-01-24 00:26:46.358133668 +0100
@@ -117,6 +117,7 @@
"-lm"))
(defmethod compiler-options ((compiler (eql :msvc)) (c elephant-c-source) &key
input-file output-file)
+ (declare (ignorable c input-file output-file))
(error "MSVC compiler option not supported yet"))
;; LOAD
diff -rN -u old-elephant/src/elephant/controller.lisp
new-elephant/src/elephant/controller.lisp
--- old-elephant/src/elephant/controller.lisp 2007-01-24 00:26:46.314132948
+0100
+++ new-elephant/src/elephant/controller.lisp 2007-01-24 00:26:46.334133276
+0100
@@ -138,7 +138,7 @@
;;
(defclass store-controller ()
- ((spec :type (or pathname string (simple-array character))
+ ((spec :type list
:accessor controller-spec
:initarg :spec
:documentation "Backend create functions should pass in :spec during
make-instance")
diff -rN -u old-elephant/src/elephant/serializer1.lisp
new-elephant/src/elephant/serializer1.lisp
--- old-elephant/src/elephant/serializer1.lisp 2007-01-24 00:26:46.314132948
+0100
+++ new-elephant/src/elephant/serializer1.lisp 2007-01-24 00:26:46.338133341
+0100
@@ -19,7 +19,6 @@
(defpackage :elephant-serializer1
(:use :cl :elephant :elephant-memutil)
(:import-from :elephant
- *resourced-byte-spec*
get-cached-instance
slot-definition-allocation
slot-definition-name
diff -rN -u old-elephant/src/elephant/serializer2.lisp
new-elephant/src/elephant/serializer2.lisp
--- old-elephant/src/elephant/serializer2.lisp 2007-01-24 00:26:46.310132883
+0100
+++ new-elephant/src/elephant/serializer2.lisp 2007-01-24 00:26:46.342133406
+0100
@@ -20,7 +20,6 @@
(:use :cl :elephant :elephant-memutil)
(:import-from :elephant
*circularity-initial-hash-size*
- *resourced-byte-spec*
get-cached-instance
controller-symbol-cache
controller-symbol-id-cache
@@ -77,6 +76,11 @@
(defconstant +struct+ 20)
(defconstant +class+ 21)
+;; Numerical constants
+(defconstant +most-positive-fixnum+ 22)
+(defconstant +most-negative-fixnum+ 23)
+
+
(defconstant +nil+ #x3F)
;; Arrays
@@ -158,9 +162,6 @@
(serialize-symbol frob bs sc))
(string
(serialize-string frob bs))
- ((integer #.most-negative-fixnum #.most-positive-fixnum)
- (buffer-write-byte +fixnum32+ bs)
- (buffer-write-int frob bs))
(null
(buffer-write-byte +nil+ bs))
(persistent
@@ -197,25 +198,31 @@
(declare (dynamic-extent svs))
(%serialize (/ (length svs) 2))
(loop for item in svs
- do (%serialize item)))))))
+ do (%serialize item)))))))
(integer
- (let* ((num (abs frob))
- (word-size (ceiling (/ (integer-length num) 32)))
- (needed (* word-size 4)))
- (declare (type fixnum word-size needed))
- (if (< frob 0)
- (buffer-write-byte +negative-bignum+ bs)
- (buffer-write-byte +positive-bignum+ bs))
- (buffer-write-int needed bs)
- (loop for i fixnum from 0 below word-size
- ;; this ldb is consing on CMUCL!
- ;; there is an OpenMCL function which should work
- ;; and non-cons
- do
- #+(or cmu sbcl)
- (buffer-write-uint (ldb (int-byte-spec i) num) bs) ;;
(%bignum-ref num i) bs)
- #+(or allegro lispworks openmcl)
- (buffer-write-uint (ldb (int-byte-spec i) num) bs))))
+ (cond
+ ((= frob most-positive-fixnum)
+ (buffer-write-byte +most-positive-fixnum+ bs))
+ ((= frob most-negative-fixnum)
+ (buffer-write-byte +most-negative-fixnum+ bs))
+ (t
+ (let* ((num (abs frob))
+ (word-size (ceiling (/ (integer-length num) 32)))
+ (needed (* word-size 4)))
+ (declare (type fixnum word-size needed))
+ (if (< frob 0)
+ (buffer-write-byte +negative-bignum+ bs)
+ (buffer-write-byte +positive-bignum+ bs))
+ (buffer-write-int needed bs)
+ (loop for i fixnum from 0 below word-size
+ ;; this ldb is consing on CMUCL!
+ ;; there is an OpenMCL function which should work
+ ;; and non-cons
+ do
+ #+(or cmu sbcl)
+ (buffer-write-uint (ldb (int-byte-spec i) num) bs) ;;
(%bignum-ref num i) bs)
+ #+(or allegro lispworks openmcl)
+ (buffer-write-uint (ldb (int-byte-spec i) num)
bs))))))
(rational
(buffer-write-byte +rational+ bs)
(%serialize (numerator frob))
@@ -251,10 +258,10 @@
(%serialize (hash-table-rehash-threshold frob))
(%serialize (hash-table-count frob))
(loop for key being the hash-key of frob
- using (hash-value value)
- do
- (%serialize key)
- (%serialize value))))))
+ using (hash-value value)
+ do
+ (%serialize key)
+ (%serialize value))))))
;; (structure-object
;; (buffer-write-byte +struct+ bs)
;; (let ((idp (gethash frob *circularity-hash*)))
@@ -286,17 +293,17 @@
(let ((rank (array-rank frob)))
(buffer-write-int rank bs)
(loop for i fixnum from 0 below rank
- do (buffer-write-int (array-dimension frob i)
- bs)))
+ do (buffer-write-int (array-dimension frob i)
+ bs)))
(when (array-has-fill-pointer-p frob)
(buffer-write-int (fill-pointer frob) bs))
(loop for i fixnum from 0 below (array-total-size frob)
- do
- (%serialize (row-major-aref frob i)))))))
+ do
+ (%serialize (row-major-aref frob i)))))))
)))
- (%serialize frob)
- (release-circularity-hash *circularity-hash*)
- bs)))
+ (%serialize frob)
+ (release-circularity-hash *circularity-hash*)
+ bs)))
(defun slots-and-values (o)
(loop for sd in (compute-slots (class-of o))
@@ -329,6 +336,10 @@
(cond
((= tag +fixnum32+)
(buffer-read-fixnum bs))
+ ((= tag +most-positive-fixnum+)
+ most-positive-fixnum)
+ ((= tag +most-negative-fixnum+)
+ most-negative-fixnum)
((= tag +nil+) nil)
((= tag +utf8-string+)
(deserialize-string :utf8 bs))
@@ -378,7 +389,7 @@
((= tag +hash-table+)
(let* ((id (buffer-read-fixnum bs))
(maybe-hash (lookup-id id)))
- (declare (dynamic-extent id maybe-cons)
+ (declare (dynamic-extent id maybe-hash)
(type fixnum id))
(if maybe-hash maybe-hash
(let ((h (make-hash-table :test (%deserialize bs)
diff -rN -u old-elephant/src/elephant/serializer2-locks.lisp
new-elephant/src/elephant/serializer2-locks.lisp
--- old-elephant/src/elephant/serializer2-locks.lisp 2007-01-24
00:26:46.314132948 +0100
+++ new-elephant/src/elephant/serializer2-locks.lisp 2007-01-24
00:26:46.342133406 +0100
@@ -20,7 +20,8 @@
(:use :cl :elephant :elephant-memutil)
(:import-from :elephant
*circularity-initial-hash-size*
- *resourced-byte-spec*
+ #+(or cmu sbcl allegro)
+ *resourced-byte-spec*
get-cached-instance
slot-definition-allocation
slot-definition-name
diff -rN -u old-elephant/src/memutil/memutil.lisp
new-elephant/src/memutil/memutil.lisp
--- old-elephant/src/memutil/memutil.lisp 2007-01-24 00:26:46.318133014
+0100
+++ new-elephant/src/memutil/memutil.lisp 2007-01-24 00:26:46.346133472
+0100
@@ -82,7 +82,7 @@
(length :int))
:returning :void))
-(eval-when (compile)
+(eval-when (:compile-toplevel)
(declaim
#-elephant-without-optimize (optimize (speed 3) (safety 1) (space 0) (debug
0))
(inline read-int read-uint read-float read-double
_______________________________________________
elephant-devel site list
elephant-devel@common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel
