Re: [elephant-devel] working with many millions of objects

2007-02-15 Thread Ian Eslick
Some experiments performed on an old topic... On Oct 11, 2006, at 8:57 PM, Red Daly wrote: I was importing into sleepycat using standard elephant routines. I am not aware of an 'import mode' for sleepycat, but I will look into that when I have a chance. Another consideration using sleepy

Re: [elephant-devel] working with many millions of objects

2006-10-11 Thread Ian Eslick
I don't think that performance of the base Elephant is good enough to justify an every-object-is-persistent model. It's just not seamless enough or efficient enough yet. DCM takes a good step in the performance direction for most applications (single lisp images) and I'm hoping that some of the

Re: [elephant-devel] working with many millions of objects

2006-10-11 Thread Ian Eslick
Can you tell me a little bit about what the import operations look like - that is to say how many objects are created per-transaction, how many slots per created object, etc? Things are only cached in-memory during a transaction. To ensure ACID properties (unless you've turned off synchronization

Re: [elephant-devel] working with many millions of objects

2006-10-11 Thread Robert L. Read
On Wed, 2006-10-11 at 17:57 -0700, Red Daly wrote: I was importing into sleepycat using standard elephant routines. I am not aware of an 'import mode' for sleepycat, but I will look into that when I have a chance. I do not think there is anyway to import things directly and to deal with

Re: [elephant-devel] working with many millions of objects

2006-10-11 Thread Red Daly
I was importing into sleepycat using standard elephant routines. I am not aware of an 'import mode' for sleepycat, but I will look into that when I have a chance. Another consideration using sleepycat is that using BTrees with a large working set demands large amounts of memory relative to a

Re: [elephant-devel] working with many millions of objects

2006-10-10 Thread Robert L. Read
Yes, it's amusing. In my own work I use the Postgres backend; I know very little about SleepyCat.  It seems to me that this is more of a SleepyCat issue, then an Elephant issue.  Perhaps you should ask the SleepyCat list? Are you importing things into SleepyCat directly in the correct seriali