Re: [elephant-devel] Optimization

2009-01-13 Thread Elliott Slaughter
Thanks for your comments. Let me try to provide you with a little more information. On Thu, Jan 8, 2009 at 4:38 AM, Ian Eslick wrote: > > On Jan 8, 2009, at 3:22 AM, Elliott Slaughter wrote: > > > Another issue is that the cache-style declaration in persistent > > classes doesn't get inherited by

Re: [elephant-devel] Optimization

2009-01-08 Thread Ian Eslick
By the way, a write through mode is complex because if a transaction is rolled back your index could differ from your memory state so you would then have to keep track if slot writes to undo them on rollback... Are you caching blocks of objects for short term operations or long term ones? S

Re: [elephant-devel] Optimization

2009-01-08 Thread Ian Eslick
Great feedback! Thank you. It's always great to have different use cases. Comments below. On Jan 8, 2009, at 3:22 AM, Elliott Slaughter wrote: > Hi, > > On Mon, Dec 29, 2008 at 2:46 PM, Ian Eslick > wrote: > > I checked the cache-checkout model to elephant-unstable. > > Thanks for your wo

Re: [elephant-devel] Optimization

2009-01-08 Thread Elliott Slaughter
Hi, On Mon, Dec 29, 2008 at 2:46 PM, Ian Eslick wrote: > > I checked the cache-checkout model to elephant-unstable. > Thanks for your work on slot caching. I've been playing around with checkout caching for the last week or so, and a couple of comments and critiques. - On checkout, the cached

Re: [elephant-devel] Optimization

2008-12-29 Thread Ian Eslick
I checked the cache-checkout model to elephant-unstable. Also checked in some new tests. The cache-checkout model is pretty simple, although there are a number of ways to shoot yourself in the foot if you don't use it wisely. I haven't given this a serious review for concurrency issues.

Re: [elephant-devel] Optimization

2008-12-29 Thread Ian Eslick
I ran your tests on my new elephant tree that includes the checkout protocol (which is now available via elephant-unstable) and the local improvements to persistent slot access time. Writes are about 2x slower than reads so a read/write test cycle is also informative. Attached is my annot

Re: [elephant-devel] Optimization

2008-12-29 Thread Ian Eslick
Try out my recent checkin of the checkin/checkout protocol to elephant- unstable. I hope to check in an improvement to persistent-slot access which improves persistent slot access performance by approximately 2x shortly. Regards, Ian On Dec 29, 2008, at 3:15 PM, Elliott Slaughter wrote: > On

Re: [elephant-devel] Optimization

2008-12-29 Thread Elliott Slaughter
On Sun, Dec 28, 2008 at 2:09 PM, Ian Eslick wrote: > I implemented a quick check-in/check-out protocol for persistent objects. Nice. Mind sharing? A standard-object slot allows about 50M updates-sec > A persistent-object slot (inside a txn) is about 60k updates-sec > A cached persistent-objec

Re: [elephant-devel] Optimization

2008-12-28 Thread Ian Eslick
I implemented a quick check-in/check-out protocol for persistent objects. I have to think a little more about concurrency as I'm sure there are some holes. I implemented a simple test loop which increments a local variable by the value in an instance slot. A standard-object slot allows abo

Re: [elephant-devel] Optimization

2008-12-28 Thread Ian Eslick
Sounds like you'd like to perform 100k database ops per second which is fairly significant. What kind of data are you writing? With sufficient BDB cache, the nosync option, and a transaction wrapper I'm surprised it's 10x slower - I'd expect more like 4-5x - which is still too slow for you

Re: [elephant-devel] Optimization

2008-12-27 Thread Elliott Slaughter
There are several replies to my original query, so I will attempt to address all of them here. On Wed, Dec 24, 2008 at 2:31 PM, Ian Eslick wrote: > A couple of quick thoughts on your problem: > > 1) Are you wrapping the critical sections of your code in with- > transaction? This causes all data

Re: [elephant-devel] Optimization

2008-12-27 Thread Alex Mizrahi
ES> Are cached slots usable yet (at least in a single-threaded situation)? When I use ":cached t" in my slot definitions, I get errors. i postmodern backend rather than doing caching on a slot level we're doing it on btree level. that is, get-value and (setf get-value) functions do lookups/writ

Re: [elephant-devel] Optimization

2008-12-24 Thread Ian Eslick
A couple of quick thoughts on your problem: 1) Are you wrapping the critical sections of your code in with- transaction? This causes all database pages you touch to be cached within the body of the transaction. This avoids all 'sync' operations and transaction setup/teardown caused by a rea

Re: [elephant-devel] Optimization

2008-12-23 Thread Leslie P. Polzer
Dear Elliott, On Sun, Dec 21, 2008 at 09:35:06PM -0800, Elliott Slaughter wrote: > I am exploring Elephant/BDB as a backend for a video game I am writing. I > have ignored performance considerations so far, but on some larger maps > (around 100 objects), performance is abominably slow (around 5

Re: [elephant-devel] Optimization

2008-12-22 Thread Elliott Slaughter
This is using the BDB backend. On Mon, Dec 22, 2008 at 6:13 AM, Ian Eslick wrote: > Which backend are you using? > > Sent from my iPhone > > On Dec 22, 2008, at 12:35 AM, "Elliott Slaughter" < > elliottslaugh...@gmail.com > > wrote: > > > Hi, > > > > I am exploring Elephant/BDB as a backend for

Re: [elephant-devel] Optimization

2008-12-22 Thread Ian Eslick
Which backend are you using? Sent from my iPhone On Dec 22, 2008, at 12:35 AM, "Elliott Slaughter" wrote: > Hi, > > I am exploring Elephant/BDB as a backend for a video game I am > writing. I have ignored performance considerations so far, but on > some larger maps (around 100 objects), per

[elephant-devel] Optimization

2008-12-21 Thread Elliott Slaughter
Hi, I am exploring Elephant/BDB as a backend for a video game I am writing. I have ignored performance considerations so far, but on some larger maps (around 100 objects), performance is abominably slow (around 5 fps). The same maps run about 10 times faster (about 50 fps) when using completely in-