Re: wc_db performance (was: wc_db API discussion)

2011-03-17 Thread Paul Burba
On Tue, Mar 15, 2011 at 6:22 PM, Paul Burba wrote: > On Mon, Mar 14, 2011 at 2:36 PM, Hyrum K Wright wrote: >> On Sat, Mar 12, 2011 at 6:47 AM, Stefan Sperling wrote: >>> On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote: 2011/3/11 Branko Čibej : >... > For the second tas

Re: wc_db performance (was: wc_db API discussion)

2011-03-15 Thread Paul Burba
On Mon, Mar 14, 2011 at 2:36 PM, Hyrum K Wright wrote: > On Sat, Mar 12, 2011 at 6:47 AM, Stefan Sperling wrote: >> On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote: >>> 2011/3/11 Branko Čibej : >>> >... >>> > For the second task, I think the first order of business is to change >>> > t

Re: wc_db performance (was: wc_db API discussion)

2011-03-14 Thread Hyrum K Wright
On Sat, Mar 12, 2011 at 6:47 AM, Stefan Sperling wrote: > On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote: >> 2011/3/11 Branko Čibej : >> >... >> > For the second task, I think the first order of business is to change >> > the wc-db tree crawler to do one query instead of zillions, or a

Re: wc_db performance (was: wc_db API discussion)

2011-03-13 Thread Paul Burba
On Sat, Mar 12, 2011 at 7:47 AM, Stefan Sperling wrote: > On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote: >> 2011/3/11 Branko Čibej : >> >... >> > For the second task, I think the first order of business is to change >> > the wc-db tree crawler to do one query instead of zillions, or a

Re: wc_db performance (was: wc_db API discussion)

2011-03-12 Thread Branko Čibej
Thanks, Stefan, this looks very promising. I won't make any promises about grabbing some of that code, given how much time I have on my hands ... but I'm glad that at least the proplist bits can be reused. -- Brane On 12.03.2011 13:47, Stefan Sperling wrote: > On Fri, Mar 11, 2011 at 10:43:46PM -

Re: wc_db performance (was: wc_db API discussion)

2011-03-12 Thread Stefan Sperling
On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote: > 2011/3/11 Branko Čibej : > >... > > For the second task, I think the first order of business is to change > > the wc-db tree crawler to do one query instead of zillions, or at least, > > where several queries are required, to do them all

Re: wc_db performance (was: wc_db API discussion)

2011-03-11 Thread Greg Stein
2011/3/11 Branko Čibej : >... > For the second task, I think the first order of business is to change > the wc-db tree crawler to do one query instead of zillions, or at least, > where several queries are required, to do them all in one transaction. stsp has been working this recently. Killing the

Re: wc_db performance (was: wc_db API discussion)

2011-03-11 Thread Greg Stein
2011/3/11 Branko Čibej : > On 12.03.2011 01:29, Greg Stein wrote: >... >> So. Not a premature optimization, but a design choice. > > Six of one, half a dozen of the other. ACTUAL is just another op-depth > with a few extra attributes, so let's compromise and call it an > implementation choice, rath

Re: wc_db performance (was: wc_db API discussion)

2011-03-11 Thread Branko Čibej
On 12.03.2011 01:11, Mark Phippard wrote: > I am glad you sent this because I was getting ready to send an email > to see if anyone is looking into the suggestions you have made here. > I think we have to get this work done soon. We cannot release with > performance like it is. How do we define t

Re: wc_db performance (was: wc_db API discussion)

2011-03-11 Thread Branko Čibej
On 12.03.2011 01:29, Greg Stein wrote: > 2011/3/11 Branko Čibej : >> This comment is somewhat orthogonal to the API discussions, but as I've >> noted before ... after my relatively brief sojourn in wc-db, I came to >> the conclusion that having separate NODES and ACTUAL_NODE tables is >> going to b

wc_db performance (was: wc_db API discussion)

2011-03-11 Thread Greg Stein
2011/3/11 Branko Čibej : > This comment is somewhat orthogonal to the API discussions, but as I've > noted before ... after my relatively brief sojourn in wc-db, I came to > the conclusion that having separate NODES and ACTUAL_NODE tables is > going to be a perpetual impediment to really speeding u