Re: [HACKERS] 8.0.X and the ARC patent

2005-03-05 Thread Greg Stark
Simon Riggs <[EMAIL PROTECTED]> writes: > The checkpoints are the source of the peak latency on transactions, so we > are in complete agreement. I realize that. I was just supporting your conclusion but saying that latency is important in its own right, not just as a means to achieving throughpu

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-05 Thread Simon Riggs
On Fri, 2005-03-04 at 20:10 -0500, Greg Stark wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > > Amdahl's Law tells me that looking at the checkpoints is the next best > > action for tuning, since they add considerably to the average response > > time. Looking at the oprofile for the run as a

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-04 Thread Greg Stark
Simon Riggs <[EMAIL PROTECTED]> writes: > Amdahl's Law tells me that looking at the checkpoints is the next best > action for tuning, since they add considerably to the average response > time. Looking at the oprofile for the run as a whole is missing out the > delayed transaction behaviour that

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-04 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Checkpoint doesn't request new buffers, but it does require the > BufMgrLock in order to write all of the dirty buffers. There is no BufMgrLock anymore in the clock-algorithm code, so I'm not sure how much of this analysis is still relevant.

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-04 Thread Simon Riggs
On Wed, 2005-03-02 at 20:55 -0500, Tom Lane wrote: > Michael Adler <[EMAIL PROTECTED]> writes: > > Looking at the "Response Time Charts" > > > 8.0.1/ARC > > http://www.osdl.org/projects/dbt2dev/results/dev4-010/309/rt.html > > > 20050301 with 2Q patch > > http://www.osdl.org/projects/dbt2dev/res

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-03 Thread Tom Lane
Josh Berkus writes: > I should be able to run more OLTP benchmarks, and a DSS benchmark, within the > next week. Please wait until I complete those before considering an 8.0.2 > release with the new code. Sure, there's no hurry to push out 8.0.2 (and we need to have some beta testing done on

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-03 Thread Greg Stark
Dave Cramer <[EMAIL PROTECTED]> writes: > I was just looking at the config parameters, and you have the shared buffers > set to 60k, and the effective cache set to 1k I was actually going to suggest that the performance degradation might be because of an excessively high shared_buffers setti

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Josh Berkus
Tom, I should be able to run more OLTP benchmarks, and a DSS benchmark, within the next week. Please wait until I complete those before considering an 8.0.2 release with the new code.The machine and setup Mark is testing on is kind of end-of-the-curve; I have some more middle-of-the-road

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Bruce Momjian
Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: > > On Wed, Mar 02, 2005 at 03:15:54PM -0500, Tom Lane wrote: > >> Thanks. This seems odd though, since it appears to level out at > >> something above 4K TPM. Your previous run > >> http://www.osdl.org/projects/dbt2dev/results/dev4-010/311/

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Tom Lane
Michael Adler <[EMAIL PROTECTED]> writes: > Looking at the "Response Time Charts" > 8.0.1/ARC > http://www.osdl.org/projects/dbt2dev/results/dev4-010/309/rt.html > 20050301 with 2Q patch > http://www.osdl.org/projects/dbt2dev/results/dev4-010/313/rt.html > It seems like the average response tim

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > On Wed, Mar 02, 2005 at 03:15:54PM -0500, Tom Lane wrote: >> Thanks. This seems odd though, since it appears to level out at >> something above 4K TPM. Your previous run >> http://www.osdl.org/projects/dbt2dev/results/dev4-010/311/ >> shows it dropping to 3

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
Between those two runs, the vm parameters are the same. Mark On Wed, Mar 02, 2005 at 11:04:21AM -0500, Dave Cramer wrote: > OK. I doubt that it impacts the results of the particular test, but it > is non-intuitive (in my mind at least) > Did you change anything else between 263 and 264? From the

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
On Wed, Mar 02, 2005 at 03:15:54PM -0500, Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: > > Ah, ok. I've reapplied the 2Q patch to CVS from 20050301: > > http://www.osdl.org/projects/dbt2dev/results/dev4-010/313/ > > > I ran it for 3 hours, just in case, and the charts suggest it fl

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Ah, ok. I've reapplied the 2Q patch to CVS from 20050301: > http://www.osdl.org/projects/dbt2dev/results/dev4-010/313/ > I ran it for 3 hours, just in case, and the charts suggest it flattens > out after 2 hours. Thanks. This seems odd though, since

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Michael Adler
On Wed, Mar 02, 2005 at 07:21:41AM -0800, Mark Wong wrote: > Ah, ok. I've reapplied the 2Q patch to CVS from 20050301: > http://www.osdl.org/projects/dbt2dev/results/dev4-010/313/ > > I ran it for 3 hours, just in case, and the charts suggest it flattens > out after 2 hours. Looking at the

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Dave Cramer
OK. I doubt that it impacts the results of the particular test, but it is non-intuitive (in my mind at least) Did you change anything else between 263 and 264? From the table it appears that you are changing vm parameters as well as database configuration parameters between runs ? Dave Mark Wong

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
Yes, those parameters are based on a series of test results here: http://www.osdl.org/projects/dbt2dev/results/pgsql/rc4.html Run 264 provided the best results, so I'm trying to continue with the database parameters used there. Mark On Wed, Mar 02, 2005 at 10:41:57AM -0500, Dave Cramer w

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Dave Cramer
I was just looking at the config parameters, and you have the shared buffers set to 60k, and the effective cache set to 1k Dave Mark Wong wrote: On Tue, Mar 01, 2005 at 05:17:07PM -0500, Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: On Tue, Mar 01, 2005 at 04:57:11PM -0500, T

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-02 Thread Mark Wong
On Tue, Mar 01, 2005 at 05:17:07PM -0500, Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: > > On Tue, Mar 01, 2005 at 04:57:11PM -0500, Tom Lane wrote: > >> Curious. The immediate question is "does it ever flatten out, and > >> if so at what TPM rate compared to 8.0.1?" Could you run the

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-01 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > On Tue, Mar 01, 2005 at 04:57:11PM -0500, Tom Lane wrote: >> Curious. The immediate question is "does it ever flatten out, and >> if so at what TPM rate compared to 8.0.1?" Could you run the same >> test for a longer duration? > The comparison was against

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-01 Thread Mark Wong
On Tue, Mar 01, 2005 at 04:57:11PM -0500, Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: > > About a 6% increase, but if you look at the performance over time, > > it's degrading steadily. The latter throughput peaks near 5000. > > Curious. The immediate question is "does it ever flatte

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-01 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > About a 6% increase, but if you look at the performance over time, > it's degrading steadily. The latter throughput peaks near 5000. Curious. The immediate question is "does it ever flatten out, and if so at what TPM rate compared to 8.0.1?" Could you run

Re: [HACKERS] 8.0.X and the ARC patent

2005-03-01 Thread Mark Wong
On Sat, Feb 19, 2005 at 12:57:52AM -0500, Tom Lane wrote: > So far I've not been able to measure any consistent difference, but you > know how much I trust pgbench ;-). I hope that Mark Wong can give us > some results on the OSDL setup soon. Sorry for the delay, broken laptop, vacation, etc. Her

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Bruce Momjian
Tom Lane wrote: > "Marc G. Fournier" <[EMAIL PROTECTED]> writes: > >> Yea, probably September, but you can't dump a huge feature on us in > >> August either without having talked about it first, so knowing the date > >> might not be that helpful. > > > I thought we were looking at a 12-18 month cy

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Tom Lane
"Matthew T. O'Connor" writes: > One thing to consider while discussing the length of the cycle is what > features are people planning on putting in? The 8.0 cycle had to be > long due to the many huge improvements. I'm not aware of any 8.1 plans > that are that ambitious, so why plan a long c

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Matthew T. O'Connor
Tom Lane wrote: Although we've dropped the idea of letting the ARC problem drive a very short 8.1 cycle, I would still like to see us shoot for a relatively short 8.1 cycle --- less than a year for sure. The main reason is that I think we'll be flushing out performance and feature issues in the Wi

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Tom Lane
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: >> Yea, probably September, but you can't dump a huge feature on us in >> August either without having talked about it first, so knowing the date >> might not be that helpful. > I thought we were looking at a 12-18 month cycle for 8.1? Which would pu

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Bruce Momjian
Marc G. Fournier wrote: > On Fri, 25 Feb 2005, Bruce Momjian wrote: > > > Simon Riggs wrote: > >> On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote: > >>> For development, this means we will _not_ have a shortened, non-initdb > >>> 8.1 release but a regular release cycle with the typical big

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Simon Riggs
On Fri, 2005-02-25 at 00:18 -0500, Bruce Momjian wrote: > Simon Riggs wrote: > > On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote: > > > For development, this means we will _not_ have a shortened, non-initdb > > > 8.1 release but a regular release cycle with the typical big batch of > > > fea

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Simon Riggs
On Fri, 2005-02-25 at 02:47 -0400, Marc G. Fournier wrote: > On Fri, 25 Feb 2005, Bruce Momjian wrote: > > > Simon Riggs wrote: > >> On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote: > >>> For development, this means we will _not_ have a shortened, non-initdb > >>> 8.1 release but a regular

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-25 Thread Andrew Dunstan
Bruce Momjian wrote: Simon Riggs wrote: On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote: For development, this means we will _not_ have a shortened, non-initdb 8.1 release but a regular release cycle with the typical big batch of features. Might we set a rough date for Beta f

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-24 Thread Marc G. Fournier
On Fri, 25 Feb 2005, Bruce Momjian wrote: Simon Riggs wrote: On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote: For development, this means we will _not_ have a shortened, non-initdb 8.1 release but a regular release cycle with the typical big batch of features. Might we set a rough date for B

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-24 Thread Bruce Momjian
Simon Riggs wrote: > On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote: > > For development, this means we will _not_ have a shortened, non-initdb > > 8.1 release but a regular release cycle with the typical big batch of > > features. > > Might we set a rough date for Beta freeze for 8.1 then

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-21 Thread Simon Riggs
On Mon, 2005-02-14 at 18:17 -0500, Bruce Momjian wrote: > For development, this means we will _not_ have a shortened, non-initdb > 8.1 release but a regular release cycle with the typical big batch of > features. Might we set a rough date for Beta freeze for 8.1 then? September 30th 2005 ? I see

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-18 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> Still to be determined: what we lose in extra I/O from the presumably >> less efficient cache management; also what sort of slowdown occurs on >> a single-CPU machine that isn't going to get any benefit from the >> increased amount of lock management. Bu

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-18 Thread Bruce Momjian
Tom Lane wrote: > Still to be determined: what we lose in extra I/O from the presumably > less efficient cache management; also what sort of slowdown occurs on > a single-CPU machine that isn't going to get any benefit from the > increased amount of lock management. But it looks promising. Yea, t

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread Thomas Hallgren
[EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: Might it be possible to contact IBM directly and ask if they will allow usage of the patent for PostgreSQL. They've let 500 patents for open source, maybe they'll give a write off for this as well. There is an advantage beyond just not having to re-

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread Tom Lane
Bruce Momjian writes: > And ARC has locking requirements that will make it very hard to fix our > SMP buffer management problems in 8.1. I am working on a buffer manager rewrite using the BufMgrLock breakup and "clock sweep" management algorithm we've been discussing. At the moment it's passing

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread Bruce Momjian
Tom Lane wrote: > [EMAIL PROTECTED] writes: > > Might it be possible to contact IBM directly and ask if they will allow > > usage of the patent for PostgreSQL. They've let 500 patents for open > > source, maybe they'll give a write off for this as well. > > If there were hard evidence that the ARC

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread Tom Lane
[EMAIL PROTECTED] writes: > Might it be possible to contact IBM directly and ask if they will allow > usage of the patent for PostgreSQL. They've let 500 patents for open > source, maybe they'll give a write off for this as well. If there were hard evidence that the ARC algorithm was far better th

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread Bruce Momjian
Joshua D. Drake wrote: > Bruce Momjian wrote: > > >[EMAIL PROTECTED] wrote: > > > > > >>Might it be possible to contact IBM directly and ask if they will allow > >>usage of the patent for PostgreSQL. They've let 500 patents for open > >>source, maybe they'll give a write off for this as well. >

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread Joshua D. Drake
Bruce Momjian wrote: [EMAIL PROTECTED] wrote: Might it be possible to contact IBM directly and ask if they will allow usage of the patent for PostgreSQL. They've let 500 patents for open source, maybe they'll give a write off for this as well. There is an advantage beyond just not having to re-w

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread pgsql
> [EMAIL PROTECTED] wrote: >> Might it be possible to contact IBM directly and ask if they will allow >> usage of the patent for PostgreSQL. They've let 500 patents for open >> source, maybe they'll give a write off for this as well. >> >> There is an advantage beyond just not having to re-write th

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread Bruce Momjian
[EMAIL PROTECTED] wrote: > Might it be possible to contact IBM directly and ask if they will allow > usage of the patent for PostgreSQL. They've let 500 patents for open > source, maybe they'll give a write off for this as well. > > There is an advantage beyond just not having to re-write the code

Re: [HACKERS] 8.0.X and the ARC patent

2005-02-14 Thread pgsql
Might it be possible to contact IBM directly and ask if they will allow usage of the patent for PostgreSQL. They've let 500 patents for open source, maybe they'll give a write off for this as well. There is an advantage beyond just not having to re-write the code, but it would also be sort of an I