Re: [HACKERS] bgwriter changes

2004-12-20 Thread Simon Riggs
On Thu, 2004-12-16 at 11:07, Neil Conway wrote: > Zeugswetter Andreas DAZ SD wrote: > > This has the disadvantage of converging against 0 dirty pages. > > A system that has less than maxpages dirty will write every page with > > every bgwriter run. > > Yeah, I'm concerned about the bgwriter being

Re: [HACKERS] bgwriter changes

2004-12-20 Thread Simon Riggs
On Mon, 2004-12-20 at 01:17, Mark Kirkwood wrote: > Mark Kirkwood wrote: > > > It occurs to me that cranking up the number of transactions (say > > 1000->10) and seeing if said regression persists would be > > interesting. This would give the smoothing effect of the bgwriter > > (plus the

Re: [HACKERS] bgwriter changes

2004-12-19 Thread Mark Kirkwood
Mark Kirkwood wrote: It occurs to me that cranking up the number of transactions (say 1000->10) and seeing if said regression persists would be interesting. This would give the smoothing effect of the bgwriter (plus the ARC) a better chance to shine. I ran a few of these over the weekend -

Re: [HACKERS] bgwriter changes

2004-12-16 Thread Neil Conway
Zeugswetter Andreas DAZ SD wrote: This has the disadvantage of converging against 0 dirty pages. A system that has less than maxpages dirty will write every page with every bgwriter run. Yeah, I'm concerned about the bgwriter being overly aggressive if we disable bgwriter_percent. If we leave the

Re: [HACKERS] bgwriter changes

2004-12-16 Thread Zeugswetter Andreas DAZ SD
> > Only if you redefine the meaning of bgwriter_percent. At present it's > > defined by reference to the total number of dirty pages, and that can't > > be known without collecting them all. > > > > If it were, say, a percentage of the total length of the T1/T2 lists, > > then we'd have some ch

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Mark Kirkwood
Simon Riggs wrote: 100pct.patch (SR) Test results to date: 1. Mark Kirkwood ([HACKERS] [Testperf-general] BufferSync and bgwriter) pgbench 1xCPU 1xDisk shared_buffers=1 showed 8.0RC1 had regressed compared with 7.4.6, but patch improved performance significantly against 8.0RC1 It occurs to

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Jan Wieck
On 12/15/2004 12:10 PM, Tom Lane wrote: Jan Wieck <[EMAIL PROTECTED]> writes: Still, we need to avoid scanning over all the clean blocks of a large buffer pool, so there is need for a separate dirty-LRU. That's not happening, unless you want to undo the cntxDirty stuff, with unknown implications f

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > Still, we need to avoid scanning over all the clean blocks of a large > buffer pool, so there is need for a separate dirty-LRU. That's not happening, unless you want to undo the cntxDirty stuff, with unknown implications for performance and deadlock safety.

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Jan Wieck
On 12/14/2004 2:40 PM, Tom Lane wrote: "Zeugswetter Andreas DAZ SD" <[EMAIL PROTECTED]> writes: Is it possible to do a patch that produces a dirty buffer list in LRU order and stops early when eighter maxpages is reached or bgwriter_percent pages are scanned ? Only if you redefine the meaning of bg

Re: RE: Re: [HACKERS] bgwriter changes

2004-12-15 Thread simon
Zeugswetter Andreas DAZ SD <[EMAIL PROTECTED]> wrote on 15.12.2004, 15:33:16: > > > The two alternative algorithms are similar, but have these > > differences: > > The former (option (2)) finds a constant number of dirty pages, though > > has varying search time. > > This has the disadvantage o

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Zeugswetter Andreas DAZ SD
> The two alternative algorithms are similar, but have these > differences: > The former (option (2)) finds a constant number of dirty pages, though > has varying search time. This has the disadvantage of converging against 0 dirty pages. A system that has less than maxpages dirty will write eve

Re: Re: [HACKERS] bgwriter changes

2004-12-15 Thread simon
Zeugswetter Andreas DAZ SD <[EMAIL PROTECTED]> wrote on 15.12.2004, 11:39:44: > > > > > and stops early when eighter maxpages is reached or bgwriter_percent > > > > pages are scanned ? > > > > > > Only if you redefine the meaning of bgwriter_percent. At present it's > > > defined by reference t

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Zeugswetter Andreas DAZ SD
> > > and stops early when eighter maxpages is reached or bgwriter_percent > > > pages are scanned ? > > > > Only if you redefine the meaning of bgwriter_percent. At present it's > > defined by reference to the total number of dirty pages, and that can't > > be known without collecting them all.

Re: [HACKERS] bgwriter changes

2004-12-15 Thread Simon Riggs
On Tue, 2004-12-14 at 13:30, Neil Conway wrote: > In recent discussion[1] with Simon Riggs, there has been some talk of > making some changes to the bgwriter. To summarize the problem, the > bgwriter currently scans the entire T1+T2 buffer lists and returns a > list of all the currently dirty bu

Re: [HACKERS] bgwriter changes

2004-12-14 Thread Neil Conway
On Tue, 2004-12-14 at 09:23 -0500, Tom Lane wrote: > At this point in the release cycle I'm not sure we should be making > any significant changes for anything less than a crashing bug. Yes, that's true, and I am definitely hesitant to make changes during RC. That said, "adjust bgwriter defaults"

Re: [HACKERS] bgwriter changes

2004-12-14 Thread Simon Riggs
On Tue, 2004-12-14 at 19:40, Tom Lane wrote: > "Zeugswetter Andreas DAZ SD" <[EMAIL PROTECTED]> writes: > > Is it possible to do a patch that produces a dirty buffer list in LRU order > > and stops early when eighter maxpages is reached or bgwriter_percent > > pages are scanned ? > > Only if you r

Re: [HACKERS] bgwriter changes

2004-12-14 Thread Tom Lane
"Zeugswetter Andreas DAZ SD" <[EMAIL PROTECTED]> writes: > Is it possible to do a patch that produces a dirty buffer list in LRU order > and stops early when eighter maxpages is reached or bgwriter_percent > pages are scanned ? Only if you redefine the meaning of bgwriter_percent. At present it's

Re: [HACKERS] bgwriter changes

2004-12-14 Thread Zeugswetter Andreas DAZ SD
> (2) Remove bgwriter_percent. I have yet to hear anyone argue that > there's an actual need for bgwriter_percent in tuning > bgwriter behavior, One argument for it is to avoid writing very hot pages. > (3) Change the meaning of bgwriter_percent, per Simon's proposal. Make > it mean "the perce

Re: [HACKERS] bgwriter changes

2004-12-13 Thread Andrew Dunstan
Tom Lane wrote: However, due consideration should also be given to (4) Do nothing until 8.1. At this point in the release cycle I'm not sure we should be making any significant changes for anything less than a crashing bug. If that's not the policy, then I don't understand the dev cycle state

Re: [HACKERS] bgwriter changes

2004-12-13 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > ... > (2) Remove bgwriter_percent. I have yet to hear anyone argue that > there's an actual need for bgwriter_percent in tuning bgwriter behavior, > ... Of the three offered solutions, I agree that that makes the most sense (unless Jan steps up with a st

Re: [HACKERS] bgwriter changes

2004-12-13 Thread Bruce Momjian
Neil Conway wrote: > (2) Remove bgwriter_percent. I have yet to hear anyone argue that > there's an actual need for bgwriter_percent in tuning bgwriter behavior, > and one less GUC var is a good thing, all else being equal. This is > effectively the same as #1 with the default changed, only less

[HACKERS] bgwriter changes

2004-12-13 Thread Neil Conway
In recent discussion[1] with Simon Riggs, there has been some talk of making some changes to the bgwriter. To summarize the problem, the bgwriter currently scans the entire T1+T2 buffer lists and returns a list of all the currently dirty buffers. It then selects a subset of that list (computed