Re: [HACKERS] Hint Bits and Write I/O

2008-07-15 Thread Bruce Momjian
Added to TODO: * Consider decreasing the I/O caused by updating tuple hint bits http://archives.postgresql.org/pgsql-hackers/2008-05/msg00847.php --- Simon Riggs wrote: > After some discussions at PGCon, I'd like to mak

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Simon Riggs
On Fri, 2008-06-27 at 15:36 +0100, Gregory Stark wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > > The default and minimum value for this parameter is 1, so very similar to > > existing behaviour. Expected settings would be 2-5, possibly as high as 20, > > though those are just educated gu

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Simon Riggs
On Fri, 2008-06-27 at 15:25 +0100, Gregory Stark wrote: > "Alvaro Herrera" <[EMAIL PROTECTED]> writes: > > > If only VACUUM is going to set "flexible" to off, maybe it's better to > > leave the APIs as they are and have a global that's set by VACUUM only > > (and reset in a PG_CATCH block). > >

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Gregory Stark
"Simon Riggs" <[EMAIL PROTECTED]> writes: > The default and minimum value for this parameter is 1, so very similar to > existing behaviour. Expected settings would be 2-5, possibly as high as 20, > though those are just educated guesses. So the maximum is set arbitrarily as > 100. Not a fan of ar

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Gregory Stark
"Alvaro Herrera" <[EMAIL PROTECTED]> writes: > If only VACUUM is going to set "flexible" to off, maybe it's better to > leave the APIs as they are and have a global that's set by VACUUM only > (and reset in a PG_CATCH block). Ugh. Perhaps it would be simpler to have a wrapper function HTSV() macr

Re: [HACKERS] Hint Bits and Write I/O

2008-06-03 Thread Decibel!
On May 27, 2008, at 2:35 PM, Simon Riggs wrote: After some discussions at PGCon, I'd like to make some proposals for hint bit setting with the aim to reduce write overhead. For those that missed it... http://wiki.postgresql.org/wiki/Hint_Bits (see archive reference at bottom). -- Decibel!,

Re: [HACKERS] Hint Bits and Write I/O

2008-05-30 Thread Florian G. Pflug
Kevin Grittner wrote: On Wed, May 28, 2008 at 6:26 PM, in message <[EMAIL PROTECTED]>, "Florian G. Pflug" <[EMAIL PROTECTED]> wrote: I think we should put some randomness into the decision, to spread the IO caused by hit-bit updates after a batch load. Currently we have a policy of doing

Re: [HACKERS] Hint Bits and Write I/O

2008-05-29 Thread Kevin Grittner
>>> On Wed, May 28, 2008 at 6:26 PM, in message <[EMAIL PROTECTED]>, "Florian G. Pflug" <[EMAIL PROTECTED]> wrote: > I think we should put some randomness into the decision, > to spread the IO caused by hit-bit updates after a batch load. Currently we have a policy of doing a VACUUM FREEZE AN

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Florian G. Pflug
Simon Riggs wrote: Hmm, I think the question is: How many hint bits need to be set before we mark the buffer dirty? (N) Should it be 1, as it is now? Should it be never? Never is a long time. As N increases, clog accesses increase. So it would seem there is likely to be an optimal value for N

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Simon Riggs
On Wed, 2008-05-28 at 06:08 -0400, Gregory Stark wrote: > "Tom Lane" <[EMAIL PROTECTED]> writes: > > > (Although that argument might not hold water for a bulk seqscan: you'll > > have hinted all the tuples and then very possibly throw the page away > > immediately. > > That seems like precisel

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > (Although that argument might not hold water for a bulk seqscan: you'll > have hinted all the tuples and then very possibly throw the page away > immediately. That seems like precisely the case where we don't want to dirty the buffer. > So counting the

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Simon Riggs
On Tue, 2008-05-27 at 19:32 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > My proposal is to have this as a two-stage process. When we set the hint > > on a tuple in a clean buffer we mark it BM_DIRTY_HINTONLY, if not > > already dirty. If we set a hint on a buffer that is BM

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > My proposal is to have this as a two-stage process. When we set the hint > on a tuple in a clean buffer we mark it BM_DIRTY_HINTONLY, if not > already dirty. If we set a hint on a buffer that is BM_DIRTY_HINTONLY > then we mark it BM_DIRTY. I wonder if it

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Jeff Davis
On Tue, 2008-05-27 at 20:35 +0100, Simon Riggs wrote: > My proposal is to have this as a two-stage process. When we set the hint > on a tuple in a clean buffer we mark it BM_DIRTY_HINTONLY, if not > already dirty. If we set a hint on a buffer that is BM_DIRTY_HINTONLY > then we mark it BM_DIRTY. I

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Simon Riggs
On Tue, 2008-05-27 at 23:28 +0200, Florian G. Pflug wrote: > Simon Riggs wrote: > > After some discussions at PGCon, I'd like to make some proposals for > > hint bit setting with the aim to reduce write overhead. > > > > Currently, when we see an un-hinted row we set the bit, if possible and > >

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Florian G. Pflug
Simon Riggs wrote: After some discussions at PGCon, I'd like to make some proposals for hint bit setting with the aim to reduce write overhead. Currently, when we see an un-hinted row we set the bit, if possible and then dirty the block. If we were to set the bit but *not* dirty the block we ma