Re: [HACKERS] Hint Bits and Write I/O

2008-07-15 Thread Bruce Momjian
Added to TODO: * Consider decreasing the I/O caused by updating tuple hint bits http://archives.postgresql.org/pgsql-hackers/2008-05/msg00847.php --- Simon Riggs wrote: > After some discussions at PGCon, I'd like to mak

Re: [PATCHES] [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Heikki Linnakangas
Gregory Stark wrote: I'm also a bit concerned that *how many hint bits* isn't enough information to determine how important it is to write out the page. Agreed, that doesn't seem like a very good metric to me either. Or how many *unhinted* xmin/xmax values were found? If HTSV can hint xmin f

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Simon Riggs
On Fri, 2008-06-27 at 15:36 +0100, Gregory Stark wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > > The default and minimum value for this parameter is 1, so very similar to > > existing behaviour. Expected settings would be 2-5, possibly as high as 20, > > though those are just educated gu

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Simon Riggs
On Fri, 2008-06-27 at 15:25 +0100, Gregory Stark wrote: > "Alvaro Herrera" <[EMAIL PROTECTED]> writes: > > > If only VACUUM is going to set "flexible" to off, maybe it's better to > > leave the APIs as they are and have a global that's set by VACUUM only > > (and reset in a PG_CATCH block). > >

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Gregory Stark
"Simon Riggs" <[EMAIL PROTECTED]> writes: > The default and minimum value for this parameter is 1, so very similar to > existing behaviour. Expected settings would be 2-5, possibly as high as 20, > though those are just educated guesses. So the maximum is set arbitrarily as > 100. Not a fan of ar

Re: [HACKERS] Hint Bits and Write I/O

2008-06-27 Thread Gregory Stark
"Alvaro Herrera" <[EMAIL PROTECTED]> writes: > If only VACUUM is going to set "flexible" to off, maybe it's better to > leave the APIs as they are and have a global that's set by VACUUM only > (and reset in a PG_CATCH block). Ugh. Perhaps it would be simpler to have a wrapper function HTSV() macr

Re: [PATCHES] [HACKERS] Hint Bits and Write I/O

2008-06-18 Thread Simon Riggs
On Wed, 2008-06-18 at 14:53 +0100, Simon Riggs wrote: > There is one minor strangeness in the patch, which is the change of > initdb's command order when "vacuuming database template1". With the > previous ordering of ANALYZE; VACUUM FULL; VACUUM; the flexible hint bit > setting of the ANALYZE on

Re: [PATCHES] [HACKERS] Hint Bits and Write I/O

2008-06-18 Thread Alvaro Herrera
Simon Riggs wrote: > When running a VACUUM command we always dirty the block when setting > hint bits, for a number of reasons: > * VACUUM FULL expects all hint bits to be set prior to moving tuples > * Setting all hint bits allows us to truncate the clog > * it forces the VACUUM to write out its

Re: [HACKERS] Hint Bits and Write I/O

2008-06-03 Thread Decibel!
On May 27, 2008, at 2:35 PM, Simon Riggs wrote: After some discussions at PGCon, I'd like to make some proposals for hint bit setting with the aim to reduce write overhead. For those that missed it... http://wiki.postgresql.org/wiki/Hint_Bits (see archive reference at bottom). -- Decibel!,

Re: [HACKERS] Hint Bits and Write I/O

2008-05-30 Thread Florian G. Pflug
Kevin Grittner wrote: On Wed, May 28, 2008 at 6:26 PM, in message <[EMAIL PROTECTED]>, "Florian G. Pflug" <[EMAIL PROTECTED]> wrote: I think we should put some randomness into the decision, to spread the IO caused by hit-bit updates after a batch load. Currently we have a policy of doing

Re: [HACKERS] Hint Bits and Write I/O

2008-05-29 Thread Kevin Grittner
>>> On Wed, May 28, 2008 at 6:26 PM, in message <[EMAIL PROTECTED]>, "Florian G. Pflug" <[EMAIL PROTECTED]> wrote: > I think we should put some randomness into the decision, > to spread the IO caused by hit-bit updates after a batch load. Currently we have a policy of doing a VACUUM FREEZE AN

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Florian G. Pflug
Simon Riggs wrote: Hmm, I think the question is: How many hint bits need to be set before we mark the buffer dirty? (N) Should it be 1, as it is now? Should it be never? Never is a long time. As N increases, clog accesses increase. So it would seem there is likely to be an optimal value for N

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Simon Riggs
On Wed, 2008-05-28 at 06:08 -0400, Gregory Stark wrote: > "Tom Lane" <[EMAIL PROTECTED]> writes: > > > (Although that argument might not hold water for a bulk seqscan: you'll > > have hinted all the tuples and then very possibly throw the page away > > immediately. > > That seems like precisel

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > (Although that argument might not hold water for a bulk seqscan: you'll > have hinted all the tuples and then very possibly throw the page away > immediately. That seems like precisely the case where we don't want to dirty the buffer. > So counting the

Re: [HACKERS] Hint Bits and Write I/O

2008-05-28 Thread Simon Riggs
On Tue, 2008-05-27 at 19:32 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > My proposal is to have this as a two-stage process. When we set the hint > > on a tuple in a clean buffer we mark it BM_DIRTY_HINTONLY, if not > > already dirty. If we set a hint on a buffer that is BM

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > My proposal is to have this as a two-stage process. When we set the hint > on a tuple in a clean buffer we mark it BM_DIRTY_HINTONLY, if not > already dirty. If we set a hint on a buffer that is BM_DIRTY_HINTONLY > then we mark it BM_DIRTY. I wonder if it

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Jeff Davis
On Tue, 2008-05-27 at 20:35 +0100, Simon Riggs wrote: > My proposal is to have this as a two-stage process. When we set the hint > on a tuple in a clean buffer we mark it BM_DIRTY_HINTONLY, if not > already dirty. If we set a hint on a buffer that is BM_DIRTY_HINTONLY > then we mark it BM_DIRTY. I

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Simon Riggs
On Tue, 2008-05-27 at 23:28 +0200, Florian G. Pflug wrote: > Simon Riggs wrote: > > After some discussions at PGCon, I'd like to make some proposals for > > hint bit setting with the aim to reduce write overhead. > > > > Currently, when we see an un-hinted row we set the bit, if possible and > >

Re: [HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Florian G. Pflug
Simon Riggs wrote: After some discussions at PGCon, I'd like to make some proposals for hint bit setting with the aim to reduce write overhead. Currently, when we see an un-hinted row we set the bit, if possible and then dirty the block. If we were to set the bit but *not* dirty the block we ma

[HACKERS] Hint Bits and Write I/O

2008-05-27 Thread Simon Riggs
After some discussions at PGCon, I'd like to make some proposals for hint bit setting with the aim to reduce write overhead. Currently, when we see an un-hinted row we set the bit, if possible and then dirty the block. If we were to set the bit but *not* dirty the block we may be able to find a r