Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Pavan Deolasee
On Thu, Jul 21, 2011 at 4:01 PM, Robert Haas wrote: > > > I think we are better off doing only equality comparisons and dodging > this problem altogether. > > Fair enough. > > Just-plain-dead line pointers would have lp_off = 0. Dead-vacuumed > line pointers would have lp_off != 0. The first

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Robert Haas
On Thu, Jul 21, 2011 at 12:51 PM, Pavan Deolasee wrote: > The way we force the subsequent vacuum to also look at the pages scanned and > pruned by previous failed vacuum, all the pages that have dead-vacuum line > pointers would have a new stamp once the vacuum finishes successfully and > the pg_c

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Pavan Deolasee
On Thu, Jul 21, 2011 at 12:17 PM, Robert Haas wrote: > On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas > > I think you can sidestep that > > if you check that the page's vacuum LSN <= vacuum LSN in pg_class, > instead > > of equality. > > I don't think that works, because the point of storin

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Pavan Deolasee
On Thu, Jul 21, 2011 at 11:51 AM, Robert Haas wrote: > On Tue, Jul 12, 2011 at 4:47 PM, Pavan Deolasee > wrote: > > Comments ? > > I was going to spend some time reviewing this, but I see that (1) it > has bit-rotted slightly - there is a failing hunk in pg_class.h and > (2) some of the comments

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Robert Haas
On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas wrote: > How does this interact with the visibility map? If you set the visibility > map bit after vacuuming indexes, a subsequent vacuum will not visit the > page. The second vacuum will update relindxvacxlogid/off, but it will not > clean up t

Re: [HACKERS] Single pass vacuum - take 1

2011-07-21 Thread Robert Haas
On Tue, Jul 12, 2011 at 4:47 PM, Pavan Deolasee wrote: > Comments ? I was going to spend some time reviewing this, but I see that (1) it has bit-rotted slightly - there is a failing hunk in pg_class.h and (2) some of the comments downthread seem to suggest that you're thinking about whether to re

Re: [HACKERS] Single pass vacuum - take 1

2011-07-19 Thread Alvaro Herrera
Excerpts from Pavan Deolasee's message of lun jul 18 14:50:03 -0400 2011: > On Mon, Jul 18, 2011 at 3:14 AM, Simon Riggs wrote: > > I will be happy to remove it again when we have shown there are no > > bugs getting this wrong is a data loss issue. > > Though I understand the fear for data lo

Re: [HACKERS] Single pass vacuum - take 1

2011-07-18 Thread Pavan Deolasee
On Mon, Jul 18, 2011 at 3:14 AM, Simon Riggs wrote: > On Mon, Jul 18, 2011 at 2:20 AM, Robert Haas > wrote: > > On Fri, Jul 15, 2011 at 7:23 AM, Simon Riggs > wrote: > >> My additional requests would be that we can easily tell which blocks > >> have been modified like this, that we have a way t

Re: [HACKERS] Single pass vacuum - take 1

2011-07-18 Thread Simon Riggs
On Mon, Jul 18, 2011 at 2:20 AM, Robert Haas wrote: > On Fri, Jul 15, 2011 at 7:23 AM, Simon Riggs wrote: >> My additional requests would be that we can easily tell which blocks >> have been modified like this, that we have a way to turn this off if >> we get bugs for next few releases, that we c

Re: [HACKERS] Single pass vacuum - take 1

2011-07-17 Thread Robert Haas
On Fri, Jul 15, 2011 at 7:23 AM, Simon Riggs wrote: > My additional requests would be that we can easily tell which blocks > have been modified like this, that we have a way to turn this off if > we get bugs for next few releases, that we check it all works with Hot > Standby just fine and that al

Re: [HACKERS] Single pass vacuum - take 1

2011-07-15 Thread Simon Riggs
On Fri, Jul 15, 2011 at 12:56 AM, Pavan Deolasee wrote: > On Thu, Jul 14, 2011 at 6:22 PM, Simon Riggs wrote: >> This is a very rare issue, because of all the work yourself and Heikki >> have put in. >> > > I don't think its rare case since vacuum on any table, small or large, would > take two p

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Pavan Deolasee
On Thu, Jul 14, 2011 at 6:22 PM, Simon Riggs wrote: > On Thu, Jul 14, 2011 at 4:57 PM, Pavan Deolasee > wrote: > > > Thanks Simon for looking at the patch. > > Sorry, I didn't notice there was a patch attached. Not reviewed it. I > thought we were still just talking. > > No problem. Please revie

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Simon Riggs
On Thu, Jul 14, 2011 at 4:57 PM, Pavan Deolasee wrote: > Thanks Simon for looking at the patch. Sorry, I didn't notice there was a patch attached. Not reviewed it. I thought we were still just talking. > I am not sure if the use case is really narrow. This is a very rare issue, because of all

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Alvaro Herrera
Excerpts from Pavan Deolasee's message of jue jul 14 13:54:36 -0400 2011: > On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas < > heikki.linnakan...@enterprisedb.com> wrote: > > Seems odd to store relindxvacxlogid/off as two int32 columns. Store it in > > one uint64 column, or invent a new data

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Pavan Deolasee
On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 14.07.2011 18:57, Pavan Deolasee wrote: > >> On Thu, Jul 14, 2011 at 11:46 AM, Simon Riggs >> wrote: >> >>> I'd say that seems way too complex for such a small use case and we've >>> >>> only j

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Heikki Linnakangas
On 14.07.2011 18:57, Pavan Deolasee wrote: On Thu, Jul 14, 2011 at 11:46 AM, Simon Riggs wrote: I'd say that seems way too complex for such a small use case and we've only just fixed the bugs from 8.4 vacuum map complexity. The code's looking very robust now and I'm uneasy that such changes are

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Simon Riggs
On Tue, Jul 12, 2011 at 9:47 PM, Pavan Deolasee wrote: > http://archives.postgresql.org/pgsql-hackers/2011-05/msg01119.php > PFA a patch which implements the idea with some variation. > At the start of the first pass, we remember the current LSN. Every page that > needs some work is HOT-pruned so

Re: [HACKERS] Single pass vacuum - take 1

2011-07-14 Thread Pavan Deolasee
On Thu, Jul 14, 2011 at 11:46 AM, Simon Riggs wrote: > > > Hi Pavan, > > I'd say that seems way too complex for such a small use case and we've > only just fixed the bugs from 8.4 vacuum map complexity. The code's > looking very robust now and I'm uneasy that such changes are really > worth it. >

[HACKERS] Single pass vacuum - take 1

2011-07-12 Thread Pavan Deolasee
Hi All, As per discussion here http://archives.postgresql.org/pgsql-hackers/2011-05/msg01119.php PFA a patch which implements the idea with some variation. At the start of the first pass, we remember the current LSN. Every page that needs some work is HOT-pruned so that dead tuples are truncated