Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-22 Thread Michael Glaesemann
On Sep 22, 2005, at 3:55 PM, David Fetter wrote: On Thu, Sep 22, 2005 at 03:52:21PM +1000, Gavin Sherry wrote: On Wed, 31 Aug 2005, Tom Lane wrote: Well, I was just poking around the executor and noticed this in ExecDelete(): /* * Note: Normally one would think that we have to dele

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-21 Thread David Fetter
On Thu, Sep 22, 2005 at 03:52:21PM +1000, Gavin Sherry wrote: > On Wed, 31 Aug 2005, Tom Lane wrote: > > > BTW ... the original Berkeley papers on Postgres make frequent reference > > to a "vacuum daemon", which seems to be essentially what we're trying to > > build with autovacuum. Does anyone k

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-21 Thread Gavin Sherry
On Wed, 31 Aug 2005, Tom Lane wrote: > BTW ... the original Berkeley papers on Postgres make frequent reference > to a "vacuum daemon", which seems to be essentially what we're trying to > build with autovacuum. Does anyone know if the Berkeley implementation > ever actually had auto vacuuming, o

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-01 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > My first proposal is to add an extra parameter onto the > index_bulk_delete() call - ntuples. If ntuples == 0 then btbulkdelete() > will avoid scanning and return immediately. If a scan occurs, then we > keep track of how many tuples have been marked delete

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-01 Thread Simon Riggs
On Thu, 2005-09-01 at 10:29 +0100, Simon Riggs wrote: > On Wed, 2005-08-31 at 22:21 -0400, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > On Wed, 2005-08-31 at 19:24 -0400, Tom Lane wrote: > > >> If you don't remove any tuples, > > >> you don't scan the indexes anyway IIRC. > >

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-01 Thread Alvaro Herrera
On Thu, Sep 01, 2005 at 04:21:58AM -, Andrew - Supernews wrote: > On 2005-09-01, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > > On Thu, Sep 01, 2005 at 01:57:02AM +0100, Simon Riggs wrote: > > > >> > If you're using autovacuum then the problem is already taken care of. > >> > >> autovacuum will

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-01 Thread Tom Lane
Josh Berkus writes: > So, will per-table XID tracking allow us to avoid *ever* vacuuming some > tables? If your definition of "ever" is less than a billion transactions, sure. (As Simon points out, with time-partitioned data sets that could often be arranged, so it's not a completely silly resp

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-09-01 Thread Simon Riggs
On Wed, 2005-08-31 at 22:21 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Wed, 2005-08-31 at 19:24 -0400, Tom Lane wrote: > >> If you don't remove any tuples, > >> you don't scan the indexes anyway IIRC. > > > No. Even if you remove *zero* tuples, an index is still scanne

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Andrew - Supernews
On 2005-09-01, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > On Thu, Sep 01, 2005 at 01:57:02AM +0100, Simon Riggs wrote: > >> > If you're using autovacuum then the problem is already taken care of. >> >> autovacuum will respond only to UPDATEs and DELETEs. In the scenario I >> outline, these will *

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Josh Berkus
Tom, > If you're using autovacuum then the problem is already taken care of. > It will be taken care of better in 8.2, if we add per-table tracking > of XID wraparound risk, but it's handled now.  The only way that this > proposal makes any sense is if you are trying not to vacuum at all, ever. H

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Josh Berkus
Gavin, Tom, > Well, from my reading of some of the early papers, VACUUM was kind of > different to what it is now. The idea was that expired data would be moved > out the heap and stored else where. A timetravel mechanism could be used > to see previous versions of the row. And from talking to a

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Gavin Sherry
On Wed, 31 Aug 2005, Tom Lane wrote: > Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: > >> I really really do not like proposals to introduce still another kind > >> of VACUUM. We have too many already; any casual glance through the > >> archives will show that most PG users don't have a gri

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Tom Lane
Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: >> I really really do not like proposals to introduce still another kind >> of VACUUM. We have too many already; any casual glance through the >> archives will show that most PG users don't have a grip on when to use >> VACUUM FULL vs VACUUM. Th

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Christopher Kings-Lynne
I really really do not like proposals to introduce still another kind of VACUUM. We have too many already; any casual glance through the archives will show that most PG users don't have a grip on when to use VACUUM FULL vs VACUUM. Throwing in some more types will make that problem exponentially

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Wed, 2005-08-31 at 19:24 -0400, Tom Lane wrote: >> If you don't remove any tuples, >> you don't scan the indexes anyway IIRC. > No. Even if you remove *zero* tuples, an index is still scanned twice. > Once to not delete the rows and once to not delete t

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Alvaro Herrera
On Thu, Sep 01, 2005 at 01:57:02AM +0100, Simon Riggs wrote: > > If you're using autovacuum then the problem is already taken care of. > > autovacuum will respond only to UPDATEs and DELETEs. In the scenario I > outline, these will *never* occur on the largest tables. A VACUUM would > still event

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Simon Riggs
On Wed, 2005-08-31 at 19:24 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > A new command is proposed - VACUUM MINIMAL. The *sole* purpose of this > > command is to do the absolute minimum required to avoid transaction id > > wraparound. (Better names welcome) > > I do not

Re: [HACKERS] Minimally avoiding Transaction Wraparound in VLDBs

2005-08-31 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > A new command is proposed - VACUUM MINIMAL. The *sole* purpose of this > command is to do the absolute minimum required to avoid transaction id > wraparound. (Better names welcome) I do not see the point. If you only need to run it every billion trans