Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Jim C. Nasby
On Thu, Mar 01, 2007 at 01:05:28PM +, Simon Riggs wrote: > ISTM a radical approach is needed, so I'm very open to discussion about > this and how we cope. > > If we break down the main thoughts into a few parts: > > 1. would like a way to CLUSTER/VACUUM FULL where we don't have to move > all

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Jim C. Nasby
On Thu, Mar 01, 2007 at 04:54:44PM +0200, Hannu Krosing wrote: > > > I was not suggesting that we remove visible rows through truncation. > > > > Sure, unless you suggest to not truncate during this vacuum run ? > > But we are talking about vacuum full, so truncation is essential. > > > > It was

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Joshua D. Drake
> This means that > > VACUUM FULL mytable; > > would translate to: > > VACUUM mytable; -- make free space > COMPACT mytable; -- move tuples in a bunch of small transactions > -- might have a GUC for max trx length > VACUUM mytable; -- free the tuples at the end and give spac

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Hannu Krosing
Ühel kenal päeval, N, 2007-03-01 kell 15:03, kirjutas Zeugswetter Andreas ADI SD: > > > > -- start the VACUUM from the first non-filled block > > > > > > > > So if we do this, we wouldn't need to worry about HOT tuples at > all, > > > > nor would we need to wait until all transact

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Zeugswetter Andreas ADI SD
> > > -- start the VACUUM from the first non-filled block > > > > > > So if we do this, we wouldn't need to worry about HOT tuples at all, > > > nor would we need to wait until all transactions are gone. > > > > You need to wait until you are allowed to truncate if you want > > concu

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Simon Riggs
On Thu, 2007-03-01 at 14:35 +0100, Zeugswetter Andreas ADI SD wrote: > > -- start the VACUUM from the first non-filled block > > > > So if we do this, we wouldn't need to worry about HOT tuples > > at all, nor would we need to wait until all transactions are gone. > > You need to wai

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Hannu Krosing
Ühel kenal päeval, N, 2007-03-01 kell 13:14, kirjutas Simon Riggs: > On Thu, 2007-03-01 at 14:42 +0200, Hannu Krosing wrote: > > Ühel kenal päeval, N, 2007-03-01 kell 14:32, kirjutas Hannu Krosing: > > > > > If we can trust FSM, the whole process just becomes the backward scan > > > and null updat

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Zeugswetter Andreas ADI SD
> -- start the VACUUM from the first non-filled block > > So if we do this, we wouldn't need to worry about HOT tuples > at all, nor would we need to wait until all transactions are gone. You need to wait until you are allowed to truncate if you want concurrency. Or a concurrent s

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Simon Riggs
On Thu, 2007-03-01 at 14:42 +0200, Hannu Krosing wrote: > Ühel kenal päeval, N, 2007-03-01 kell 14:32, kirjutas Hannu Krosing: > > > If we can trust FSM, the whole process just becomes the backward scan > > and null updates until the null update does not move tuple to a lower > > page. Also, for t

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Simon Riggs
On Thu, 2007-03-01 at 14:32 +0200, Hannu Krosing wrote: > Ühel kenal päeval, N, 2007-03-01 kell 12:02, kirjutas Simon Riggs: > > Use case for VACUUM FULL is very low these days. VACUUM does the most > > important part of what VACUUM FULL offers, yet does it concurrently > > rather than with a full

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Csaba Nagy
On Thu, 2007-03-01 at 13:56, Simon Riggs wrote: > > Wouldn't this be deadlock prone ? What if a non-utility transaction > > (which could even be started before the vacuum full) blocks on the table > > being vacuumed, then if the vacuum wants to wait until all non-utility > > transactions finish wil

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Simon Riggs
On Thu, 2007-03-01 at 13:21 +0100, Csaba Nagy wrote: > On Thu, 2007-03-01 at 13:02, Simon Riggs wrote: > > I would like to introduce the concept of utility transactions. This is > > any transaction that touches only one table in a transaction and is not > > returning or modifying data. All utility

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Hannu Krosing
Ühel kenal päeval, N, 2007-03-01 kell 14:32, kirjutas Hannu Krosing: > If we can trust FSM, the whole process just becomes the backward scan > and null updates until the null update does not move tuple to a lower > page. Also, for the duration of COMPACT TABLE the updated tuple should > always be

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Hannu Krosing
Ühel kenal päeval, N, 2007-03-01 kell 12:02, kirjutas Simon Riggs: > Use case for VACUUM FULL is very low these days. VACUUM does the most > important part of what VACUUM FULL offers, yet does it concurrently > rather than with a full table lock. VACUUM FULL also > - has very long execution time >

Re: [HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Csaba Nagy
On Thu, 2007-03-01 at 13:02, Simon Riggs wrote: > I would like to introduce the concept of utility transactions. This is > any transaction that touches only one table in a transaction and is not > returning or modifying data. All utility transactions wait until they > are older than all non-utility

[HACKERS] Revitalising VACUUM FULL for 8.3

2007-03-01 Thread Simon Riggs
Use case for VACUUM FULL is very low these days. VACUUM does the most important part of what VACUUM FULL offers, yet does it concurrently rather than with a full table lock. VACUUM FULL also - has very long execution time - generates lots of WAL traffic - uses lots of memory while it runs - isn't a