Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 13:31 +0200, Heikki Linnakangas wrote: > b) seems much simpler to me. OK. Least ugly wins, but she ain't pretty. > > 2. (In HS recovery) When we see first commit record for the VF xid we > > commit the transaction in clog, yet maintain locks and KnownAssigned > > xids > >

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: > ISTM premature to remove all traces of VF from code. We may yet need it > for some reason, especially if doing so creates complex dependencies on > important features. Well, it's still in the repository. > So modified proposal looks like this > > 1. (In normal running) Provi

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: > > My answer is it doesn't, we will still have the problem noted above for > > catalog tables. So we still have a must-fix issue for HS, though that is > > no barrier to the new VF patch. > > I think the VACUUM FULL patch will have to

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: > Could you just mark the transaction as committed when you see the 1st > commit record, but leave the XID in the known-assigned list and not > release locks? That would emulate pretty closely what happens in the master. OK, works for m

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: > I've just reviewed the VACUUM FULL patch to see if it does all we need > it to do, i.e. does the removal of HS code match the new VF patch. Oh, good! > My answer is it doesn't, we will still have the problem noted above for > catalog tables. So we still have a must-fix issue

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: > VACUUM FULL does a peculiar hack: once it's done moving tuples, and > before it truncates the relation, it calls RecordTransactionCommit to > mark the transaction as committed in clog and WAL, but the transaction > is still kept open i

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-03 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: > On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > > As long as there's not anything the master actually does differently > > then I can't see where there'd be any performance testing to do. What's > > bothering me about this is that it seems

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 3:26 AM, Tom Lane wrote: > Greg Stark writes: >> Well the only thing that's been discussed is having vacuum require a >> minimum age before considering a transaction visible to all to reduce >> the chance of conflicts on cleanup records. > > [ shrug... ]  Call me Cassandra

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: > On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > > As long as there's not anything the master actually does differently > > then I can't see where there'd be any performance testing to do. What's > > bothering me about this is that it seems

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Greg Stark writes: > Well the only thing that's been discussed is having vacuum require a > minimum age before considering a transaction visible to all to reduce > the chance of conflicts on cleanup records. [ shrug... ] Call me Cassandra. I am not concerned about what has or has not been discu

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > As long as there's not anything the master actually does differently > then I can't see where there'd be any performance testing to do.  What's > bothering me about this is that it seems likely that we'll find places > where the master has to do t

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Simon Riggs writes: > Tom Lane wrote: >> There's no equivalent of XLogArchivingActive()? > We've tried hard to have it "just work". But I wonder whether we should > have a parameter to allow performance testing on the master? If nobody > finds any issues then we can remove it again, or at least m

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 23:00 +0200, Heikki Linnakangas wrote: > Tom Lane wrote: > > Heikki Linnakangas writes: > >> Tom Lane wrote: > >>> There's no equivalent of XLogArchivingActive()? > > > >> XLogArchivingMode() == false enables us to skip WAL-logging in > >> operations like CLUSTER or COPY, wh

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: > That causes some headaches for Hot Standby I say leave HS as it is and we can clean up when we do the VFectomy. It isn't really a headache, the code works easily enough. I agree its ugly and it should eventually be removed. Let's no

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Hannu Krosing
On Tue, 2009-11-24 at 09:24 +0200, Heikki Linnakangas wrote: > Greg Smith wrote: > > Heikki Linnakangas wrote: > >> So I guess what I'm asking is: Does anyone see any show-stoppers in > >> removing VACUUM FULL > > Here's the disclaimers attached to the new VACUUM REPLACE implementation > > from Ita

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Greg Smith wrote: > Heikki Linnakangas wrote: >> So I guess what I'm asking is: Does anyone see any show-stoppers in >> removing VACUUM FULL > Here's the disclaimers attached to the new VACUUM REPLACE implementation > from Itagaki: > > "We still need traditional VACUUM FULL behavior for system cat

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Itagaki Takahiro wrote: > VACUUM FULL is still reserved for system catalogs in my patch > because we cannot modify relfilenodes for the catalog tables. > Do you have solutions for it? Tom had an idea on that: http://archives.postgresql.org/message-id/19750.1252094...@sss.pgh.pa.us -- Heikki L

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Itagaki Takahiro
Heikki Linnakangas wrote: > So I guess what I'm asking is: Does anyone see any show-stoppers in > removing VACUUM FULL, and does anyone want to step up to the plate and > promise to do it before release? I'm working on "New VACUUM FULL" patch, that shrinks tables using CLUSTER-like rewrites. ht

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> Tom Lane wrote: >>> There's no equivalent of XLogArchivingActive()? > >> XLogArchivingMode() == false enables us to skip WAL-logging in >> operations like CLUSTER or COPY, which is a big optimization. I don't >> see anything like that in Hot Standby

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> There's no equivalent of XLogArchivingActive()? > XLogArchivingMode() == false enables us to skip WAL-logging in > operations like CLUSTER or COPY, which is a big optimization. I don't > see anything like that in Hot Standby. There is a few small th

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> Tom Lane wrote: >>> I don't see much problem with rejecting VAC FULL in a HS master, >>> whether or not it gets removed altogether. Why not just do that >>> rather than write a lot of kluges? > >> Hmm. At the moment, no action is required in the ma

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> I don't see much problem with rejecting VAC FULL in a HS master, >> whether or not it gets removed altogether. Why not just do that >> rather than write a lot of kluges? > Hmm. At the moment, no action is required in the master to allow hot > stand

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> So I guess what I'm asking is: Does anyone see any show-stoppers in >> removing VACUUM FULL, and does anyone want to step up to the plate and >> promise to do it before release? > > I don't see much problem with rejecting VAC FULL in a HS master, >

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas writes: > So I guess what I'm asking is: Does anyone see any show-stoppers in > removing VACUUM FULL, and does anyone want to step up to the plate and > promise to do it before release? I don't see much problem with rejecting VAC FULL in a HS master, whether or not it gets remo

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Greg Smith
Heikki Linnakangas wrote: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL Here's the disclaimers attached to the new VACUUM REPLACE implementation from Itagaki: "We still need traditional VACUUM FULL behavior for system catalog because we cannot change