Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 13:31 +0200, Heikki Linnakangas wrote: > b) seems much simpler to me. OK. Least ugly wins, but she ain't pretty. > > 2. (In HS recovery) When we see first commit record for the VF xid we > > commit the transaction in clog, yet maintain locks and KnownAssigned > > xids > >

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: > ISTM premature to remove all traces of VF from code. We may yet need it > for some reason, especially if doing so creates complex dependencies on > important features. Well, it's still in the repository. > So modified proposal looks like this > > 1. (In normal running) Provi

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: > > My answer is it doesn't, we will still have the problem noted above for > > catalog tables. So we still have a must-fix issue for HS, though that is > > no barrier to the new VF patch. > > I think the VACUUM FULL patch will have to

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Fri, 2009-12-04 at 11:22 +0200, Heikki Linnakangas wrote: > Could you just mark the transaction as committed when you see the 1st > commit record, but leave the XID in the known-assigned list and not > release locks? That would emulate pretty closely what happens in the master. OK, works for m

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Heikki Linnakangas
Simon Riggs wrote: > I've just reviewed the VACUUM FULL patch to see if it does all we need > it to do, i.e. does the removal of HS code match the new VF patch. Oh, good! > My answer is it doesn't, we will still have the problem noted above for > catalog tables. So we still have a must-fix issue

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-04 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: > VACUUM FULL does a peculiar hack: once it's done moving tuples, and > before it truncates the relation, it calls RecordTransactionCommit to > mark the transaction as committed in clog and WAL, but the transaction > is still kept open i

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-12-03 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: > On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > > As long as there's not anything the master actually does differently > > then I can't see where there'd be any performance testing to do. What's > > bothering me about this is that it seems

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 3:26 AM, Tom Lane wrote: > Greg Stark writes: >> Well the only thing that's been discussed is having vacuum require a >> minimum age before considering a transaction visible to all to reduce >> the chance of conflicts on cleanup records. > > [ shrug... ]  Call me Cassandra

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Wed, 2009-11-25 at 03:12 +, Greg Stark wrote: > On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > > As long as there's not anything the master actually does differently > > then I can't see where there'd be any performance testing to do. What's > > bothering me about this is that it seems

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Greg Stark writes: > Well the only thing that's been discussed is having vacuum require a > minimum age before considering a transaction visible to all to reduce > the chance of conflicts on cleanup records. [ shrug... ] Call me Cassandra. I am not concerned about what has or has not been discu

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Greg Stark
On Wed, Nov 25, 2009 at 2:10 AM, Tom Lane wrote: > As long as there's not anything the master actually does differently > then I can't see where there'd be any performance testing to do.  What's > bothering me about this is that it seems likely that we'll find places > where the master has to do t

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Tom Lane
Simon Riggs writes: > Tom Lane wrote: >> There's no equivalent of XLogArchivingActive()? > We've tried hard to have it "just work". But I wonder whether we should > have a parameter to allow performance testing on the master? If nobody > finds any issues then we can remove it again, or at least m

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 23:00 +0200, Heikki Linnakangas wrote: > Tom Lane wrote: > > Heikki Linnakangas writes: > >> Tom Lane wrote: > >>> There's no equivalent of XLogArchivingActive()? > > > >> XLogArchivingMode() == false enables us to skip WAL-logging in > >> operations like CLUSTER or COPY, wh

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-24 Thread Simon Riggs
On Sat, 2009-11-21 at 20:20 +0200, Heikki Linnakangas wrote: > That causes some headaches for Hot Standby I say leave HS as it is and we can clean up when we do the VFectomy. It isn't really a headache, the code works easily enough. I agree its ugly and it should eventually be removed. Let's no

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Hannu Krosing
On Tue, 2009-11-24 at 09:24 +0200, Heikki Linnakangas wrote: > Greg Smith wrote: > > Heikki Linnakangas wrote: > >> So I guess what I'm asking is: Does anyone see any show-stoppers in > >> removing VACUUM FULL > > Here's the disclaimers attached to the new VACUUM REPLACE implementation > > from Ita

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Greg Smith wrote: > Heikki Linnakangas wrote: >> So I guess what I'm asking is: Does anyone see any show-stoppers in >> removing VACUUM FULL > Here's the disclaimers attached to the new VACUUM REPLACE implementation > from Itagaki: > > "We still need traditional VACUUM FULL behavior for system cat

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Heikki Linnakangas
Itagaki Takahiro wrote: > VACUUM FULL is still reserved for system catalogs in my patch > because we cannot modify relfilenodes for the catalog tables. > Do you have solutions for it? Tom had an idea on that: http://archives.postgresql.org/message-id/19750.1252094...@sss.pgh.pa.us -- Heikki L

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-23 Thread Itagaki Takahiro
Heikki Linnakangas wrote: > So I guess what I'm asking is: Does anyone see any show-stoppers in > removing VACUUM FULL, and does anyone want to step up to the plate and > promise to do it before release? I'm working on "New VACUUM FULL" patch, that shrinks tables using CLUSTER-like rewrites. ht

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> Tom Lane wrote: >>> There's no equivalent of XLogArchivingActive()? > >> XLogArchivingMode() == false enables us to skip WAL-logging in >> operations like CLUSTER or COPY, which is a big optimization. I don't >> see anything like that in Hot Standby

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> There's no equivalent of XLogArchivingActive()? > XLogArchivingMode() == false enables us to skip WAL-logging in > operations like CLUSTER or COPY, which is a big optimization. I don't > see anything like that in Hot Standby. There is a few small th

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> Tom Lane wrote: >>> I don't see much problem with rejecting VAC FULL in a HS master, >>> whether or not it gets removed altogether. Why not just do that >>> rather than write a lot of kluges? > >> Hmm. At the moment, no action is required in the ma

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> I don't see much problem with rejecting VAC FULL in a HS master, >> whether or not it gets removed altogether. Why not just do that >> rather than write a lot of kluges? > Hmm. At the moment, no action is required in the master to allow hot > stand

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
Tom Lane wrote: > Heikki Linnakangas writes: >> So I guess what I'm asking is: Does anyone see any show-stoppers in >> removing VACUUM FULL, and does anyone want to step up to the plate and >> promise to do it before release? > > I don't see much problem with rejecting VAC FULL in a HS master, >

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Tom Lane
Heikki Linnakangas writes: > So I guess what I'm asking is: Does anyone see any show-stoppers in > removing VACUUM FULL, and does anyone want to step up to the plate and > promise to do it before release? I don't see much problem with rejecting VAC FULL in a HS master, whether or not it gets remo

Re: [HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Greg Smith
Heikki Linnakangas wrote: So I guess what I'm asking is: Does anyone see any show-stoppers in removing VACUUM FULL Here's the disclaimers attached to the new VACUUM REPLACE implementation from Itagaki: "We still need traditional VACUUM FULL behavior for system catalog because we cannot change

[HACKERS] Hot standby and removing VACUUM FULL

2009-11-21 Thread Heikki Linnakangas
VACUUM FULL does a peculiar hack: once it's done moving tuples, and before it truncates the relation, it calls RecordTransactionCommit to mark the transaction as committed in clog and WAL, but the transaction is still kept open in proc array. After it's done with truncating and other cleanup, norma