Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-27 Thread Selena Deckelmann
On Tue, Jan 26, 2010 at 11:03 PM, Greg Smith wrote: > Tom Lane wrote: >> >> Right, but the question is: is there enough use-case left in it to >> justify spending community effort on polishing rough edges?  It's not >> like we haven't got plenty else to do to get 9.0 out the door. >> > > The point

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Greg Smith
Tom Lane wrote: Right, but the question is: is there enough use-case left in it to justify spending community effort on polishing rough edges? It's not like we haven't got plenty else to do to get 9.0 out the door. The point I was trying to make is that I'm on the hook to do this particula

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Simon Riggs
On Tue, 2010-01-26 at 11:01 -0800, Josh Berkus wrote: > Anyway, if we fix the *core* bogus error messages for standby mode, that > will eliminate half of what's confusing and alarming people (and all of > it for those using SR). Just done that. -- Simon Riggs www.2ndQuadrant.com --

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Josh Berkus
On 1/26/10 10:58 AM, Tom Lane wrote: > Josh Berkus writes: >> On 1/26/10 10:48 AM, Heikki Linnakangas wrote: >>> I didn't intend to replace pg_standby when I started this, it just kind >>> of happened. Maybe we should provide a sample script similar to >>> pg_standby, to be used instead of plain '

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Tom Lane
Josh Berkus writes: > On 1/26/10 10:48 AM, Heikki Linnakangas wrote: >> I didn't intend to replace pg_standby when I started this, it just kind >> of happened. Maybe we should provide a sample script similar to >> pg_standby, to be used instead of plain 'cp', that does the cleanup too. > What I'm

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Josh Berkus
On 1/26/10 10:48 AM, Heikki Linnakangas wrote: > Josh Berkus wrote: >>> *That* makes pg_standby obsolete, not streaming replication per se. >>> Setting standby_mode=on, with a valid restore_command using e.g 'cp' and >>> no connection info for walreceiver is more or less the same as using >>> pg_st

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Heikki Linnakangas
Josh Berkus wrote: >> *That* makes pg_standby obsolete, not streaming replication per se. >> Setting standby_mode=on, with a valid restore_command using e.g 'cp' and >> no connection info for walreceiver is more or less the same as using >> pg_standby. > > What about deletion of no-longer-needed W

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Josh Berkus
> *That* makes pg_standby obsolete, not streaming replication per se. > Setting standby_mode=on, with a valid restore_command using e.g 'cp' and > no connection info for walreceiver is more or less the same as using > pg_standby. What about deletion of no-longer-needed WALfile copies? --Josh Ber

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Tom Lane
Dimitri Fontaine writes: > Heikki Linnakangas writes: >> *That* makes pg_standby obsolete, not streaming replication per se. >> Setting standby_mode=on, with a valid restore_command using e.g 'cp' and >> no connection info for walreceiver is more or less the same as using >> pg_standby. > I've y

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Fujii Masao
On Tue, Jan 26, 2010 at 8:54 PM, Heikki Linnakangas wrote: > Simon Riggs wrote: >> Currently, we had presumed >> that standby_mode = off would be the same as Warm Standby replication, >> using pg_standby or other. > > That's still true. standby_mode=off is the same as what you have in PG > 8.4. Yo

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Robert Haas
On Tue, Jan 26, 2010 at 6:54 AM, Heikki Linnakangas wrote: > From a user point-of-view, it might be simplest to provide a > "restore_directory" option, besides restore_command, and handle the copy > ourselves. Well, that might not handle all possible use cases -- of course, it could be there as a

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Dimitri Fontaine
Heikki Linnakangas writes: > From a user point-of-view, it might be simplest to provide a > "restore_directory" option, besides restore_command, and handle the copy > ourselves. Even more user-friendly would be to provide default working archive and restore commands, maybe simple shell scripts us

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Heikki Linnakangas
Simon Riggs wrote: > Currently, we had presumed > that standby_mode = off would be the same as Warm Standby replication, > using pg_standby or other. That's still true. standby_mode=off is the same as what you have in PG 8.4. You can still use pg_standby etc. with that. > pg_standby checks file s

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Heikki Linnakangas
Dimitri Fontaine wrote: > Heikki Linnakangas writes: >> Yes. Just like with pg_standby. > > Hehe, I'm using walmgr.py from skytools instead, and this discussion > makes me think I'll continue doing so even if using SR, as they are > complementary solutions. > > In SR mode, the master continues t

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Simon Riggs
On Tue, 2010-01-26 at 12:12 +0200, Heikki Linnakangas wrote: > Magnus Hagander wrote: > > I think there are definite use-cases for pg_standby as well, even when > > we have SR. SR requires you to have a reasonably reliable network > > connection that lets you do an arbitrary TCP connection. There a

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Dimitri Fontaine
Heikki Linnakangas writes: > Yes. Just like with pg_standby. Hehe, I'm using walmgr.py from skytools instead, and this discussion makes me think I'll continue doing so even if using SR, as they are complementary solutions. In SR mode, the master continues to archive as usual, and the slave will

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Heikki Linnakangas
Dimitri Fontaine wrote: > Heikki Linnakangas writes: >> *That* makes pg_standby obsolete, not streaming replication per se. >> Setting standby_mode=on, with a valid restore_command using e.g 'cp' and >> no connection info for walreceiver is more or less the same as using >> pg_standby. > > I've y

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, Jan 26, 2010 at 11:24:09AM +0100, Dimitri Fontaine wrote: [...] > I've yet to understand how the files in the archive get from the master > to the slave in this case, or are you supposing in your example that the > cp in the restore_command i

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Dimitri Fontaine
Heikki Linnakangas writes: > *That* makes pg_standby obsolete, not streaming replication per se. > Setting standby_mode=on, with a valid restore_command using e.g 'cp' and > no connection info for walreceiver is more or less the same as using > pg_standby. I've yet to understand how the files in

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Magnus Hagander
2010/1/26 Heikki Linnakangas : > Magnus Hagander wrote: >> I think there are definite use-cases for pg_standby as well, even when >> we have SR. SR requires you to have a reasonably reliable network >> connection that lets you do an arbitrary TCP connection. There are a >> lot of scenarios that cou

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Heikki Linnakangas
Magnus Hagander wrote: > I think there are definite use-cases for pg_standby as well, even when > we have SR. SR requires you to have a reasonably reliable network > connection that lets you do an arbitrary TCP connection. There are a > lot of scenarios that could still use the > "here's-a-file-you

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Magnus Hagander
2010/1/26 Tom Lane : > Greg Smith writes: >> [ Greg and Selena discuss filing some rough edges off pg_standby ] > > Maybe I'm missing something, but I thought pg_standby would be mostly > dead once SR hits the streets.  Is it worth spending lots of time on? > > The ideas all sound good, I'm just w

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-26 Thread Simon Riggs
On Tue, 2010-01-26 at 11:08 +0900, Fujii Masao wrote: > On Tue, Jan 26, 2010 at 9:08 AM, Simon Riggs wrote: > > Just committed a fix: the server no longer requests 01.history > > at start of archive recovery. > > Good. > > And I think that writeTimeLineHistory() should also skip the requ

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Fujii Masao
On Tue, Jan 26, 2010 at 9:08 AM, Simon Riggs wrote: > Just committed a fix: the server no longer requests 01.history > at start of archive recovery. Good. And I think that writeTimeLineHistory() should also skip the request of 01.history. Here is the patch to do so. Comments? Re

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Greg Smith
Tom Lane wrote: Maybe I'm missing something, but I thought pg_standby would be mostly dead once SR hits the streets. Is it worth spending lots of time on? I have to do the work I outlined regardless, to support installs on earlier versions (luckily there's few backport issues for this code

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Josh Berkus
On 1/25/10 4:09 PM, Tom Lane wrote: > Greg Smith writes: >> [ Greg and Selena discuss filing some rough edges off pg_standby ] > > Maybe I'm missing something, but I thought pg_standby would be mostly > dead once SR hits the streets. Is it worth spending lots of time on? > > The ideas all sound

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Tom Lane
Greg Smith writes: > [ Greg and Selena discuss filing some rough edges off pg_standby ] Maybe I'm missing something, but I thought pg_standby would be mostly dead once SR hits the streets. Is it worth spending lots of time on? The ideas all sound good, I'm just wondering if it's useful effort a

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Simon Riggs
On Mon, 2010-01-25 at 16:34 -0500, Greg Smith wrote: > Josh Berkus wrote: > > We discussed this issue at LCA where I encountered these bogus error > > messages when I was doing the demo of HS. I consider Selena's patch to > > be a bug-fix for beta of 9.0, not a feature. Currently the database > >

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Greg Smith
Selena Deckelmann wrote: I can scan through the code tonight and look for other cases where this might be an issue. The main thing I'm looking for is to distinguish between harmful and non-harmful errors. Where I think this is going toward is where every line that comes out of this program

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Selena Deckelmann
Hi! On Mon, Jan 25, 2010 at 1:34 PM, Greg Smith wrote: > Josh Berkus wrote: >> >> We discussed this issue at LCA where I encountered these bogus error >> messages when I was doing the demo of HS.  I consider Selena's patch to >> be a bug-fix for beta of 9.0, not a feature.  Currently the database

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Greg Smith
Josh Berkus wrote: We discussed this issue at LCA where I encountered these bogus error messages when I was doing the demo of HS. I consider Selena's patch to be a bug-fix for beta of 9.0, not a feature. Currently the database reports a lot of false error messages when running in standby mode,

Re: [HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Josh Berkus
On 1/25/10 9:45 AM, Selena Deckelmann wrote: > Hi! > > I poked around a little at pg_standby because of some 'cp: file does > not exist' errors that were cropping up as pg_standby searched around > for .history files. > > I added a stat that still outputs debugging information but now we > don't

[HACKERS] Dividing progress/debug information in pg_standby, and stat before copy

2010-01-25 Thread Selena Deckelmann
Hi! I poked around a little at pg_standby because of some 'cp: file does not exist' errors that were cropping up as pg_standby searched around for .history files. I added a stat that still outputs debugging information but now we don't get cp failures, and then added a 'progress' mode. It was a b