Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Robert Haas
On Fri, Sep 24, 2010 at 10:01 AM, Dimitri Fontaine wrote: >>  Configuring whether the >> master will retain WAL for a disconnected slave on the slave is >> outright byzantine. > > Again, I can't remember having proposed such a thing. No one has, but I keep hearing we don't need the master to have

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 17:13, Simon Riggs wrote: On Fri, 2010-09-24 at 16:01 +0200, Dimitri Fontaine wrote: I'd like that we now follow Josh Berkus (and some other) advice now, and start a new thread to decide what we mean by synchronous replication, what kind of normal behaviour we want and what response

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 16:01 +0200, Dimitri Fontaine wrote: > I'd like that we now follow Josh Berkus (and some other) advice now, and > start a new thread to decide what we mean by synchronous replication, > what kind of normal behaviour we want and what responses to errors we > expect to be able

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Hi, Defending my ideas as not to be put in the bag you're wanting to put away. We have more than 2 proposals lying around here. I'm one of the guys with a proposal and no code, but still trying to be clear. Robert Haas writes: > The reason I think that we should centralize parameters on the mast

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 14:47, Simon Riggs wrote: On Fri, 2010-09-24 at 14:12 +0300, Heikki Linnakangas wrote: What I'm saying is that in a two standby situation, if you're willing to continue operation as usual in the master even if the standby is down, you're not doing synchronous replication. Oracle an

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Aidan Van Dyk
On Fri, Sep 24, 2010 at 7:47 AM, Simon Riggs wrote: > On Fri, 2010-09-24 at 14:12 +0300, Heikki Linnakangas wrote: >> What I'm saying is that in a two standby situation, if >> you're willing to continue operation as usual in the master even if >> the standby is down, you're not doing synchronous r

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Robert Haas
On Fri, Sep 24, 2010 at 6:37 AM, Simon Riggs wrote: >> > Earlier you argued that centralizing parameters would make this nice and >> > simple. Now you're pointing out that we aren't centralizing this at all, >> > and it won't be simple. We'll have to have a standby.conf set up that is >> > customi

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 14:12 +0300, Heikki Linnakangas wrote: > What I'm saying is that in a two standby situation, if > you're willing to continue operation as usual in the master even if > the standby is down, you're not doing synchronous replication. Oracle and I disagree with you on that point

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 13:57, Simon Riggs wrote: If you want high availability you need N+1 redundancy. If you want a standby server that is N=1. If you want a highly available standby configuration then N+1 = 2. Yep. Synchronous replication with one standby gives you zero data loss. When you add a 2nd s

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Robert Haas writes: > I think maybe you missed Tom's point, or else you just didn't respond > to it. If the master is wedged because it is waiting for a standby, > then you cannot commit transactions on the master. Therefore you > cannot update the system catalog which you must update to unwedge

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 11:43 +0300, Heikki Linnakangas wrote: > > To get zero data loss *and* continuous availability, you need two > > standbys offering sync rep and reply-to-first behaviour. > > Yes, that is a good point. > > I'm starting to understand what your proposal was all about. It makes

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 11:08 +0300, Heikki Linnakangas wrote: > On 24/09/10 01:11, Simon Riggs wrote: > >> But that's not what I call synchronous replication, it doesn't give > >> you the guarantees that > >> textbook synchronous replication does. > > > > Which textbook? > > I was using that word m

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Heikki Linnakangas writes: > If you want the behavior where the master doesn't acknowledge a commit to > the client until the standby (or all standbys, or one of them etc.) > acknowledges it, even if the standby is not currently connected, the master > needs to know what standby servers exist. *Th

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Tom Lane writes: > Oh, I thought part of the objective here was to try to centralize that > stuff. If we're assuming that slaves will still have local replication > configuration files, then I think we should just add any necessary info > to those files and drop this entire conversation. We're e

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Thu, 2010-09-23 at 16:09 -0400, Robert Haas wrote: > On Thu, Sep 23, 2010 at 3:46 PM, Simon Riggs wrote: > > Well, its not at all hard to see how that could be configured, because I > > already proposed a simple way of implementing parameters that doesn't > > suffer from those problems. My pro

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Thu, 2010-09-23 at 14:26 +0200, Csaba Nagy wrote: > Unfortunately it was quite long time ago we last tried, and I don't > remember exactly what was bottlenecked. Our application is quite > write-intensive, the ratio of writes to reads which actually reaches > the disk is about 50-200% (according

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 01:11, Simon Riggs wrote: On Thu, 2010-09-23 at 20:42 +0300, Heikki Linnakangas wrote: If you want the behavior where the master doesn't acknowledge a commit to the client until the standby (or all standbys, or one of them etc.) acknowledges it, even if the standby is not currently c

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 01:11, Simon Riggs wrote: But that's not what I call synchronous replication, it doesn't give you the guarantees that textbook synchronous replication does. Which textbook? I was using that word metaphorically, but for example: Wikipedia http://en.wikipedia.org/wiki/Replication_

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Markus Wanner
Simon, On 09/24/2010 12:11 AM, Simon Riggs wrote: > As I keep pointing out, waiting for an acknowledgement from something > that isn't there might just take a while. The only guarantee that > provides is that you will wait a long time. Is my data more safe? No. By now I agree that waiting for dis

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Markus Wanner
On 09/23/2010 10:09 PM, Robert Haas wrote: > I think maybe you missed Tom's point, or else you just didn't respond > to it. If the master is wedged because it is waiting for a standby, > then you cannot commit transactions on the master. Therefore you > cannot update the system catalog which you

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Thu, 2010-09-23 at 20:42 +0300, Heikki Linnakangas wrote: > If you want the behavior where the master doesn't acknowledge a > commit > to the client until the standby (or all standbys, or one of them > etc.) > acknowledges it, even if the standby is not currently connected, the > master needs

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Wed, 2010-09-22 at 15:31 -0700, Josh Berkus wrote: > > The above case is one where I can see your point and it does sound > > easier in that case. But I then think: "What happens after failover?". > > We would then need to have 12 different standby.conf files, one on each > > standby that descri

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 3:46 PM, Simon Riggs wrote: > Well, its not at all hard to see how that could be configured, because I > already proposed a simple way of implementing parameters that doesn't > suffer from those problems. My proposal did not give roles to named > standbys and is symmetrical

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Thu, 2010-09-23 at 13:07 -0400, Tom Lane wrote: > Robert Haas writes: > > Now, admittedly, in more complex topologies, and especially if you're > > using configuration options that pertain to the behavior of > > disconnected standbys (e.g. wait for them, or retain WAL for them), > > you're goin

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Heikki Linnakangas
On 23/09/10 20:03, Tom Lane wrote: Robert Haas writes: On Thu, Sep 23, 2010 at 12:52 PM, Tom Lane wrote: Um ... so how does this standby know what master to connect to, what password to offer, etc? I don't think that "pass down parameters after connecting" is likely to cover anything but a s

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: > > What other problems are there that mean we *must* have a file? > > Well, for one thing, how do you add a new slave? If its configuration > comes from a system catalog, it seems that it has to already be > replicating before it knows what its

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
On Thu, 2010-09-23 at 16:18 +0300, Heikki Linnakangas wrote: > There's a program called pg_readahead somewhere on pgfoundry by NTT that > will help if it's the single-threadedness of I/O. Before handing the WAL > file to the server, it scans it through and calls posix_fadvise for all > the block

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 1:03 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 23, 2010 at 12:52 PM, Tom Lane wrote: >>> Um ... so how does this standby know what master to connect to, what >>> password to offer, etc?  I don't think that "pass down parameters after >>> connecting" is like

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 23, 2010 at 12:52 PM, Tom Lane wrote: >> Um ... so how does this standby know what master to connect to, what >> password to offer, etc?  I don't think that "pass down parameters after >> connecting" is likely to cover anything but a small subset of the >> config

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Robert Haas writes: > Now, admittedly, in more complex topologies, and especially if you're > using configuration options that pertain to the behavior of > disconnected standbys (e.g. wait for them, or retain WAL for them), > you're going to need to adjust the configs. But I think that's likely >

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 12:52 PM, Tom Lane wrote: > Simon Riggs writes: >> On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: >>> Well, for one thing, how do you add a new slave?  If its configuration >>> comes from a system catalog, it seems that it has to already be >>> replicating before it kn

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Simon Riggs writes: > On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: >> Well, for one thing, how do you add a new slave? If its configuration >> comes from a system catalog, it seems that it has to already be >> replicating before it knows what its configuration is. > At the moment, I'm not

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 11:32 AM, Simon Riggs wrote: > On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: > >> I think it should be a separate config file, and I think it should be >> a config file that can be edited using DDL commands as you propose. >> But it CAN'T be a system catalog, becaus

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: > Simon Riggs writes: > > ISTM that we can have a system catalog and still have cascading slaves. > > If we administer the catalog via the master, why can't we administer all > > slaves, however they cascade, via the master too? > > > What other

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Simon Riggs writes: > ISTM that we can have a system catalog and still have cascading slaves. > If we administer the catalog via the master, why can't we administer all > slaves, however they cascade, via the master too? > What other problems are there that mean we *must* have a file? Well, for

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: > I think it should be a separate config file, and I think it should be > a config file that can be edited using DDL commands as you propose. > But it CAN'T be a system catalog, because, among other problems, that > rules out cascading slaves,

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Heikki Linnakangas
On 23/09/10 15:26, Csaba Nagy wrote: Unfortunately it was quite long time ago we last tried, and I don't remember exactly what was bottlenecked. Our application is quite write-intensive, the ratio of writes to reads which actually reaches the disk is about 50-200% (according to the disk stats - y

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
On Thu, 2010-09-23 at 12:02 +0300, Heikki Linnakangas wrote: > On 23/09/10 11:34, Csaba Nagy wrote: > > In the meantime our DBs are not able to keep in sync via WAL > > replication, that would need some kind of parallel WAL restore on the > > slave I guess, or I'm not able to configure it properly

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
Hi all, Some time ago I was also interested in this feature, and that time I also thought about complete setup possibility via postgres connections, meaning the transfer of the files and all configuration/slave registration to be done through normal backend connections. In the meantime our DBs ar

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Heikki Linnakangas
On 23/09/10 11:34, Csaba Nagy wrote: In the meantime our DBs are not able to keep in sync via WAL replication, that would need some kind of parallel WAL restore on the slave I guess, or I'm not able to configure it properly - in any case now we use slony which is working. It would be interestin

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Simon Riggs
On Mon, 2010-09-20 at 18:24 -0400, Robert Haas wrote: > I feel like that's really nice and simple. There are already 5 separate places to configure to make streaming rep work in a 2 node cluster (master.pg_hba.conf, master.postgresql.conf, standby.postgresql.conf, standby.recovery.conf, password

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Josh Berkus
> The above case is one where I can see your point and it does sound > easier in that case. But I then think: "What happens after failover?". > We would then need to have 12 different standby.conf files, one on each > standby that describes what the setup would look like if that standby > became t

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Yeb Havinga
Tom Lane wrote: Robert Haas writes: On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake wrote: On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves, which are a feature a lot of peop

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dimitri Fontaine
"Joshua D. Drake" writes: > Unless I am missing something the catalog only needs information for its > specific cluster. E.g; My Master is, I am master for. I think the "cluster" here is composed of all and any server partaking into the replication network, whatever its role and cascading level,

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Joshua D. Drake
On Wed, 2010-09-22 at 13:26 -0400, Tom Lane wrote: > Robert Haas writes: > > On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake > > wrote: > >> On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: > >>> But it CAN'T be a system catalog, because, among other problems, that > >>> rules out cascadin

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Tom Lane
Robert Haas writes: > On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake > wrote: >> On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: >>> But it CAN'T be a system catalog, because, among other problems, that >>> rules out cascading slaves, which are a feature a lot of people >>> probably want

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Robert Haas
On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake wrote: > On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: > >> > I mean really? >> > >> > ALTER CLUSTER ENABLE [SYNC] REPLICATION ON db.foobar.com PORT 5432 ALIAS >> > CRITICAL; >> > ALTER CLUSTER SET REPLICATION CRITICAL RECEIVE FOR 2; >> > AL

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 22/09/10 20:02, Heikki Linnakangas wrote: On 22/09/10 20:00, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves, which are a feature a lot of people probably want to eventually have. FWIW it could be a system catalog backed by

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Bruce Momjian
Heikki Linnakangas wrote: > On 22/09/10 20:00, Robert Haas wrote: > > But it CAN'T be a system catalog, because, among other problems, that > > rules out cascading slaves, which are a feature a lot of people > > probably want to eventually have. > > FWIW it could be a system catalog backed by a fl

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Joshua D. Drake
On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: > > I mean really? > > > > ALTER CLUSTER ENABLE [SYNC] REPLICATION ON db.foobar.com PORT 5432 ALIAS > > CRITICAL; > > ALTER CLUSTER SET REPLICATION CRITICAL RECEIVE FOR 2; > > ALTER CLUSTER SET REPLICATION CRITICAL FSYNC FOR 2; > > ALTER CLUSTE

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Aidan Van Dyk
On Wed, Sep 22, 2010 at 8:12 AM, Simon Riggs wrote: Not speaking to the necessity of standby registration, but... >> Thinking of this as a sysadmin, what I want is to have *one place* I can >> go an troubleshoot my standby setup.  If I have 12 synch standbys and >> they're creating too much load

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 22/09/10 20:00, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves, which are a feature a lot of people probably want to eventually have. FWIW it could be a system catalog backed by a flat file. But I'm not in favor of that fo

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Robert Haas
On Wed, Sep 22, 2010 at 12:51 PM, Joshua D. Drake wrote: > On Wed, 2010-09-22 at 17:43 +0100, Thom Brown wrote: > >> So... >> >> sync_rep_services = {critical: recv=2, fsync=2, replay=1; >>                      important: fsync=3; >>                      reporting: recv=2, apply=1} >> >> becomes

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Joshua D. Drake
On Wed, 2010-09-22 at 17:43 +0100, Thom Brown wrote: > So... > > sync_rep_services = {critical: recv=2, fsync=2, replay=1; > important: fsync=3; > reporting: recv=2, apply=1} > > becomes ... > > sync_rep_services.critical.recv = 2 > sync_rep_services.cr

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Bruce Momjian
Thom Brown wrote: > On 22 September 2010 17:23, Bruce Momjian wrote: > > Robert Haas wrote: > >> [server] > >> guc=value > >> > >> or > >> > >> server.guc=value > > ? > > > > Yes, this was my idea too. ?It uses our existing config file format. > > > > So... > > sync_rep_services

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Thom Brown
On 22 September 2010 17:23, Bruce Momjian wrote: > Robert Haas wrote: >> [server] >> guc=value >> >> or >> >> server.guc=value >   > > Yes, this was my idea too.  It uses our existing config file format. > So... sync_rep_services = {critical: recv=2, fsync=2, replay=1;

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Bruce Momjian
Robert Haas wrote: > [server] > guc=value > > or > > server.guc=value Yes, this was my idea too. It uses our existing config file format. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible fo

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Simon Riggs
On Tue, 2010-09-21 at 17:04 -0700, Josh Berkus wrote: > > That said, the timeout option also feels a bit wishy-washy to me. With a > > timeout, acknowledgment of a commit means "your transaction is safely > > committed in the master and slave. Or not, if there was some glitch with > > the slave".

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Robert Haas
On Wed, Sep 22, 2010 at 9:01 AM, Andrew Dunstan wrote: > I can't imagine trying to configure Bacula using ini file format - the mind > just boggles. Frankly, I'd rather stick with our current config format than > change to something as inadequate as ini file format. Perhaps we need to define a li

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 08:32 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 1:25 PM, Andrew Dunstan wrote: XML is not the only alternative - please don't use it as a straw man. For example, here is a fragment from the Bacula docs using their hierarchical format: FileSet { Name = Test Include {

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 1:25 PM, Andrew Dunstan wrote: > XML is not the only alternative - please don't use it as a straw man. For > example, here is a fragment from the Bacula docs using their hierarchical > format: > > FileSet { > Name = Test > Include { > File = /home/xxx/test > Opt

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 07:57 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 12:50 PM, Peter Eisentraut wrote: On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers] Count=5 [Servers

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 12:50 PM, Peter Eisentraut wrote: > On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: >> > No, it's really not hierarchical. It only has goes one level deep. >> >> I guess pgAdmin/wxWidgets are broken then :-) >> >> [Servers] >> Count=5 >> [Servers/1] >> Server=localhost

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 07:20 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 12:07 PM, Andrew Dunstan wrote: On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstan wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file f

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Peter Eisentraut
On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: > > No, it's really not hierarchical. It only has goes one level deep. > > I guess pgAdmin/wxWidgets are broken then :-) > > [Servers] > Count=5 > [Servers/1] > Server=localhost Well, by that logic, even what we have now for postgresql.conf is

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 12:07 PM, Andrew Dunstan wrote: > > > On 09/22/2010 04:54 AM, Dave Page wrote: >> >> On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstan >>  wrote: >>> >>> The ini file format is not flexible enough, IMNSHO. If we're going to >>> adopt >>> a new config file format it should hav

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstan wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: well known (let's not invent a new one) sup

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstan wrote: > > The ini file format is not flexible enough, IMNSHO. If we're going to adopt > a new config file format it should have these characteristics, among others: > > well known (let's not invent a new one) > supports hierarchical structure > reas

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 04:18 AM, Heikki Linnakangas wrote: On 21/09/10 18:12, Tom Lane wrote: Heikki Linnakangas writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the u

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Markus Wanner
Hi, On 09/21/2010 08:05 PM, Simon Riggs wrote: > Hmm, no reason? The reason is that the alternative is that the session > would hang until a standby arrived that offered that level of service. > Why would you want that behaviour? Would you really request that option? I think I now agree with Simo

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 21/09/10 18:12, Tom Lane wrote: Heikki Linnakangas writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the user to edit. I'm not a big fan of XML either. ...

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 22/09/10 03:25, Joshua D. Drake wrote: Why is this in the config file at all. It should be: synchronous_replication = TRUE/FALSE Umm, what does this do? then ALTER CLUSTER ENABLE REPLICATION FOR FOO; ALTER CLUSTER SET keep_connect ON FOO TO TRUE; Or some such thing. I like a configura

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Joshua D. Drake
> Crazy idea, but could we use format like postgresql.conf by extending > postgresql.conf syntax, e.g.: > > server1.failover = false > server1.keep_connect = true > Why is this in the config file at all. It should be: synchronous_replication = TRUE/FALSE then ALTER CLUSTER ENABLE

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Josh Berkus
> That said, the timeout option also feels a bit wishy-washy to me. With a > timeout, acknowledgment of a commit means "your transaction is safely > committed in the master and slave. Or not, if there was some glitch with > the slave". That doesn't seem like a very useful guarantee; if you're > ha

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Bruce Momjian
Robert Haas wrote: > On Tue, Sep 21, 2010 at 11:12 AM, Tom Lane wrote: > > Heikki Linnakangas writes: > >> On 21/09/10 11:52, Thom Brown wrote: > >>> My fear would be standby.conf would be edited by users who don't > >>> really know XML and then we'd have 3 different styles of config to > >>> tel

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Simon Riggs
On Tue, 2010-09-21 at 16:58 +0900, Fujii Masao wrote: > On Sat, Sep 18, 2010 at 4:36 AM, Dimitri Fontaine > wrote: > > Simon Riggs writes: > >> On Fri, 2010-09-17 at 21:20 +0900, Fujii Masao wrote: > >>> What synchronization level does each combination of sync_replication > >>> and sync_replicati

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Robert Haas
On Tue, Sep 21, 2010 at 11:12 AM, Tom Lane wrote: > Heikki Linnakangas writes: >> On 21/09/10 11:52, Thom Brown wrote: >>> My fear would be standby.conf would be edited by users who don't >>> really know XML and then we'd have 3 different styles of config to >>> tell the user to edit. > >> I'm no

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Tom Lane
Heikki Linnakangas writes: > On 21/09/10 11:52, Thom Brown wrote: >> My fear would be standby.conf would be edited by users who don't >> really know XML and then we'd have 3 different styles of config to >> tell the user to edit. > I'm not a big fan of XML either. > ... > Then again, maybe we sho

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Heikki Linnakangas
On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the user to edit. I'm not a big fan of XML either. That said, the format could use some hierarchy. If we add many more p

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Fujii Masao
On Mon, Sep 20, 2010 at 3:27 PM, Heikki Linnakangas wrote: > However, the "wait forever" behavior becomes useful if you have a monitoring > application outside the DB that decides when enough is enough and tells the > DB that the slave can be considered dead. So "wait forever" actually means > "wa

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Thom Brown
On 21 September 2010 09:37, Dave Page wrote: > On Tue, Sep 21, 2010 at 9:34 AM, Thom Brown wrote: >> I really don't think an XML config would improve anything.  In fact it >> would just introduce more ways to break the config by the mere fact it >> has to be well-formed.  I'd be in favour of one

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Dave Page
On Tue, Sep 21, 2010 at 9:34 AM, Thom Brown wrote: > I really don't think an XML config would improve anything.  In fact it > would just introduce more ways to break the config by the mere fact it > has to be well-formed.  I'd be in favour of one similar to > pg_hba.conf, because then, at least, w

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Thom Brown
On 21 September 2010 09:29, Fujii Masao wrote: > On Sun, Sep 19, 2010 at 7:20 AM, Robert Haas wrote: >> On Sat, Sep 18, 2010 at 5:42 PM, Josh Berkus wrote: >>> There are considerable benefits to having a standby registry with a >>> table-like interface.  Particularly, one where we could change >

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Fujii Masao
On Sun, Sep 19, 2010 at 7:20 AM, Robert Haas wrote: > On Sat, Sep 18, 2010 at 5:42 PM, Josh Berkus wrote: >> There are considerable benefits to having a standby registry with a >> table-like interface.  Particularly, one where we could change >> replication via UPDATE (or ALTER STANDBY) statement

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Dimitri Fontaine
Robert Haas writes: >> So, here, we have two quite different things to be concerned >> about. First is the configuration, and I say that managing a distributed >> setup will be easier for the DBA. > > Yeah, I disagree with that, but I suppose it's a question of opinion. I'd be willing to share yo

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Fujii Masao
On Sat, Sep 18, 2010 at 4:36 AM, Dimitri Fontaine wrote: > Simon Riggs writes: >> On Fri, 2010-09-17 at 21:20 +0900, Fujii Masao wrote: >>> What synchronization level does each combination of sync_replication >>> and sync_replication_service lead to? >> >> There are only 4 possible outcomes. Ther

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Simon Riggs
On Mon, 2010-09-20 at 22:42 +0100, Thom Brown wrote: > On 20 September 2010 22:14, Robert Haas wrote: > > Well, if you need to talk to "all the other standbys" and see who has > > the furtherst-advanced xlog pointer, it seems like you have to have a > > list somewhere of who they all are. > > Whe

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Robert Haas
On Mon, Sep 20, 2010 at 5:42 PM, Thom Brown wrote: > On 20 September 2010 22:14, Robert Haas wrote: >> Well, if you need to talk to "all the other standbys" and see who has >> the furtherst-advanced xlog pointer, it seems like you have to have a >> list somewhere of who they all are. > > When the

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Thom Brown
On 20 September 2010 22:14, Robert Haas wrote: > Well, if you need to talk to "all the other standbys" and see who has > the furtherst-advanced xlog pointer, it seems like you have to have a > list somewhere of who they all are. When they connect to the master to get the stream, don't they in eff

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Robert Haas
On Mon, Sep 20, 2010 at 4:10 PM, Dimitri Fontaine wrote: > Robert Haas writes: >>   So the "wait forever" case is, in my opinion, >> sufficient to demonstrate that we need it, but it's not even my >> primary reason for wanting to have it. > > You're talking about standby registration on the maste

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Dimitri Fontaine
Hi, I'm somewhat sorry to have to play this game, as I sure don't feel smarter by composing this email. Quite the contrary. Robert Haas writes: > So the "wait forever" case is, in my opinion, > sufficient to demonstrate that we need it, but it's not even my > primary reason for wanting to have

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Robert Haas
On Mon, Sep 20, 2010 at 8:50 AM, Simon Riggs wrote: > Please respond to the main point: Following some thought and analysis, > AFAICS there is no sensible use case that requires standby registration. I disagree. You keep analyzing away the cases that require standby registration, but I don't bel

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Heikki Linnakangas
On 20/09/10 15:50, Simon Riggs wrote: On Mon, 2010-09-20 at 15:16 +0300, Heikki Linnakangas wrote: On 20/09/10 12:17, Simon Riggs wrote: err... what is the difference between a timeout and stonith? STONITH ("Shoot The Other Node In The Head") means that the other node is somehow disabled so t

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Simon Riggs
On Mon, 2010-09-20 at 15:16 +0300, Heikki Linnakangas wrote: > On 20/09/10 12:17, Simon Riggs wrote: > > err... what is the difference between a timeout and stonith? > > STONITH ("Shoot The Other Node In The Head") means that the other node > is somehow disabled so that it won't unexpectedly come

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Heikki Linnakangas
On 20/09/10 12:17, Simon Riggs wrote: err... what is the difference between a timeout and stonith? STONITH ("Shoot The Other Node In The Head") means that the other node is somehow disabled so that it won't unexpectedly come back alive. A timeout means that the slave hasn't been seen for a wh

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Simon Riggs
On Mon, 2010-09-20 at 09:27 +0300, Heikki Linnakangas wrote: > On 18/09/10 22:59, Robert Haas wrote: > > On Sat, Sep 18, 2010 at 4:50 AM, Simon Riggs wrote: > >> Waiting might sound attractive. In practice, waiting will make all of > >> your connections lock up and it will look to users as if thei

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Markus Wanner
Hi, On 09/17/2010 01:56 PM, Fujii Masao wrote: And standby registration is required when we support "wait forever when synchronous standby isn't connected at the moment" option that Heikki explained upthread. That requirement can be reduced to say that the master only needs to known how many

Re: [HACKERS] Configuring synchronous replication

2010-09-19 Thread Heikki Linnakangas
On 19/09/10 01:20, Robert Haas wrote: On Sat, Sep 18, 2010 at 5:42 PM, Josh Berkus wrote: There are considerable benefits to having a standby registry with a table-like interface. Particularly, one where we could change replication via UPDATE (or ALTER STANDBY) statements. I think that using

Re: [HACKERS] Configuring synchronous replication

2010-09-19 Thread Heikki Linnakangas
On 18/09/10 22:59, Robert Haas wrote: On Sat, Sep 18, 2010 at 4:50 AM, Simon Riggs wrote: Waiting might sound attractive. In practice, waiting will make all of your connections lock up and it will look to users as if their master has stopped working as well. (It has!). I can't imagine why anyon

Re: [HACKERS] Configuring synchronous replication

2010-09-19 Thread Josh Berkus
> I've designed a way to tune sync rep so it is usable and useful. And > putting that feature into 9.1 costs very little, if anything. My patch > to do this is actually smaller than any other attempt to implement this > and I claim faster too. You don't need to use the per-transaction > controls,

  1   2   >