Re: [HACKERS] Switching timeline over streaming replication

2012-12-26 Thread Heikki Linnakangas
On 23.12.2012 16:37, Fujii Masao wrote: On Fri, Dec 21, 2012 at 1:48 AM, Fujii Masao wrote: On Sat, Dec 15, 2012 at 9:36 AM, Fujii Masao wrote: I found another "requested timeline does not contain minimum recovery point" error scenario in HEAD: 1. Set up the master 'M', one standby 'S1', and

Re: [HACKERS] Switching timeline over streaming replication

2012-12-23 Thread Fujii Masao
On Fri, Dec 21, 2012 at 1:48 AM, Fujii Masao wrote: > On Sat, Dec 15, 2012 at 9:36 AM, Fujii Masao wrote: >> On Sat, Dec 8, 2012 at 12:51 AM, Heikki Linnakangas >> wrote: >>> On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: >

Re: [HACKERS] Switching timeline over streaming replication

2012-12-21 Thread Thom Brown
On 21 December 2012 18:13, Heikki Linnakangas wrote: > On 21.12.2012 01:50, Thom Brown wrote: > >> Now I'm getting this on all standbys after promoting the first standby in >> a >> chain. >> > > ... > > > TRAP: FailedAssertion("!(((sentPtr)<**= (SendRqstPtr)))", File: > > "walsender.c", Line: 1425

Re: [HACKERS] Switching timeline over streaming replication

2012-12-21 Thread Heikki Linnakangas
On 21.12.2012 01:50, Thom Brown wrote: Now I'm getting this on all standbys after promoting the first standby in a chain. > ... > TRAP: FailedAssertion("!(((sentPtr)<= (SendRqstPtr)))", File: > "walsender.c", Line: 1425) Sigh. I'm sounding like a broken record, but I just committed another fix

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Thom Brown
On 20 December 2012 12:45, Heikki Linnakangas wrote: > On 17.12.2012 15:05, Thom Brown wrote: > >> I just set up 120 chained standbys, and for some reason I'm seeing these >> errors: >> >> LOG: replication terminated by primary server >> DETAIL: End of WAL reached on timeline 1 >> LOG: record w

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Joshua Berkus
> I just committed a patch that should make the "requested WAL segment > 00020003 has already been removed" errors go away. > The > trick was for walsenders to not switch to the new timeline until at > least one record has been replayed on it. That closes the window > where > the

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Fujii Masao
On Sat, Dec 15, 2012 at 9:36 AM, Fujii Masao wrote: > On Sat, Dec 8, 2012 at 12:51 AM, Heikki Linnakangas > wrote: >> On 06.12.2012 15:39, Amit Kapila wrote: >>> >>> On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit Kapila wrote: > > On

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Andres Freund
On 2012-12-20 14:45:05 +0200, Heikki Linnakangas wrote: > On 17.12.2012 15:05, Thom Brown wrote: > >I just set up 120 chained standbys, and for some reason I'm seeing these > >errors: > > > >LOG: replication terminated by primary server > >DETAIL: End of WAL reached on timeline 1 > >LOG: record

Re: [HACKERS] Switching timeline over streaming replication

2012-12-20 Thread Heikki Linnakangas
On 17.12.2012 15:05, Thom Brown wrote: I just set up 120 chained standbys, and for some reason I'm seeing these errors: LOG: replication terminated by primary server DETAIL: End of WAL reached on timeline 1 LOG: record with zero length at 0/301EC10 LOG: fetching timeline history file for tim

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Joshua Berkus
Heikki, The next time I get the issue, and I'm not paying for 5 cloud servers by the hour, I'll give you a login. --Josh - Original Message - > On 19.12.2012 17:27, Heikki Linnakangas wrote: > > On 19.12.2012 15:55, Heikki Linnakangas wrote: > >> On 19.12.2012 04:57, Josh Berkus wrote:

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Joshua Berkus
Heikki, > The problem goes away after some time, after the 1st standby has > streamed the contents of 00020003 and written it to > disk, and the cascaded standby reconnects. But it would be nice to > avoid > that situation. I'm not sure how to do that yet, we might need to > track

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Heikki Linnakangas
On 19.12.2012 17:27, Heikki Linnakangas wrote: On 19.12.2012 15:55, Heikki Linnakangas wrote: On 19.12.2012 04:57, Josh Berkus wrote: Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran in

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Heikki Linnakangas
On 19.12.2012 15:55, Heikki Linnakangas wrote: On 19.12.2012 04:57, Josh Berkus wrote: Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran into a weird issue when starting up "r3": it refus

Re: [HACKERS] Switching timeline over streaming replication

2012-12-19 Thread Heikki Linnakangas
On 19.12.2012 04:57, Josh Berkus wrote: Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran into a weird issue when starting up "r3": it refused to come out of "the database is starting up"

Re: [HACKERS] Switching timeline over streaming replication

2012-12-18 Thread Josh Berkus
Heikki, I ran into an unexpected issue while testing. I just wanted to fire up a chain of 5 replicas to see if I could connect them in a loop. However, I ran into a weird issue when starting up "r3": it refused to come out of "the database is starting up" mode until I did a write on the master.

Re: [HACKERS] Switching timeline over streaming replication

2012-12-17 Thread Josh Berkus
Since Thom already did the destruction test, I only chained 7 standbies, just to see if I could reproduce his error. In the process, I accidentally connected one standby to itself. This failed, but the error message wasn't very helpful; it just gave me "FATAL: could not connect, the database syste

Re: [HACKERS] Switching timeline over streaming replication

2012-12-17 Thread Thom Brown
On 17 December 2012 12:07, Heikki Linnakangas wrote: > On 15.12.2012 01:09, Josh Berkus wrote: > >> Tested this on yesterday's snapshot. Worked great. >> > > Thanks for the testing! > > > Now I wanna test a chain of cascading replicas ... how far can we chain >> these? >> > > There's no limit in

Re: [HACKERS] Switching timeline over streaming replication

2012-12-17 Thread Heikki Linnakangas
On 15.12.2012 01:09, Josh Berkus wrote: Tested this on yesterday's snapshot. Worked great. Thanks for the testing! Now I wanna test a chain of cascading replicas ... how far can we chain these? There's no limit in theory. I tested with one master and two chained standbys myself. Give it a

Re: [HACKERS] Switching timeline over streaming replication

2012-12-14 Thread Fujii Masao
On Sat, Dec 8, 2012 at 12:51 AM, Heikki Linnakangas wrote: > On 06.12.2012 15:39, Amit Kapila wrote: >> >> On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: >>> >>> On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: >>

Re: [HACKERS] Switching timeline over streaming replication

2012-12-14 Thread Josh Berkus
Heikki, Tested this on yesterday's snapshot. Worked great. Test: 4 Ubuntu 10.04 LTS Cloud Servers (GoGrid) Configuration: Compiled 9.3(12-12-12) with: pg_stat_statements, citext, ISN, btree_gist, pl/perl Setup Test: Master-Master Replicated to: master-replica using pg_basebackup -x.

Re: [HACKERS] Switching timeline over streaming replication

2012-12-10 Thread Amit Kapila
> From: Heikki Linnakangas [mailto:hlinnakan...@vmware.com] > Sent: Friday, December 07, 2012 9:22 PM > To: Amit Kapila > Cc: 'PostgreSQL-development'; 'Thom Brown' > Subject: Re: [HACKERS] Switching timeline over streaming replication > > On 06.12.2012

Re: [HACKERS] Switching timeline over streaming replication

2012-12-07 Thread Heikki Linnakangas
On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a coup

Re: [HACKERS] Switching timeline over streaming replication

2012-12-06 Thread Amit Kapila
On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: > On 05.12.2012 14:32, Amit Kapila wrote: > > On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: > >> After some diversions to fix bugs and refactor existing code, I've > >> committed a couple of small parts of this pat

Re: [HACKERS] Switching timeline over streaming replication

2012-12-05 Thread Heikki Linnakangas
On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch, which just add some sanity checks to notice incorrect PITR scenarios. Here'

Re: [HACKERS] Switching timeline over streaming replication

2012-12-05 Thread Amit Kapila
On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: > After some diversions to fix bugs and refactor existing code, I've > committed a couple of small parts of this patch, which just add some > sanity checks to notice incorrect PITR scenarios. Here's a new version > of the main patch b

Re: [HACKERS] Switching timeline over streaming replication

2012-12-04 Thread Heikki Linnakangas
After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch, which just add some sanity checks to notice incorrect PITR scenarios. Here's a new version of the main patch based on current HEAD. - Heikki streaming-tli-switch-8.patch.gz Des

Re: [HACKERS] Switching timeline over streaming replication

2012-12-03 Thread Heikki Linnakangas
On 03.12.2012 14:21, senthilnathan wrote: Is this patch available in version 9.2.1 ? Nope, this is for 9.3. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Switching timeline over streaming replication

2012-12-03 Thread senthilnathan
Is this patch available in version 9.2.1 ? Senthil -- View this message in context: http://postgresql.1045698.n5.nabble.com/Switching-timeline-over-streaming-replication-tp5723547p5734744.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via pgsql-hackers

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-27 Thread Heikki Linnakangas
On 26.11.2012 14:53, Amit Kapila wrote: On Friday, November 23, 2012 7:03 PM Heikki Linnakangas This is what I came up with. It adds a new function, OpenFile, that returns a raw file descriptor like BasicOpenFile, but the file descriptor is associated with the current subtransaction and automati

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Tom Lane
Amit Kapila writes: > On Monday, November 26, 2012 7:01 PM Heikki Linnakangas wrote: >> Hmm, if it's just for locking purposes, how about using a lwlock or a >> heavy-weight lock instead? > Its not only for lock, the main idea is that we create temp file and write > modified configuration in that

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Amit Kapila
On Monday, November 26, 2012 7:01 PM Heikki Linnakangas wrote: > On 26.11.2012 14:53, Amit Kapila wrote: > > I have one usecase in feature (SQL Command to edit postgresql.conf) > very > > similar to OpenFile/CloseFile, but I want that when CloseFile is > called from > > abort, it should remove(unli

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Heikki Linnakangas
On 26.11.2012 14:53, Amit Kapila wrote: I have one usecase in feature (SQL Command to edit postgresql.conf) very similar to OpenFile/CloseFile, but I want that when CloseFile is called from abort, it should remove(unlink) the file as well and during open it has to retry few times if open is not s

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Amit Kapila
On Friday, November 23, 2012 7:03 PM Heikki Linnakangas > On 15.11.2012 17:16, Heikki Linnakangas wrote: > > On 15.11.2012 16:55, Tom Lane wrote: > >> Heikki Linnakangas writes: > >>> This is a fairly general issue, actually. Looking around, I can see > >>> at least two similar cases in existing co

Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-23 Thread Heikki Linnakangas
On 15.11.2012 17:16, Heikki Linnakangas wrote: On 15.11.2012 16:55, Tom Lane wrote: Heikki Linnakangas writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file descriptors on error: Um,

Re: [HACKERS] Switching timeline over streaming replication

2012-11-22 Thread Amit Kapila
On Wednesday, November 21, 2012 11:36 PM Heikki Linnakangas wrote: > On 20.11.2012 15:33, Amit Kapila wrote: > > Defect-2: > > 1. start primary A > > 2. start standby B following A > > 3. start cascade standby C following B. > > 4. Start another standby D following C. > > 5

Re: [HACKERS] Switching timeline over streaming replication

2012-11-21 Thread Heikki Linnakangas
On 20.11.2012 15:33, Amit Kapila wrote: Defect-2: 1. start primary A 2. start standby B following A 3. start cascade standby C following B. 4. Start another standby D following C. 5. Execute the following commands in the primary A. create table tbl(f int);

Re: [HACKERS] Switching timeline over streaming replication

2012-11-20 Thread Amit Kapila
On Monday, November 19, 2012 10:54 PM Heikki Linnakangas wrote: > On 10.10.2012 17:54, Thom Brown wrote: > > Hmm... I get something different. When I promote standby B, standby > > C's log shows: > > > > The following problems are observed while testing of the patch. > > Defect-1: > > > >

Re: [HACKERS] Switching timeline over streaming replication

2012-11-19 Thread Heikki Linnakangas
On 10.10.2012 17:54, Thom Brown wrote: > Hmm... I get something different. When I promote standby B, standby > C's log shows: > > LOG: walreceiver ended streaming and awaits new instructions > LOG: re-handshaking at position 0/400 on tli 1 > LOG: fetching timeline history file for timeline

Re: [HACKERS] Switching timeline over streaming replication

2012-11-16 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: > On 15.11.2012 12:44, Heikki Linnakangas wrote: > > Here's an updated version of this patch, rebased with master, > > including the recent replication timeout changes, and some other > cleanup. > > > > On 12.10.2012 09:34, Amit Kapil

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 16:55, Tom Lane wrote: Heikki Linnakangas writes: This is a fairly general issue, actually. Looking around, I can see at least two similar cases in existing code, with BasicOpenFile, where we will leak file descriptors on error: Um, don't we automatically clean those up during t

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Tom Lane
Heikki Linnakangas writes: > This is a fairly general issue, actually. Looking around, I can see at > least two similar cases in existing code, with BasicOpenFile, where we > will leak file descriptors on error: Um, don't we automatically clean those up during transaction abort? If we don't, we

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: > On 15.11.2012 12:44, Heikki Linnakangas wrote: > > Here's an updated version of this patch, rebased with master, > > including the recent replication timeout changes, and some other > cleanup. > > > > On 12.10.2012 09:34, Amit Kapil

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: > ... ./pg_basebac

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
On 10.10.2012 17:26, Amit Kapila wrote: 36.+SendTimeLineHistory(TimeLineHistoryCmd *cmd) { .. if (nread<= 0) +ereport(ERROR, +(errcode_for_file_access(), + errmsg("could not read file \"%s\

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Heikki Linnakangas
Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote: The test is finished from myside. one more issue: > ... ./pg_basebackup -P -D ../../data_sub -X fetch -p 2303 pg_bas

Re: [HACKERS] Switching timeline over streaming replication

2012-10-23 Thread Alvaro Herrera
Heikki Linnakangas wrote: > Attached is a new version of the patch. I committed the refactoring > of XLogPageRead() already, as that was a readability improvement > even without this patch. All the reported issues should be fixed > now, although I will continue testing this tomorrow. I added vario

Re: [HACKERS] Switching timeline over streaming replication

2012-10-11 Thread Amit Kapila
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- > ow...@postgresql.org] On Behalf Of Amit Kapila > Sent: Wednesday, October 10, 2012 7:57 PM > To: 'Heikki Linnakangas' > Cc: 'PostgreSQL-development' > Subje

Re: [HACKERS] Switching timeline over streaming replication

2012-10-10 Thread Thom Brown
On 10 October 2012 15:26, Amit Kapila wrote: > On Tuesday, October 09, 2012 10:32 PM Heikki Linnakangas wrote: >> On 06.10.2012 15:58, Amit Kapila wrote: >> > One more test seems to be failed. Apart from this, other tests are >> passed. >> > > It seems there is one more defect, please check the s

Re: [HACKERS] Switching timeline over streaming replication

2012-10-10 Thread Amit Kapila
On Tuesday, October 09, 2012 10:32 PM Heikki Linnakangas wrote: > On 06.10.2012 15:58, Amit Kapila wrote: > > One more test seems to be failed. Apart from this, other tests are > passed. > > It seems there is one more defect, please check the same Defect: 1. start primary A 2. s

Re: [HACKERS] Switching timeline over streaming replication

2012-10-09 Thread Heikki Linnakangas
On 06.10.2012 15:58, Amit Kapila wrote: One more test seems to be failed. Apart from this, other tests are passed. 2. a. Master M-1 b. Standby S-1 follows M-1 c. insert 10 records on M-1. verify all records are visible on M-1,S-1 d. Stop S-1 e. insert 2 records on M-1. f. Sto

Re: Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-08 Thread Simon Riggs
On 4 October 2012 18:07, Fujii Masao wrote: > On Thu, Oct 4, 2012 at 4:59 PM, Heikki Linnakangas > wrote: >> On 03.10.2012 18:15, Amit Kapila wrote: >>> >>> On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Hmm, should a base backup be aborted when the standby is promoted?

Re: Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-08 Thread Heikki Linnakangas
On 04.10.2012 20:07, Fujii Masao wrote: On Thu, Oct 4, 2012 at 4:59 PM, Heikki Linnakangas But I wonder why promoting a standby renders the backup invalid in the first place? Fujii, Simon, can you explain that? Simon had the same question and I answered it before. http://archives.postgresql.o

Re: [HACKERS] Switching timeline over streaming replication

2012-10-06 Thread Amit Kapila
On Thursday, October 04, 2012 7:22 PM Heikki Linnakangas wrote: > > On Wednesday, October 03, 2012 8:45 PM Heikki Linnakangas wrote: > > On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: > > > Thanks for the thorough review! I committed the xlog.c refactoring > > patch > > > now. Attac

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Amit Kapila
On Thursday, October 04, 2012 8:40 PM Heikki Linnakangas wrote: > On 03.10.2012 18:15, Amit Kapila wrote: > > 35.WalSenderMain(void) > > { > > .. > > +if (walsender_shutdown_requested) > > +ereport(FATAL, > > + > (errcode(ERRCODE_ADMIN_SHUTDOWN), > > +

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Tom Lane
Simon Riggs writes: > On 4 October 2012 17:23, Heikki Linnakangas wrote: >> Perhaps we could make walsenders even more like regular backends than what I >> was proposing, so that the replication commands are parsed and executed just >> like regular utility commands. However, that'd require some t

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Simon Riggs
On 4 October 2012 17:23, Heikki Linnakangas wrote: > On 04.10.2012 19:00, Tom Lane wrote: >> >> Heikki Linnakangas writes: >>> >>> So I propose the attached patch. I made small changes to postgres.c to >>> make it call exec_replication_command() instead of exec_simple_query(), >>> and reject exte

Re: Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Fujii Masao
On Thu, Oct 4, 2012 at 4:59 PM, Heikki Linnakangas wrote: > On 03.10.2012 18:15, Amit Kapila wrote: >> >> On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: >>> >>> Hmm, should a base backup be aborted when the standby is promoted? Does >>> the promotion render the backup corrupt? >> >

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Heikki Linnakangas
On 04.10.2012 19:00, Tom Lane wrote: Heikki Linnakangas writes: So I propose the attached patch. I made small changes to postgres.c to make it call exec_replication_command() instead of exec_simple_query(), and reject extend query protocol, in a WAL sender process. A lot of code related to hand

Re: Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Tom Lane
Heikki Linnakangas writes: > So I propose the attached patch. I made small changes to postgres.c to > make it call exec_replication_command() instead of exec_simple_query(), > and reject extend query protocol, in a WAL sender process. A lot of code > related to handling the main command loop an

Sharing more infrastructure between walsenders and regular backends (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Heikki Linnakangas
On 03.10.2012 18:15, Amit Kapila wrote: 35.WalSenderMain(void) { .. +if (walsender_shutdown_requested) +ereport(FATAL, +(errcode(ERRCODE_ADMIN_SHUTDOWN), + errmsg("terminating r

Re: [HACKERS] Switching timeline over streaming replication

2012-10-04 Thread Amit Kapila
> On Wednesday, October 03, 2012 8:45 PM Heikki Linnakangas wrote: > On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: > > Thanks for the thorough review! I committed the xlog.c refactoring > patch > > now. Attached is a new version of the main patch, comments on specific > > points be

Promoting a standby during base backup (was Re: [HACKERS] Switching timeline over streaming replication)

2012-10-04 Thread Heikki Linnakangas
On 03.10.2012 18:15, Amit Kapila wrote: On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: Hmm, should a base backup be aborted when the standby is promoted? Does the promotion render the backup corrupt? I think currently it does so. Pls refer 1. do_pg_stop_backup(char *labelfile,

Re: [HACKERS] Switching timeline over streaming replication

2012-10-03 Thread Amit Kapila
On Tuesday, October 02, 2012 4:21 PM Heikki Linnakangas wrote: > Thanks for the thorough review! I committed the xlog.c refactoring patch > now. Attached is a new version of the main patch, comments on specific > points below. I didn't adjust the docs per your comments yet, will do > that next. I

Re: [HACKERS] Switching timeline over streaming replication

2012-09-30 Thread Amit kapila
> On Friday, September 28, 2012 6:38 PM Amit Kapila wrote: > On Tuesday, September 25, 2012 6:29 PM Heikki Linnakangas wrote: > On 25.09.2012 10:08, Heikki Linnakangas wrote: > > On 24.09.2012 16:33, Amit Kapila wrote: > >> In any case, it will be better if you can split it into multiple > patches:

Re: [HACKERS] Switching timeline over streaming replication

2012-09-28 Thread Amit Kapila
> On Tuesday, September 25, 2012 6:29 PM Heikki Linnakangas wrote: > On 25.09.2012 10:08, Heikki Linnakangas wrote: > > On 24.09.2012 16:33, Amit Kapila wrote: > >> In any case, it will be better if you can split it into multiple > patches: > >> 1. Having new functionality of "Switching timeline ov

Re: [HACKERS] Switching timeline over streaming replication

2012-09-27 Thread Euler Taveira
On 27-09-2012 01:30, Amit Kapila wrote: > I understood this point, but currently in documentation of Timelines, this > usecase is not documented (Section 24.3.5). > Timeline documentation was written during PITR implementation. There wasn't SR yet. AFAICS it doesn't cite SR but is sufficiently ge

Re: [HACKERS] Switching timeline over streaming replication

2012-09-27 Thread Hannu Krosing
On 09/26/2012 01:02 AM, m...@rpzdesign.com wrote: John: Who has the money for oracle RAC or funding arrogant bastard Oracle CEO Ellison to purchase another island? Postgres needs CHEAP, easy to setup, self healing, master-master-master-master and it needs it yesterday. I was able to patch

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread Amit Kapila
On Thursday, September 27, 2012 6:30 AM Josh Berkus wrote: > > Yes that is correct. I thought timeline change happens only when > somebody > > does PITR. > > Can you please tell me why we change timeline after promotion, > because the > > original > > Timeline concept was for PITR and I am no

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread m...@rpzdesign.com
Josh: The good part is you are the first person to ask for a copy and I will send you the hook code that I have and you can be a good sport and put it on GitHub, that is great, you can give us both credit for a joint effort, I do the code, you put it GitHub. The not so good part is that the co

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread Josh Berkus
> Yes that is correct. I thought timeline change happens only when somebody > does PITR. > Can you please tell me why we change timeline after promotion, because the > original > Timeline concept was for PITR and I am not able to trace from code the > reason > why on promotion it is requi

Re: [HACKERS] Switching timeline over streaming replication

2012-09-26 Thread Josh Berkus
> I was able to patch the 9.2.0 code base in 1 day and change my entire > architecture strategy for replication > into self healing async master-master-master and the tiniest bit of > sharding code imaginable Sounds cool. Do you have a fork available on Github? I'll try it out. -- Josh Berku

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread m...@rpzdesign.com
John: Who has the money for oracle RAC or funding arrogant bastard Oracle CEO Ellison to purchase another island? Postgres needs CHEAP, easy to setup, self healing, master-master-master-master and it needs it yesterday. I was able to patch the 9.2.0 code base in 1 day and change my entire

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread John R Pierce
On 09/25/12 11:01 AM, m...@rpzdesign.com wrote: At some point, every master - slave replicator gets to the point where they need to start thinking about master-master replication. master-master and transactional integrity are mutually exclusive, except perhaps in special cases like Oracle

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Daniel Farina
On Tue, Sep 25, 2012 at 11:01 AM, m...@rpzdesign.com wrote: > Amit: > > At some point, every master - slave replicator gets to the point where they > need > to start thinking about master-master replication. Even in a master-master system, the ability to cleanly swap leaders managing a member of

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread m...@rpzdesign.com
Amit: At some point, every master - slave replicator gets to the point where they need to start thinking about master-master replication. Instead of getting stuck in the weeds to finally realize that master-master is the ONLY way to go, many developers do not start out planning for master - m

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Amit Kapila
> On Tuesday, September 25, 2012 6:29 PM Heikki Linnakangas wrote: > On 25.09.2012 10:08, Heikki Linnakangas wrote: > > On 24.09.2012 16:33, Amit Kapila wrote: > >> In any case, it will be better if you can split it into multiple > patches: > >> 1. Having new functionality of "Switching timeline ov

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Heikki Linnakangas
On 25.09.2012 14:10, Amit Kapila wrote: On Tuesday, September 25, 2012 12:39 PM Heikki Linnakangas wrote: On 24.09.2012 16:33, Amit Kapila wrote: On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: I've been working on the often-requested feature to handle timeline changes ove

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Amit Kapila
On Tuesday, September 25, 2012 12:39 PM Heikki Linnakangas wrote: > On 24.09.2012 16:33, Amit Kapila wrote: > > On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: > >> I've been working on the often-requested feature to handle timeline > >> changes over streaming replication. At th

Re: [HACKERS] Switching timeline over streaming replication

2012-09-25 Thread Heikki Linnakangas
On 24.09.2012 16:33, Amit Kapila wrote: On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: I've been working on the often-requested feature to handle timeline changes over streaming replication. At the moment, if you kill the master and promote a standby server, and you have anoth

Re: [HACKERS] Switching timeline over streaming replication

2012-09-24 Thread Amit Kapila
> On Monday, September 24, 2012 9:08 PM m...@rpzdesign.com wrote: > What a disaster waiting to happen. Maybe the only replication should be > master-master replication > so there is no need to sequence timelines or anything, all servers are > ready masters, no backups or failovers. > If you really

Re: [HACKERS] Switching timeline over streaming replication

2012-09-24 Thread Amit Kapila
On Tuesday, September 11, 2012 10:53 PM Heikki Linnakangas wrote: > I've been working on the often-requested feature to handle timeline > changes over streaming replication. At the moment, if you kill the > master and promote a standby server, and you have another standby > server that you'd like t