Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-17 Thread Kyotaro HORIGUCHI
Thank you for committing. > On 02/14/2014 10:38 AM, Kyotaro HORIGUCHI wrote: > > Finally, the patch you will find attached is fixed only in > > styling mentioned above from your last patch. This patch applies > > current HEAD and I confirmed that it fixes this issue but I have > > not checked the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-14 Thread Heikki Linnakangas
On 02/14/2014 10:38 AM, Kyotaro HORIGUCHI wrote: Finally, the patch you will find attached is fixed only in styling mentioned above from your last patch. This patch applies current HEAD and I confirmed that it fixes this issue but I have not checked the lastSourceFailed section. Simple file remov

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-14 Thread Kyotaro HORIGUCHI
Hello, Before taking up the topic.. At Thu, 13 Feb 2014 19:45:38 +0200, Heikki Linnakangas wrote > On 02/13/2014 06:47 PM, Heikki Linnakangas wrote: > > On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: > >> The behavior where we prefer a segment from archive with lower TLI > >> over > >> a file

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 06:47 PM, Heikki Linnakangas wrote: On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: The behavior where we prefer a segment from archive with lower TLI over a file with higher TLI in pg_xlog actually changed in commit a068c391ab0. Arguably changing it wasn't a good idea, but the p

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: The behavior where we prefer a segment from archive with lower TLI over a file with higher TLI in pg_xlog actually changed in commit a068c391ab0. Arguably changing it wasn't a good idea, but the problem your test script demonstrates can be fixed b

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 04:07 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-02-13 <52fcd02c.3060...@vmware.com> Is removing the "test ! -f" part and hence overwriting files in the archive safe, i.e. are the files the same? No. Not in general, anyway. If the old master keeps running, even for a m

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Christoph Berg
Re: Heikki Linnakangas 2014-02-13 <52fcd02c.3060...@vmware.com> > >Is removing the "test ! -f" part and hence overwriting files in the > >archive safe, i.e. are the files the same? > > No. Not in general, anyway. If the old master keeps running, even > for a moment, after the partial file was copi

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 03:53 PM, Christoph Berg wrote: Is removing the "test ! -f" part and hence overwriting files in the archive safe, i.e. are the files the same? No. Not in general, anyway. If the old master keeps running, even for a moment, after the partial file was copied, it will have created m

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Christoph Berg
Re: Heikki Linnakangas 2014-02-13 <52fcca40.3060...@vmware.com> > I was testing this with streaming replication; 9.1 and 9.2 behave > the same in that scenario. But they differ when doing archive > recovery. > > Is this an argument for back-patching the "don't archive last > segment from old timel

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 02:42 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-02-13 <52fc9468.4050...@vmware.com> With 9.1, it works, but 9.2 and 9.3 don't archive anything until I remove the "test ! -f" part. (An alternative fix would be to declare the behavior ok and adjust that example in the con

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Christoph Berg
Re: Heikki Linnakangas 2014-02-13 <52fc9468.4050...@vmware.com> > >With 9.1, it works, but 9.2 and 9.3 don't archive anything until I > >remove the "test ! -f" part. (An alternative fix would be to declare > >the behavior ok and adjust that example in the config.) > > Hmm, the behavior is the same

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 01:37 PM, Kyotaro HORIGUCHI wrote: At Thu, 13 Feb 2014 10:11:22 +0200, Heikki Linnakangas wrote in <52fc7e2a.9060...@vmware.com> On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: Wouldn't it be better to not archive the old segment, and instead switch to a new segment after writi

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Kyotaro HORIGUCHI
Hello, I might have been misunderstood your words. At Thu, 13 Feb 2014 10:11:22 +0200, Heikki Linnakangas wrote in <52fc7e2a.9060...@vmware.com> > On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: > > Wouldn't it be better to not archive the old segment, and instead > > switch > > to

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/12/2014 01:24 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-01-13 <52d3caff.3010...@vmware.com> Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the or

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: Wouldn't it be better to not archive the old segment, and instead switch to a new segment after writing the end-of-recovery checkpoint, so that the segment on the new timeline is archived sooner? It would be better to zero-fill and switch segment

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Kyotaro HORIGUCHI
Hello, > I need PostgreSQL9.3 which fixed this problem. > > It didn't happen in PostgreSQL9.2, so I agree > with your proposal which changes are done > against 93_STABLE and master. > > Can you fix this in next release(9.3.3)? I was going to push to move this a bit, but... > >>> Wouldn't it be

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Christoph Berg
Re: Heikki Linnakangas 2014-01-13 <52d3caff.3010...@vmware.com> > >>Actually, why is the partially-filled 00010002 file > >>archived in the first place? Looking at the code, it's been like that > >>forever, but it seems like a bad idea. If the original server is still > >>up and run

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Tomonari Katsumata
Hi Heikki, I need PostgreSQL9.3 which fixed this problem. It didn't happen in PostgreSQL9.2, so I agree with your proposal which changes are done against 93_STABLE and master. Can you fix this in next release(9.3.3)? Tomonari Katsumata (2014/01/13 20:16), Heikki Linnakangas wrote

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-13 Thread Heikki Linnakangas
On 01/09/2014 10:55 PM, Josh Berkus wrote: On 01/09/2014 12:05 PM, Heikki Linnakangas wrote: Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the original server is

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Josh Berkus
On 01/09/2014 12:05 PM, Heikki Linnakangas wrote: > Actually, why is the partially-filled 00010002 file > archived in the first place? Looking at the code, it's been like that > forever, but it seems like a bad idea. If the original server is still > up and running, and writing mor

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 10:36 PM, Tom Lane wrote: Heikki Linnakangas writes: On 01/09/2014 10:16 PM, Tom Lane wrote: Don't we want to archive both? If you want to recover to the end of the old timeline, you're going to need that file too, no? Hmm. It should be the responsibility of the original serv

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tom Lane
Heikki Linnakangas writes: > On 01/09/2014 10:16 PM, Tom Lane wrote: >> Don't we want to archive both? If you want to recover to the end of the >> old timeline, you're going to need that file too, no? > Hmm. It should be the responsibility of the original server to archive > the segment on the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 10:16 PM, Tom Lane wrote: Heikki Linnakangas writes: Actually, why is the partially-filled 00010002 file archived in the first place? ... So, the rationale is that otherwise it would take a long time until that segment is archived. To be precise, I don't think t

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tom Lane
Heikki Linnakangas writes: > Actually, why is the partially-filled 00010002 file > archived in the first place? ... > So, the rationale is that otherwise it would take a long time until that > segment is archived. To be precise, I don't think the segment with the > old TLI woul

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 08:18 PM, Heikki Linnakangas wrote: On 12/12/2013 04:00 AM, Kyotaro HORIGUCHI wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for th

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 12/12/2013 04:00 AM, Kyotaro HORIGUCHI wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for that of the same segment id but for newer timeline. A

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Amit Langote
On Thu, Dec 12, 2013 at 11:00 AM, Kyotaro HORIGUCHI wrote: > Hello, we happened to see server crash on archive recovery under > some condition. > > After TLI was incremented, there should be the case that the WAL > file for older timeline is archived but not for that of the same > segment id but f

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tomonari Katsumata
Hi, Somebody is reading this thread? This problem seems still remaining on REL9_3_STABLE. Many users would face this problem, so we should resolve this in next release. I think his patch is reasonable to fix this problem. Please check this again. regards, -- Tomonari Ka

[HACKERS] [BUG] Archive recovery failure on 9.3+.

2013-12-11 Thread Kyotaro HORIGUCHI
Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for that of the same segment id but for newer timeline. Archive recovery should fail for the case with PANIC