Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-15 Thread Andres Freund
On 2014-02-12 13:33:31 +0100, Andres Freund wrote: > On 2014-02-12 21:23:54 +0900, MauMau wrote: > > Maybe we could consider in that direction, but there is a problem. Archive > > recovery slows down compared to 9.1, because of repeated restartpoints. > > Archive recovery should be as fast as poss

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-12 Thread Andres Freund
On 2014-02-12 21:23:54 +0900, MauMau wrote: > Maybe we could consider in that direction, but there is a problem. Archive > recovery slows down compared to 9.1, because of repeated restartpoints. > Archive recovery should be as fast as possible, because it typically applies > dozens or hundreds of

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-12 Thread MauMau
From: "Andres Freund" On 2014-02-02 23:50:40 +0900, Fujii Masao wrote: Right. If standby_mode is enabled, checkpoint_segment can trigger the restartpoint. But the problem is that the timing of restartpoint depends on not only the checkpoint parameters (i.e., checkpoint_timeout and checkpoint_se

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-02 Thread Andres Freund
On 2014-02-02 23:50:40 +0900, Fujii Masao wrote: > On Sun, Feb 2, 2014 at 5:49 AM, Andres Freund wrote: > > On 2014-01-24 22:31:17 +0900, MauMau wrote: > >> I haven't tried reducing checkpoint_timeout. > > > > Did you try reducing checkpoint_segments? As I pointed out, at least if > > standby_mode

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-02 Thread Fujii Masao
On Sun, Feb 2, 2014 at 5:49 AM, Andres Freund wrote: > On 2014-01-24 22:31:17 +0900, MauMau wrote: >> From: "Fujii Masao" >> >On Wed, Jan 22, 2014 at 6:37 AM, Heikki Linnakangas >> >>>Thanks! The patch looks good to me. Attached is the updated version of >> >>>the patch. I added the comments. >>

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-01 Thread Andres Freund
On 2014-01-24 22:31:17 +0900, MauMau wrote: > From: "Fujii Masao" > >On Wed, Jan 22, 2014 at 6:37 AM, Heikki Linnakangas > >>>Thanks! The patch looks good to me. Attached is the updated version of > >>>the patch. I added the comments. > > Thank you very much. Your comment looks great. I tested

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-02-01 Thread Andres Freund
On 2014-01-21 23:37:43 +0200, Heikki Linnakangas wrote: > On 01/21/2014 07:31 PM, Fujii Masao wrote: > >On Fri, Dec 20, 2013 at 9:21 PM, MauMau wrote: > >>From: "Fujii Masao" > >> > >>>! if (source == XLOG_FROM_ARCHIVE && StandbyModeRequested) > >>> > >>>Even when standby_mode is not enabled,

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-01-24 Thread MauMau
From: "Fujii Masao" On Wed, Jan 22, 2014 at 6:37 AM, Heikki Linnakangas Thanks! The patch looks good to me. Attached is the updated version of the patch. I added the comments. Thank you very much. Your comment looks great. I tested some recovery situations, and confirmed that WAL segments

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-01-21 Thread Fujii Masao
On Wed, Jan 22, 2014 at 6:37 AM, Heikki Linnakangas wrote: > On 01/21/2014 07:31 PM, Fujii Masao wrote: >> >> On Fri, Dec 20, 2013 at 9:21 PM, MauMau wrote: >>> >>> From: "Fujii Masao" >>> ! if (source == XLOG_FROM_ARCHIVE && StandbyModeRequested) Even when standby_mode is not

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-01-21 Thread Heikki Linnakangas
On 01/21/2014 07:31 PM, Fujii Masao wrote: On Fri, Dec 20, 2013 at 9:21 PM, MauMau wrote: From: "Fujii Masao" ! if (source == XLOG_FROM_ARCHIVE && StandbyModeRequested) Even when standby_mode is not enabled, we can use cascade replication and it needs the accumulated WAL files. So I thi

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2014-01-21 Thread Fujii Masao
On Fri, Dec 20, 2013 at 9:21 PM, MauMau wrote: > From: "Fujii Masao" > >> ! if (source == XLOG_FROM_ARCHIVE && StandbyModeRequested) >> >> Even when standby_mode is not enabled, we can use cascade replication and >> it needs the accumulated WAL files. So I think that >> AllowCascadeReplicatio

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-12-20 Thread MauMau
From: "Fujii Masao" ! if (source == XLOG_FROM_ARCHIVE && StandbyModeRequested) Even when standby_mode is not enabled, we can use cascade replication and it needs the accumulated WAL files. So I think that AllowCascadeReplication() should be added into this condition. ! snprintf(rec

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-12-18 Thread Fujii Masao
On Mon, Dec 16, 2013 at 9:22 PM, MauMau wrote: > Hi, Fujii san, > >> On Wed, Aug 7, 2013 at 7:03 AM, Fujii Masao wrote: >>> >>> On second thought, probably we cannot remove the restored WAL files early >>> because they might be required for fast promotion which is new feature in >>> 9.3. >>> In f

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-12-16 Thread MauMau
Hi, Fujii san, On Wed, Aug 7, 2013 at 7:03 AM, Fujii Masao wrote: On second thought, probably we cannot remove the restored WAL files early because they might be required for fast promotion which is new feature in 9.3. In fast promotion, an end-of-recovery checkpoint is not executed. After t

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-08-07 Thread Jeff Janes
On Wed, Aug 7, 2013 at 7:03 AM, Fujii Masao wrote: > On Fri, Aug 2, 2013 at 12:24 AM, MauMau wrote: >> From: "Fujii Masao" >> However, isn't StandbyRequested true (= standby_mode set to on) to enable warm standby? >>> >>> >>> We can set up warm-standby by using pg_standby even if stand

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-08-07 Thread Fujii Masao
On Fri, Aug 2, 2013 at 12:24 AM, MauMau wrote: > From: "Fujii Masao" > >>> However, isn't StandbyRequested true (= standby_mode set to on) to enable >>> warm standby? >> >> >> We can set up warm-standby by using pg_standby even if standby_mode = off. > > > I see. However, I understand that pg_st

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-08-07 Thread Fujii Masao
On Fri, Aug 2, 2013 at 10:52 AM, Michael Paquier wrote: > > > > On Fri, Aug 2, 2013 at 12:24 AM, MauMau wrote: >> >> From: "Fujii Masao" >> However, isn't StandbyRequested true (= standby_mode set to on) to enable warm standby? >>> >>> >>> We can set up warm-standby by using pg_st

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-08-02 Thread Dimitri Fontaine
Michael Paquier writes: > By reading this thread, -1 for the addition of a new GUC parameter related > to cascading, it looks like an overkill for the possible gain. And +1 for > the removal of WAL file once it is replayed in archive recovery if > cascading replication is not allowed. However, wha

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-08-01 Thread Michael Paquier
On Fri, Aug 2, 2013 at 12:24 AM, MauMau wrote: > From: "Fujii Masao" > > However, isn't StandbyRequested true (= standby_mode set to on) to enable >>> warm standby? >>> >> >> We can set up warm-standby by using pg_standby even if standby_mode = off. >> > > I see. However, I understand that pg_

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-08-01 Thread MauMau
From: "Fujii Masao" However, isn't StandbyRequested true (= standby_mode set to on) to enable warm standby? We can set up warm-standby by using pg_standby even if standby_mode = off. I see. However, I understand that pg_standby is a legacy feature, and the current way to setup a warm stand

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-07-31 Thread Fujii Masao
On Thu, Aug 1, 2013 at 7:37 AM, MauMau wrote: > From: "Fujii Masao" > >> - if (source == XLOG_FROM_ARCHIVE) >> + if (source == XLOG_FROM_ARCHIVE && >> + StandbyModeRequested && AllowCascadeReplication()) >> >> I think that the condition of StandbyModeRequested should be removed >> because someone

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-07-31 Thread MauMau
From: "Fujii Masao" - if (source == XLOG_FROM_ARCHIVE) + if (source == XLOG_FROM_ARCHIVE && + StandbyModeRequested && AllowCascadeReplication()) I think that the condition of StandbyModeRequested should be removed because someone might want to set up the cascade standby from the standby of warm

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-07-31 Thread Fujii Masao
On Wed, Jul 31, 2013 at 10:43 PM, MauMau wrote: > Hello, Fujii san, all, > > From: "Fujii Masao" >> >> On Sun, Jul 28, 2013 at 7:59 AM, MauMau wrote: >> Do you think this should be fixed? >> >> I think so. >> >>> How should it be fixed? >> >> >> What about removing the restored archived file as

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-07-31 Thread MauMau
Hello, Fujii san, all, From: "Fujii Masao" On Sun, Jul 28, 2013 at 7:59 AM, MauMau wrote: Do you think this should be fixed? I think so. How should it be fixed? What about removing the restored archived file as soon as it's replayed if cascading replication is not enabled (i.e., max_wal_s

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-07-30 Thread MauMau
From: "Fujii Masao" So, I think it is a bug that the disk space increases if not using cascading replication. Those who migrated from 9.1 and do not use 9.2 features would be surprised like me. Do you think this should be fixed? I think so. Thanks for your agreement. I'll try to submit

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-07-29 Thread Fujii Masao
On Sun, Jul 28, 2013 at 7:59 AM, MauMau wrote: > Hello, > > I'm sorry I've been touching several things recently before fixing any of > them. > > I've noticed undesirable disk space increase while performing archive > recovery with PostgreSQL 9.3. This happens with 9.2, too. > > I just performed

[HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-07-27 Thread MauMau
Hello, I'm sorry I've been touching several things recently before fixing any of them. I've noticed undesirable disk space increase while performing archive recovery with PostgreSQL 9.3. This happens with 9.2, too. I just performed archived recovery with the following parameters in recove