Re: A few new options for CHECKPOINT

2020-12-16 Thread Bruce Momjian
On Mon, Dec 7, 2020 at 11:22:01AM +0900, Michael Paquier wrote: > On Sun, Dec 06, 2020 at 10:03:08AM -0500, Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > >> You keep making this statement, and I don't necessarily disagree, but if > >> that is the case, please explain

Re: A few new options for CHECKPOINT

2020-12-14 Thread Bossart, Nathan
On 12/5/20, 9:11 AM, "Stephen Frost" wrote: > * Bossart, Nathan (bossa...@amazon.com) wrote: >> On 12/5/20, 6:41 AM, "Stephen Frost" wrote: >> > Assuming we actually want to do this, which I still generally don't >> > agree with since it isn't really clear if it'll actually end up doing >> > some

Re: A few new options for CHECKPOINT

2020-12-06 Thread Michael Paquier
On Sun, Dec 06, 2020 at 10:03:08AM -0500, Stephen Frost wrote: > * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: >> You keep making this statement, and I don't necessarily disagree, but if >> that is the case, please explain why don't we have >> checkpoint_completion_target set to 0.9 by default?

Re: A few new options for CHECKPOINT

2020-12-06 Thread Stephen Frost
Greetings, * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > On 2020-Dec-05, Stephen Frost wrote: > > So- just to be clear, CHECKPOINTs are more-or-less always happening in > > PG, and running this command might do something or might end up doing > > nothing depending on if a checkpoint is alrea

Re: A few new options for CHECKPOINT

2020-12-05 Thread Alvaro Herrera
On 2020-Dec-05, Stephen Frost wrote: > So- just to be clear, CHECKPOINTs are more-or-less always happening in > PG, and running this command might do something or might end up doing > nothing depending on if a checkpoint is already in progress and this > request just gets consolidated into an exis

Re: A few new options for CHECKPOINT

2020-12-05 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 12/5/20, 6:41 AM, "Stephen Frost" wrote: > > Assuming we actually want to do this, which I still generally don't > > agree with since it isn't really clear if it'll actually end up doing > > something, or not, wouldn't it make more se

Re: A few new options for CHECKPOINT

2020-12-05 Thread Bossart, Nathan
On 12/5/20, 6:41 AM, "Stephen Frost" wrote: > Assuming we actually want to do this, which I still generally don't > agree with since it isn't really clear if it'll actually end up doing > something, or not, wouldn't it make more sense to have a command that > just sits and waits for the currently

Re: A few new options for CHECKPOINT

2020-12-05 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > This all seems reasonable to me. I've attached a new version of the > patch. diff --git a/doc/src/sgml/ref/checkpoint.sgml b/doc/src/sgml/ref/checkpoint.sgml index 2afee6d7b5..2b1e56fbd7 100644 --- a/doc/src/sgml/ref/checkpoint.sgml +++

Re: A few new options for CHECKPOINT

2020-12-04 Thread Michael Paquier
On Sat, Dec 05, 2020 at 12:11:13AM +, Bossart, Nathan wrote: > On 12/4/20, 3:33 PM, "Alvaro Herrera" wrote: >> Instead of adding checkpt_option_list, how about utility_option_list? >> It seems intended for reuse. +1. It is intended for reuse. > Ah, good call. That simplifies the grammar ch

Re: A few new options for CHECKPOINT

2020-12-04 Thread Alvaro Herrera
On 2020-Dec-04, Bossart, Nathan wrote: > On 12/4/20, 1:47 PM, "Alvaro Herrera" wrote: > > On the UI of this patch, you're proposing to add the option FAST. I'm > > not a fan of this option name and propose that (if we have it) we use > > the name SPREAD instead (defaults to false). > > > > Now w

Re: A few new options for CHECKPOINT

2020-12-04 Thread Stephen Frost
Greetings, * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > I think starting a spread checkpoint has some usefulness, if your > checkpoint interval is very large but your completion target is not very > close to 1. In that case, you're expressing that you want a checkpoint > to start now and n

Re: A few new options for CHECKPOINT

2020-12-04 Thread Alvaro Herrera
On the UI of this patch, you're proposing to add the option FAST. I'm not a fan of this option name and propose that (if we have it) we use the name SPREAD instead (defaults to false). Now we don't actually explain the term "spread" much in the documentation; we just say "the writes are spread".

Re: A few new options for CHECKPOINT

2020-11-30 Thread Bossart, Nathan
On 11/29/20, 7:21 PM, "Stephen Frost" wrote: > Checkpoints are always happening though, that's kind of my point..? > Sure, you get lucky sometimes that the time you snapshot might have less > outstanding WAL than at some other time, but I'm not convinced that this > patch is really going to give a

Re: A few new options for CHECKPOINT

2020-11-29 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/28/20, 9:50 AM, "Stephen Frost" wrote: > >> Granted, I used a rather aggressive checkpoint_timeout, but I think > >> this demonstrates that waiting for a non-immediate checkpoint to > >> complete can lower the amount of WAL needed

Re: A few new options for CHECKPOINT

2020-11-28 Thread Bossart, Nathan
On 11/28/20, 9:50 AM, "Stephen Frost" wrote: >> Granted, I used a rather aggressive checkpoint_timeout, but I think >> this demonstrates that waiting for a non-immediate checkpoint to >> complete can lower the amount of WAL needed for recovery, even though >> it might not lower it as much as waiti

Re: A few new options for CHECKPOINT

2020-11-28 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/27/20, 10:58 AM, "Stephen Frost" wrote: > > If you'd like to show that I'm wrong, and it's entirely possible that I > > am, then retry the above with actual load on the system, and also > > actually look at how much outstanding WAL

Re: A few new options for CHECKPOINT

2020-11-27 Thread Bossart, Nathan
On 11/27/20, 10:58 AM, "Stephen Frost" wrote: > If you'd like to show that I'm wrong, and it's entirely possible that I > am, then retry the above with actual load on the system, and also > actually look at how much outstanding WAL you end up with given the > different scenarios which has to be re

Re: A few new options for CHECKPOINT

2020-11-27 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/27/20, 8:29 AM, "Stephen Frost" wrote: > > Also note that, in all other cases (that is, when there *is* outstanding > > WAL since the last checkpoint), pg_start_backup actually just waits for > > the existing checkpoint to complete

Re: A few new options for CHECKPOINT

2020-11-27 Thread Bossart, Nathan
On 11/27/20, 8:29 AM, "Stephen Frost" wrote: > Also note that, in all other cases (that is, when there *is* outstanding > WAL since the last checkpoint), pg_start_backup actually just waits for > the existing checkpoint to complete- and while it's waiting for that to > happen, there'll be addition

Re: A few new options for CHECKPOINT

2020-11-27 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > Thanks to all for the feedback. I've attached v2 of the patch. I've > removed the WAIT and FORCE options and renamed IMMEDIATE to FAST. > > On 11/25/20, 7:52 AM, "Stephen Frost" wrote: > > I'm a bit confused by the idea here.. The wh

Re: A few new options for CHECKPOINT

2020-11-25 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 11/24/20, 4:03 PM, "tsunakawa.ta...@fujitsu.com" > wrote: > > From: Bossart, Nathan > >> The main purpose of this patch is to give users more control over their > >> manually > >> requested checkpoints or restartpoints. I suspect

Re: A few new options for CHECKPOINT

2020-11-25 Thread Laurenz Albe
On Wed, 2020-11-25 at 11:41 +0100, Bernd Helmle wrote: > Am Mittwoch, den 25.11.2020, 13:47 +0900 schrieb Michael Paquier: > > > I can see the use case for IMMEDIATE, but I fail to see the use cases > > for WAIT and FORCE. CHECKPOINT_FORCE is internally implied for the > > end-of-recovery and shu

Re: A few new options for CHECKPOINT

2020-11-25 Thread Bernd Helmle
Am Mittwoch, den 25.11.2020, 13:47 +0900 schrieb Michael Paquier: > I can see the use case for IMMEDIATE, but I fail to see the use cases > for WAIT and FORCE.  CHECKPOINT_FORCE is internally implied for the > end-of-recovery and shutdown checkpoints.  WAIT could be a dangerous > thing if disabled,

Re: A few new options for CHECKPOINT

2020-11-24 Thread Fujii Masao
On 2020/11/25 13:47, Michael Paquier wrote: On Wed, Nov 25, 2020 at 01:07:47AM +, tsunakawa.ta...@fujitsu.com wrote: From: Bossart, Nathan It may be useful for backups taken with the "consistent snapshot" approach. As noted in the documentation [0], running CHECKPOINT before taking the

Re: A few new options for CHECKPOINT

2020-11-24 Thread Michael Paquier
On Wed, Nov 25, 2020 at 01:07:47AM +, tsunakawa.ta...@fujitsu.com wrote: > From: Bossart, Nathan >> It may be useful for backups taken with the "consistent snapshot" >> approach. As noted in the documentation [0], running CHECKPOINT >> before taking the snapshot can reduce recovery time. How

RE: A few new options for CHECKPOINT

2020-11-24 Thread tsunakawa.ta...@fujitsu.com
From: Bossart, Nathan > It may be useful for backups taken with the "consistent snapshot" > approach. As noted in the documentation [0], running CHECKPOINT > before taking the snapshot can reduce recovery time. However, users > might wish to avoid the IO spike caused by an immediate checkpoint.

Re: A few new options for CHECKPOINT

2020-11-24 Thread Bossart, Nathan
On 11/24/20, 4:03 PM, "tsunakawa.ta...@fujitsu.com" wrote: > From: Bossart, Nathan >> The main purpose of this patch is to give users more control over their >> manually >> requested checkpoints or restartpoints. I suspect the most useful option is >> IMMEDIATE, which can help avoid checkpoint

RE: A few new options for CHECKPOINT

2020-11-24 Thread tsunakawa.ta...@fujitsu.com
From: Bossart, Nathan > The main purpose of this patch is to give users more control over their > manually > requested checkpoints or restartpoints. I suspect the most useful option is > IMMEDIATE, which can help avoid checkpoint- related IO spikes. However, I > didn't see any strong reason to