Re: max_wal_size
David, I think part of the issue is your operating with a mental model of this contrary to the one the documentation was written for. I made this assumption(checkpoints triggered by size, checkpoint_completion_target=1) only for simplifying example. (Although there is a practical task: calculate the possible approximate size of all WAL files. This is especially important for versions prior to 11, where WAL files were stored for the previous checkpoint.) In any case "grow to" means that it doesn't matter whether at the end of the previous checkpoint 1gb or 9gb were in use - once the usage goes "up to" 10gp the automatic checkpoint will trigger. If the word "to" wasn't there your interpretation would be correct. Its a fair argument to say such an important distinction shouldn't be placed on the word "to" but it isn't wrong. Thank you for an explanation. But give me another chance to explain. Now we have the glossary. According to the glossary, a checkpoint has two meanings. The first meaning is that this is a point: "A point in the WAL sequence at which it is guaranteed that the heap and index data files have been updated with all information from shared memory modified before that checkpoint; a checkpoint record is written and flushed to WAL to mark that point." Simple illustration: A B C WAL |---|---|---|---> Points A,B,C are records in the WAL. The second meaning is that this is an action (performed by checkpointer process or CP): "A checkpoint is also the act of carrying out all the actions that are necessary to reach a checkpoint as defined above." Add a process to the illustration. Checkpoint_completion_target here is the default value of 0.5: A B C WAL |---|-+-|-+-|-+-> a b c CP +-+ +-+ +-+ b e b e b e Checkpointer process for point A starts at a(b) and ends at a(e). At the end of the work, CP makes a WAL record(plus symbol on WAL line). Back to the max_wal_size description: "Maximum size to let the WAL grow to between automatic WAL checkpoints." As I understand now, this is true when we talk about the checkpoint as a process. "Maximum size to let the WAL grow to between a(b) and b(e), between b(b) and c(e), etc". But this not true when we talk about the checkpoint as a point. "Maximum size to let the WAL grow to between A and B". My concerns that someone reading the description of the parameter may confuse the two meanings of checkpoint: as a point and as a process. After all, this happened to me. - Pavel Luzanov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Re: max_wal_size
On Thu, May 28, 2020 at 2:21 AM wrote: > > Now we have the glossary. According to the glossary, a checkpoint has two > meanings. > > The first meaning is that this is a point: > "A point in the WAL sequence at which it is guaranteed that the heap and > index data files have been updated with all information from shared memory > modified before that checkpoint; a checkpoint record is written and flushed > to WAL to mark that point." > > The second meaning is that this is an action (performed by checkpointer > process or CP): > "A checkpoint is also the act of carrying out all the actions that are > necessary to reach a checkpoint as defined above." > > As I understand now, this is true when we talk about the checkpoint as a > process. > "Maximum size to let the WAL grow to between a(b) and b(e), between b(b) > and c(e), etc". > > But this not true when we talk about the checkpoint as a point. > "Maximum size to let the WAL grow to between A and B". > > My concerns that someone reading the description of the parameter may > confuse the two meanings of checkpoint: as a point and as a process. After > all, this happened to me. > Illuminating perspective. The question for the documentation then becomes do we introduce the concept of a "checkpoint point/record" here to try and clear up a possible (but I'd argue rare) misunderstanding at the risk of complicating things for the reader who only has the process meaning in mind. This is a tuning knob for the system and, as with the other two that reference "time", it almost by necessity deals with a process/verb interpretation of checkpoint, not the point/noun one. While I'm quite familiar with the presence of the noun meaning it didn't even occur to me to frame the discussion in those terms. As above, I'm not immediately convinced that that is a bad thing. I don't know whether a change along those lines to the configuration reference page is useful or not - I wouldn't make one without a comprehensive re-read of the actual background/learning section of the documentation. My gut instinct is that I just don't think the documentation can do this situation justice and that, as illustrated by this thread, there are better resources available for these less common situations. That said adding something akin to your diagram and showing how the different parts relate to each other is definitely worth considering - though probably in the main section, not the reference section. David J.
Re: wal_init_zero and wal_recycle
On 2020/05/28 8:43, Thomas Munro wrote: On Wed, May 27, 2020 at 7:09 PM Simon Riggs wrote: On Wed, 27 May 2020 at 04:27, Fujii Masao wrote: Hi, The group of wal_init_zero and wal_recycle is WAL_SETTINGS in guc.c, but their descriptions are located in "19.6. Replication"/"19.6.1. Sending Servers" section. This seems a documentation bug. They should be located in "19.5. Write Ahead Log"/"19.5.1. Settings". Thought? +1 Thanks! Patch attached. Since they are located just before wal_buffers in postgresql.conf.sample, I moved the descriptions of them also just before that of wal_buffers in "Write Ahead Log"/"Settings" section. BTW, while reading guc.c, I found the GUC context of wal_recycle is PGC_SUSET. But isn't it better to set it to PGC_SIGHUP because wal_recycle doesn't affect any backends? Basically checkpointer and startup process use wal_recycle. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index a2694e548a..862baae0c8 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -2820,6 +2820,41 @@ include_dir 'conf.d' + + wal_init_zero (boolean) + + wal_init_zero configuration parameter + + + + +If set to on (the default), this option causes new +WAL files to be filled with zeroes. On some file systems, this ensures +that space is allocated before we need to write WAL records. However, +Copy-On-Write (COW) file systems may not benefit +from this technique, so the option is given to skip the unnecessary +work. If set to off, only the final byte is written +when the file is created so that it has the expected size. + + + + + + wal_recycle (boolean) + + wal_recycle configuration parameter + + + + +If set to on (the default), this option causes WAL +files to be recycled by renaming them, avoiding the need to create new +ones. On COW file systems, it may be faster to create new ones, so the +option is given to disable this behavior. + + + + wal_buffers (integer) @@ -3752,41 +3787,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows - - wal_init_zero (boolean) - - wal_init_zero configuration parameter - - - - -If set to on (the default), this option causes new -WAL files to be filled with zeroes. On some file systems, this ensures -that space is allocated before we need to write WAL records. However, -Copy-On-Write (COW) file systems may not benefit -from this technique, so the option is given to skip the unnecessary -work. If set to off, only the final byte is written -when the file is created so that it has the expected size. - - - - - - wal_recycle (boolean) - - wal_recycle configuration parameter - - - - -If set to on (the default), this option causes WAL -files to be recycled by renaming them, avoiding the need to create new -ones. On COW file systems, it may be faster to create new ones, so the -option is given to disable this behavior. - - - - max_slot_wal_keep_size (integer)
