On Thu, Feb 08, 2024 at 02:42:00PM +0330, Majid Garoosi wrote: > Thank you very much for your review.
Something to be aware of, but the community lists use bottom-posting for replies because it is easier to follow the logic of a thread this way. See here: https://en.wikipedia.org/wiki/Posting_style#Bottom-posting > I generally agree with your suggestions, so just applied them. > You can find the new patch in the attached file. Thanks for the patch, that looks rather fine. I have spent some time polishing the docs, adding a mention that increasing the value can show benefits depending on what you do. How does the attached look to you? -- Michael
From 2bacd4e4aaff53d26d8017e905e1dd02a67e93ff Mon Sep 17 00:00:00 2001 From: Michael Paquier <mich...@paquier.xyz> Date: Fri, 9 Feb 2024 14:18:31 +0900 Subject: [PATCH v3] Introduce wal_sender_max_send_size as a GUC Blah, blah. Author: Majid Garoosi Discussion: https://postgr.es/m/ --- src/include/replication/walsender.h | 1 + src/include/utils/guc_hooks.h | 1 + src/backend/replication/walsender.c | 26 +++++++------------ src/backend/utils/init/postinit.c | 11 ++++++++ src/backend/utils/misc/guc_tables.c | 11 ++++++++ src/backend/utils/misc/postgresql.conf.sample | 1 + doc/src/sgml/config.sgml | 23 ++++++++++++++++ 7 files changed, 57 insertions(+), 17 deletions(-) diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h index 1b58d50b3b..f4d3c73f4d 100644 --- a/src/include/replication/walsender.h +++ b/src/include/replication/walsender.h @@ -31,6 +31,7 @@ extern PGDLLIMPORT bool wake_wal_senders; /* user-settable parameters */ extern PGDLLIMPORT int max_wal_senders; extern PGDLLIMPORT int wal_sender_timeout; +extern PGDLLIMPORT int wal_sender_max_send_size; extern PGDLLIMPORT bool log_replication_commands; extern void InitWalSender(void); diff --git a/src/include/utils/guc_hooks.h b/src/include/utils/guc_hooks.h index 5300c44f3b..b0ac07171c 100644 --- a/src/include/utils/guc_hooks.h +++ b/src/include/utils/guc_hooks.h @@ -84,6 +84,7 @@ extern bool check_maintenance_io_concurrency(int *newval, void **extra, extern void assign_maintenance_io_concurrency(int newval, void *extra); extern bool check_max_connections(int *newval, void **extra, GucSource source); extern bool check_max_wal_senders(int *newval, void **extra, GucSource source); +extern bool check_wal_sender_max_send_size(int *newval, void **extra, GucSource source); extern bool check_max_slot_wal_keep_size(int *newval, void **extra, GucSource source); extern void assign_max_wal_size(int newval, void *extra); diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 77c8baa32a..a55516f589 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -96,17 +96,6 @@ #include "utils/timeout.h" #include "utils/timestamp.h" -/* - * Maximum data payload in a WAL data message. Must be >= XLOG_BLCKSZ. - * - * We don't have a good idea of what a good value would be; there's some - * overhead per message in both walsender and walreceiver, but on the other - * hand sending large batches makes walsender less responsive to signals - * because signals are checked only between messages. 128kB (with - * default 8k blocks) seems like a reasonable guess for now. - */ -#define MAX_SEND_SIZE (XLOG_BLCKSZ * 16) - /* Array of WalSnds in shared memory */ WalSndCtlData *WalSndCtl = NULL; @@ -124,6 +113,8 @@ int max_wal_senders = 10; /* the maximum number of concurrent * walsenders */ int wal_sender_timeout = 60 * 1000; /* maximum time to send one WAL * data message */ +int wal_sender_max_send_size = XLOG_BLCKSZ * 16; /* Maximum data payload + * in a WAL data message */ bool log_replication_commands = false; /* @@ -2950,8 +2941,8 @@ WalSndSegmentOpen(XLogReaderState *state, XLogSegNo nextSegNo, /* * Send out the WAL in its normal physical/stored form. * - * Read up to MAX_SEND_SIZE bytes of WAL that's been flushed to disk, - * but not yet sent to the client, and buffer it in the libpq output + * Read up to wal_sender_max_send_size bytes of WAL that's been flushed to + * disk, but not yet sent to the client, and buffer it in the libpq output * buffer. * * If there is no unsent WAL remaining, WalSndCaughtUp is set to true, @@ -3132,8 +3123,9 @@ XLogSendPhysical(void) /* * Figure out how much to send in one message. If there's no more than - * MAX_SEND_SIZE bytes to send, send everything. Otherwise send - * MAX_SEND_SIZE bytes, but round back to logfile or page boundary. + * wal_sender_max_send_size bytes to send, send everything. Otherwise send + * wal_sender_max_send_size bytes, but round back to logfile or page + * boundary. * * The rounding is not only for performance reasons. Walreceiver relies on * the fact that we never split a WAL record across two messages. Since a @@ -3143,7 +3135,7 @@ XLogSendPhysical(void) */ startptr = sentPtr; endptr = startptr; - endptr += MAX_SEND_SIZE; + endptr += wal_sender_max_send_size; /* if we went beyond SendRqstPtr, back off */ if (SendRqstPtr <= endptr) @@ -3162,7 +3154,7 @@ XLogSendPhysical(void) } nbytes = endptr - startptr; - Assert(nbytes <= MAX_SEND_SIZE); + Assert(nbytes <= wal_sender_max_send_size); /* * OK to read and send the slice. diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c index 1ad3367159..8d61b006c4 100644 --- a/src/backend/utils/init/postinit.c +++ b/src/backend/utils/init/postinit.c @@ -617,6 +617,17 @@ check_max_wal_senders(int *newval, void **extra, GucSource source) return true; } +/* + * GUC check_hook for wal_sender_max_send_size + */ +bool +check_wal_sender_max_send_size(int *newval, void **extra, GucSource source) +{ + if (*newval > wal_segment_size) + return false; + return true; +} + /* * Early initialization of a backend (either standalone or under postmaster). * This happens even before InitPostgres. diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index 7fe58518d7..eff414fd18 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -2902,6 +2902,17 @@ struct config_int ConfigureNamesInt[] = NULL, NULL, NULL }, + { + {"wal_sender_max_send_size", PGC_SIGHUP, REPLICATION_SENDING, + gettext_noop("Sets size of the maximum data payload in a WAL data message."), + NULL, + GUC_UNIT_BYTE, + }, + &wal_sender_max_send_size, + 16 * XLOG_BLCKSZ, XLOG_BLCKSZ, INT_MAX, + check_wal_sender_max_send_size, NULL, NULL + }, + { {"commit_delay", PGC_SUSET, WAL_SETTINGS, gettext_noop("Sets the delay in microseconds between transaction commit and " diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index da10b43dac..cb430d6e48 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -323,6 +323,7 @@ #wal_keep_size = 0 # in megabytes; 0 disables #max_slot_wal_keep_size = -1 # in megabytes; -1 disables #wal_sender_timeout = 60s # in milliseconds; 0 disables +#wal_sender_max_send_size = 128kB # min 8kB #track_commit_timestamp = off # collect timestamp of transaction commit # (change requires restart) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 61038472c5..4f4285aed8 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -4405,6 +4405,29 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows </listitem> </varlistentry> + <varlistentry id="guc-wal-sender-max-send-size" xreflabel="wal_sender_max_send_size"> + <term><varname>wal_sender_max_send_size</varname> (<type>integer</type>) + <indexterm> + <primary><varname>wal_sender_max_send_size</varname> configuration parameter</primary> + </indexterm> + </term> + <listitem> + <para> + Set the maximum size of a message sent by a WAL sender process during + physical replication. The default value is 16 times the WAL block size, + i.e. <literal>128kB</literal>. This parameter can be changed without + restarting the server. + </para> + <para> + Increasing this value can show performance benefits in some + environments. A larger value reduces the responsiveness of WAL sender + processes to signals as these are checked only between messages. + Setting smaller values increases the overhead of the messages both + in WAL senders and WAL receivers. + </para> + </listitem> + </varlistentry> + <varlistentry id="guc-track-commit-timestamp" xreflabel="track_commit_timestamp"> <term><varname>track_commit_timestamp</varname> (<type>boolean</type>) <indexterm> -- 2.43.0
signature.asc
Description: PGP signature
