On Fri, May 17, 2024 at 02:21:23PM -0500, Nathan Bossart wrote:
> On Fri, May 17, 2024 at 12:48:37PM -0500, Nathan Bossart wrote:
>> Cool. I'll at least fix the back-branches as-is, but I'll see about
>> revamping this stuff for v18.
>
> Attached is probably the absolute least we should do for the back-branches.
Any concerns with doing something like this [0] for the back-branches? The
constant would be 6 instead of 7 on v14 through v16.
I wrote a quick sketch for what a runtime-computed GUC might look like for
v18. We don't have agreement on this approach, but I figured I'd post
something while we search for a better one.
[0]
https://postgr.es/m/attachment/160360/v1-0001-fix-kernel-resources-docs-on-back-branches.patch
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
>From f5abd8150c317a0eaf4fb2aff79164ce929d2d96 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nat...@postgresql.org>
Date: Tue, 21 May 2024 14:02:22 -0500
Subject: [PATCH v1 1/1] add semaphores_required GUC
---
doc/src/sgml/config.sgml | 14 ++++++++++++++
doc/src/sgml/runtime.sgml | 28 +++++++++++++++++-----------
src/backend/storage/ipc/ipci.c | 6 +++++-
src/backend/utils/misc/guc_tables.c | 12 ++++++++++++
4 files changed, 48 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 698169afdb..f6afc941df 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -11215,6 +11215,20 @@ dynamic_library_path =
'C:\tools\postgresql;H:\my_project\lib;$libdir'
</listitem>
</varlistentry>
+ <varlistentry id="guc-semaphores-required"
xreflabel="semaphores_required">
+ <term><varname>semaphores_required</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>semaphores_required</varname> configuration
parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Reports the number of semaphores that are needed for the server based
+ on the number of allowed connections, worker processes, etc.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-ssl-library" xreflabel="ssl_library">
<term><varname>ssl_library</varname> (<type>string</type>)
<indexterm>
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 6047b8171d..e5ebfb8a8b 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -781,13 +781,13 @@ psql: error: connection to server on socket
"/tmp/.s.PGSQL.5432" failed: No such
<row>
<entry><varname>SEMMNI</varname></entry>
<entry>Maximum number of semaphore identifiers (i.e., sets)</entry>
- <entry>at least <literal>ceil((max_connections +
autovacuum_max_workers + max_wal_senders + max_worker_processes + 5) /
16)</literal> plus room for other applications</entry>
+ <entry>at least <literal>ceil(semaphores_required / 16)</literal> plus
room for other applications</entry>
</row>
<row>
<entry><varname>SEMMNS</varname></entry>
<entry>Maximum number of semaphores system-wide</entry>
- <entry><literal>ceil((max_connections + autovacuum_max_workers +
max_wal_senders + max_worker_processes + 5) / 16) * 17</literal> plus room for
other applications</entry>
+ <entry><literal>ceil(semaphores_required / 16) * 17</literal> plus
room for other applications</entry>
</row>
<row>
@@ -836,27 +836,33 @@ psql: error: connection to server on socket
"/tmp/.s.PGSQL.5432" failed: No such
<para>
When using System V semaphores,
- <productname>PostgreSQL</productname> uses one semaphore per allowed
connection
- (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
- (<xref linkend="guc-autovacuum-max-workers"/>) and allowed background
- process (<xref linkend="guc-max-worker-processes"/>), in sets of 16.
+ <productname>PostgreSQL</productname> uses one semaphore per allowed
connection,
+ worker process, etc., in sets of 16.
Each such set will
also contain a 17th semaphore which contains a <quote>magic
number</quote>, to detect collision with semaphore sets used by
other applications. The maximum number of semaphores in the system
is set by <varname>SEMMNS</varname>, which consequently must be at least
- as high as <varname>max_connections</varname> plus
- <varname>autovacuum_max_workers</varname> plus
<varname>max_wal_senders</varname>,
- plus <varname>max_worker_processes</varname>, plus one extra for each 16
- allowed connections plus workers (see the formula in <xref
+ as high as <xref linkend="guc-semaphores-required"/> plus one extra for
+ each set of 16 required semaphores (see the formula in <xref
linkend="sysvipc-parameters"/>). The parameter <varname>SEMMNI</varname>
determines the limit on the number of semaphore sets that can
exist on the system at one time. Hence this parameter must be at
- least <literal>ceil((max_connections + autovacuum_max_workers +
max_wal_senders + max_worker_processes + 5) / 16)</literal>.
+ least <literal>ceil(semaphores_required / 16)</literal>.
Lowering the number
of allowed connections is a temporary workaround for failures,
which are usually confusingly worded <quote>No space
left on device</quote>, from the function <function>semget</function>.
+ The number of semaphores required by <productname>PostgreSQL</productname>
+ is provided by the runtime-computed parameter
+ <varname>semaphores_required</varname>, which can be determined before
+ starting the server with a <command>postgres</command> command like:
+<programlisting>
+$ <userinput>postgres -D $PGDATA -C semaphores_required</userinput>
+</programlisting>
+ The value of <varname>semaphores_required</varname> should be input into
+ the aforementioned formulas to determine appropriate values for
+ <varname>SEMMNI</varname> and <varname>SEMMNS</varname>.
</para>
<para>
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 521ed5418c..3e030accac 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -372,11 +372,12 @@ InitializeShmemGUCs(void)
Size size_b;
Size size_mb;
Size hp_size;
+ int num_semas;
/*
* Calculate the shared memory size and round up to the nearest
megabyte.
*/
- size_b = CalculateShmemSize(NULL);
+ size_b = CalculateShmemSize(&num_semas);
size_mb = add_size(size_b, (1024 * 1024) - 1) / (1024 * 1024);
sprintf(buf, "%zu", size_mb);
SetConfigOption("shared_memory_size", buf,
@@ -395,4 +396,7 @@ InitializeShmemGUCs(void)
SetConfigOption("shared_memory_size_in_huge_pages", buf,
PGC_INTERNAL,
PGC_S_DYNAMIC_DEFAULT);
}
+
+ sprintf(buf, "%d", num_semas);
+ SetConfigOption("semaphores_required", buf, PGC_INTERNAL,
PGC_S_DYNAMIC_DEFAULT);
}
diff --git a/src/backend/utils/misc/guc_tables.c
b/src/backend/utils/misc/guc_tables.c
index 46c258be28..86157394d8 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -599,6 +599,7 @@ static int segment_size;
static int shared_memory_size_mb;
static int shared_memory_size_in_huge_pages;
static int wal_block_size;
+static int semaphores_required;
static bool data_checksums;
static bool integer_datetimes;
@@ -2291,6 +2292,17 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"semaphores_required", PGC_INTERNAL, PRESET_OPTIONS,
+ gettext_noop("Shows the number of semaphores required
for the server."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE |
GUC_RUNTIME_COMPUTED
+ },
+ &semaphores_required,
+ 0, 0, INT_MAX,
+ NULL, NULL, NULL
+ },
+
{
{"commit_timestamp_buffers", PGC_POSTMASTER, RESOURCES_MEM,
gettext_noop("Sets the size of the dedicated buffer
pool used for the commit timestamp cache."),
--
2.39.3 (Apple Git-146)
