Hi folks
The attached patch expands the xfunc docs and bgworker docs a little,
providing a starting point for developers to learn how to do some common
tasks the Postgres Way.
It mentions in brief these topics:
* longjmp() based exception handling with elog(ERROR), PG_CATCH() and
PG_RE_THROW() etc
* Latches, spinlocks, LWLocks, heavyweight locks, condition variables
* shm, DSM, DSA, shm_mq
* syscache, relcache, relation_open(), invalidations
* deferred signal handling, CHECK_FOR_INTERRUPTS()
* Resource cleanup hooks and callbacks like on_exit, before_shmem_exit, the
resowner callbacks, etc
* signal handling in bgworkers
All very superficial, but all things I really wish I'd known a little
about, or even that I needed to learn about, when I started working on
postgres.
I'm not sure it's in quite the right place. I wonder if there should be a
separate part of xfunc.sgml that covers the slightly more advanced bits of
postgres backend and function coding like this, lists relevant README files
in the source tree, etc.
I avoided going into details like how resource owners work. I don't want
the docs to have to cover all that in detail; what I hope to do is start
providing people with clear references to the right place in the code,
READMEs, etc to look when they need to understand specific topics.
From 96ce89cb7e1a97d9d247fbacba73ade85a01ea38 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig.rin...@2ndquadrant.com>
Date: Mon, 18 Jan 2021 14:05:15 +0800
Subject: [PATCH v1 2/2] Expand docs on PostgreSQL extension coding
Expand the docs on PostgreSQL extension coding and background worker
development a little so that key topics like allocation, interrupt
handling, exit callbacks, transaction callbacks, PG_TRY()/PG_CATCH(),
resource owners, transaction and snapshot state, etc are at least
briefly mentioned with a few pointers to where to learn more.
Add some detail on background worker signal handling.
---
doc/src/sgml/bgworker.sgml | 86 ++++++++++++++++++--
doc/src/sgml/xfunc.sgml | 162 ++++++++++++++++++++++++++++++++++++-
2 files changed, 239 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/bgworker.sgml b/doc/src/sgml/bgworker.sgml
index 7fd673ab54..9216b8e0ea 100644
--- a/doc/src/sgml/bgworker.sgml
+++ b/doc/src/sgml/bgworker.sgml
@@ -1,6 +1,6 @@
<!-- doc/src/sgml/bgworker.sgml -->
-<chapter id="bgworker">
+<chapter id="bgworker" xreflabel="Background Worker Processes">
<title>Background Worker Processes</title>
<indexterm zone="bgworker">
@@ -29,6 +29,12 @@
</para>
</warning>
+ <para>
+ All code that will be executed in PostgreSQL background workers is expected
+ to follow the basic rules set out for all PostgreSQL extension code in <xref
+ linkend="xfunc-writing-code"/>.
+ </para>
+
<para>
Background workers can be initialized at the time that
<productname>PostgreSQL</productname> is started by including the module name in
@@ -95,6 +101,11 @@ typedef struct BackgroundWorker
buffers, or any custom data structures which the worker itself may
wish to create and use.
</para>
+ <para>
+ See <xref linkend="xfunc-shared-addin"/> for information on how to
+ request extension shared memory allocations, <literal>LWLock</literal>s,
+ etc.
+ </para>
</listitem>
</varlistentry>
@@ -212,9 +223,9 @@ typedef struct BackgroundWorker
Signals are initially blocked when control reaches the
background worker's main function, and must be unblocked by it; this is to
allow the process to customize its signal handlers, if necessary.
- Signals can be unblocked in the new process by calling
- <function>BackgroundWorkerUnblockSignals</function> and blocked by calling
- <function>BackgroundWorkerBlockSignals</function>.
+ It is important that all background workers set up and unblock signal
+ handling before they enter their main loops. Signal handling in background
+ workers is discussed separately in <xref linkend="bgworker-signals"/>.
</para>
<para>
@@ -296,13 +307,74 @@ typedef struct BackgroundWorker
</para>
<para>
- The <filename>src/test/modules/worker_spi</filename> module
- contains a working example,
- which demonstrates some useful techniques.
+ Background workers that require inter-process communication and/or
+ synchronisation should use PostgreSQL's built-in IPC and concurrency
+ features as discussed in <xref linkend="xfunc-concurrency-synchronization"/>
+ and <xref linkend="xfunc-ipc"/>.
+ </para>
+
+ <para>
+ If a background worker needs to sleep or wait for activity it should
+ always use <link linkend="xfunc-sleeping-interrupts-blocking">PostgreSQL's
+ interupt-aware APIs</link> for the purpose. Do not <function>usleep()</function>,
+ <function>system()</function>, make blocking system calls, etc.
+ </para>
+
+ <para>
+ The <filename>src/test/modules/worker_spi</filename> and
+ <filename>src/test/modules/test_shm_mq</filename> contain working examples
+ that demonstrates some useful techniques.
</para>
<para>
The maximum number of registered background workers is limited by
<xref linkend="guc-max-worker-processes"/>.
</para>
+
+ <sect1 id="bgworker-signals" xreflabel="Signal Handling in Background Workers">
+ <title>Signal Handling in Background Workers</title>
+
+ <para>
+ Signals can be unblocked in the new process by calling
+ <function>BackgroundWorkerUnblockSignals</function> and blocked by calling
+ <function>BackgroundWorkerBlockSignals</function>.
+ The default signal handlers installed for background workers <emphasis>do
+ not interrupt queries or blocking calls into other postgres code</emphasis>
+ so they are only suitable for simple background workers that frequently and
+ predictably return control to their main loop. If your worker uses the
+ default background worker signal handling it should call
+ <function>HandleMainLoopInterrupts()</function> on each pass through its
+ main loop.
+ </para>
+
+ <para>
+ Background workers that may make blocking calls into core PostgreSQL code
+ and/or run user-supplied queries should generally replace the default bgworker
+ signal handlers with the handlers used for normal user backends. This will
+ ensure that the worker will respond in a timely manner to a termination
+ request, query cancel request, recovery conflict interrupt, deadlock detector
+ request, etc. To install these handlers, before unblocking interrupts
+ run:
+ <programlisting>
+ pqsignal(SIGTERM, die);
+ pqsignal(SIGUSR1, procsignal_sigusr1_handler);
+ pqsignal(SIGHUP, SignalHandlerForConfigReload);
+ </programlisting>
+ Then ensure that your main loop and any other code that could run for some
+ time contains <function>CHECK_FOR_INTERRUPTS();</function> calls. A
+ background worker using these signal handlers must use <link
+ linkend="xfunc-resource-management">PostgreSQL's resource management APIs
+ and callbacks</link> to handle cleanup rather than relying on control
+ returning to the main loop because the signal handlers may call
+ <function>proc_exit()</function> directly. This is recommended practice
+ for all types of extension code anyway.
+ </para>
+
+ <para>
+ See the comments in <filename>src/include/miscadmin.h</filename> in the
+ postgres headers for more details on signal handling.
+ </para>
+
+ </sect1>
+
</chapter>
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index 2863f7c206..9ff1d003a9 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -2558,7 +2558,7 @@ CREATE FUNCTION concat_text(text, text) RETURNS text
</para>
</sect2>
- <sect2>
+ <sect2 id="xfunc-writing-code" xreflabel="Writing PostgreSQL Extension Code">
<title>Writing Code</title>
<para>
@@ -2615,7 +2615,8 @@ CREATE FUNCTION concat_text(text, text) RETURNS text
<function>malloc</function> and <function>free</function>.
The memory allocated by <function>palloc</function> will be
freed automatically at the end of each transaction, preventing
- memory leaks.
+ memory leaks. See <filename>src/backend/utils/mmgr/README</filename>
+ for more details on PostgreSQL's memory management.
</para>
</listitem>
@@ -2659,6 +2660,163 @@ CREATE FUNCTION concat_text(text, text) RETURNS text
error messages to this effect.
</para>
</listitem>
+
+ <listitem>
+ <para>
+ Use <function>elog()</function> and <function>ereport()</function> for
+ logging output and raising errors instead of any direct stdio calls.
+ Do not attempt to use C++ exceptions or Windows Structured Exception
+ Handling, and never call <function>exit()</function> directly.
+ PostgreSQL uses its own <function>longjmp()</function> based
+ exception system for <literal>ERROR</literal>-level <function>elog()</function>
+ and <function>ereport()</function> calls. These can be handled and forwarded with
+ <literal>PG_TRY()</literal> and <literal>PG_RE_THROW()</literal>:
+<programlisting>
+ PG_TRY();
+ {
+ ...
+ }
+ PG_CATCH();
+ {
+ ....
+ /* Do not attempt to swallow the exception, always rethrow */
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+</programlisting>
+ See <filename>src/include/utils/elog.h</filename> and
+ <filename>src/backend/utils/error/elog.c</filename> for details.
+ </para>
+ </listitem>
+
+ <listitem id="xfunc-threading">
+ <para>
+ Individual PostgreSQL backends are single-threaded.
+ Never call any PostgreSQL function or access any PostgreSQL-managed data
+ structure from a thread other than the main
+ thread. If at all possible your extension should not start any threads
+ and should only use the main thread. PostgreSQL generally uses subprocesses
+ that coordinate over shared memory instead of threads - see
+ <xref linkend="bgworker"/>.
+ </para>
+ </listitem>
+
+ <listitem id="xfunc-file-and-pipe-io">
+ <para>
+ To execute subprocesses and open files, use the routines provided by
+ the file descriptor manager like <function>BasicOpenFile</function>
+ and <function>OpenPipeStream</function> instead of a direct
+ <function>open()</function>, <function>fopen()</function>,
+ <function>popen()</function> or <function>system()</function>.
+ See <filename>src/include/storage/fd.h</filename>.
+ </para>
+ </listitem>
+
+ <listitem id="xfunc-sleeping-interrupts-blocking"
+ xreflabel="Sleeping, Blocking and Interrupt Handling in Extensions">
+ <para>
+ Extension code should always be structured as a non-blocking
+ loop over any ongoing external activities. Avoid making blocking calls
+ into 3rd party libraries or uninterruptible
+ system calls. Use PostgreSQL's provided wait
+ primitives like <function>WaitEventSetWait</function> where necessary. Any
+ potentially long-running loop should periodically call <function>
+ CHECK_FOR_INTERRUPTS()</function> to give PostgreSQL a chance to interrupt
+ the function in case of a shutdown request, query cancel, etc. This means
+ you should <function>sleep()</function> or <function>usleep()</function>
+ for any nontrivial amount of time - use <function>WaitLatch()</function>
+ or its variants instead. For details on interrupt handling see
+ comments near the definition of the <literal>CHECK_FOR_INTERRUPTS()</literal>
+ macro in <filename>src/include/util/miscadmin.h</filename>.
+ </para>
+ </listitem>
+
+ <listitem id="xfunc-resource-management"
+ xreflabel="Resource Management in Extensions">
+ <para>
+ Use PostgreSQL's resource tracking and registration mechanisms
+ and the associated callbacks instead of relying on flow control
+ based cleanup. Your extension function could be terminated mid-execution
+ by PostgreSQL if any function that it calls makes a
+ <function>CHECK_FOR_INTERRUPTS()</function> check. It may not
+ get the opportunity to take cleanup actions via a
+ <literal>PG_TRY()</literal> ... <literal>PG_CATCH()</literal>
+ block if this happens. So all cleanup of resources not already
+ managed by the PostgreSQL runtime must be handled using appropriate
+ callbacks provided by PostgreSQL. These include, but are not limited to,
+ the callback registration APIs:
+ <itemizedlist>
+ <listitem><para><function>before_shmem_exit()</function></para></listitem>
+ <listitem><para><function>on_shmem_exit()</function></para></listitem>
+ <listitem><para><function>on_proc_exit()</function></para></listitem>
+ <listitem><para><literal>PG_ENSURE_ERROR_CLEANUP()</literal></para></listitem>
+ <listitem><para><function>RegisterXactCallback()</function></para></listitem>
+ <listitem><para><function>RegisterResourceReleaseCallback()</function></para></listitem>
+ </itemizedlist>
+ More information on resource management, cleanup and lifecycle can be
+ found in the PostgreSQL headers and sources.
+ </para>
+ </listitem>
+
+ <listitem id="xfunc-concurrency-synchronization"
+ xreflabel="Concurrency and Synchronization in Extensions">
+ <para>
+ Use the PostgreSQL runtime's concurrency and synchronisation primitives
+ rather than platform-native primitives when coordinating activity
+ between PostgreSQL processes. These include signals and ProcSignal multiplexed
+ signals, heavyweight locks ("normal" database object locks),
+ <link linkend="xfunc-shared-addin">LWLock</link>s,
+ Spinlocks, latches, condition variables, and more. Details on all of these
+ is far outside the scope of this document, and the best reference is
+ the relevant source code.
+ </para>
+ </listitem>
+
+ <listitem id="xfunc-ipc"
+ xreflabel="Inter-Process Communication in Extensions">
+ <para>
+ Sometimes relation-based state management for extensions is not
+ sufficient to meet their needs. In that case the extension may need to
+ use PostgreSQL's shared-memory based inter-process communication
+ features, and should certainly do so instead of inventing its own or
+ trying to use platform level features. An extension may use
+ <link linkend="xfunc-shared-addin">"raw" shared memory requested from
+ the postmaster at startup</link> or higher level features like dynamic
+ shared memory segments (<acronym>DSM</acronym>),
+ dynamic shared areas (<acronym>DSA</acronym>),
+ or shared memory message queues (<acronym>shm_mq</acronym>). Examples
+ of the use of some these features can be found in the
+ <filename>src/test/modules/test_shm_mq/</filename> example extension. Others
+ can be found in various main PostgreSQL backend code.
+ </para>
+ </listitem>
+
+ <listitem id="xfunc-relcache-syscache">
+ <para>
+ Look up system catalogs and table information using the relcache and syscache
+ APIs (<function>SearchSysCache...</function>,
+ <function>relation_open()</function>, etc) rather than attempting to run
+ SQL queries to fetch the information. Ensure that your function holds
+ any necessary locks on the target objects. Take care not to make any calls
+ that could trigger cache invalidations while still accessing any
+ syscache cache entries, as this can cause subtle bugs. See
+ <filename>src/backend/utils/cache/syscache.c</filename>,
+ <filename>src/backend/utils/cache/relcache.c</filename>,
+ <filename>src/backend/access/common/relation.c</filename> and their
+ headers for details.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ There are many README files in the PostgreSQL source tree that discuss
+ specific development topics in more detail than be covered here. Many
+ subsystems' source files also have detailed comments that explain their
+ correct use. The existing sources are your best reference for more
+ complex tasks.
+ </para>
+ </listitem>
+
</itemizedlist>
</para>
</sect2>
--
2.29.2
