Fix handling of shutdown requests during an early crash restart A smart or fast shutdown during an early crash restart can wait forever on the checkpointer and I/O workers. FatalError is still set, so PM_STOP_BACKENDS adds them as processes to wait for, but these are relaunched by the restart and ignore the SIGTERM that a shutdown sends. AbortStartTime has been reset at that point, meaning that SIGKILL would also not fire.
SIGQUIT is now sent instead of SIGTERM when FatalError is set, via HandleFatalError(), arming AbortStartTime. Going through that routine rather than signaling the children directly matters for the startup process: TerminateChildren() marks the startup process as signaled, so its exit is not later reported as a failure. Note that after sending the SIGTERM, we check for any running processes, assuming that auxiliary processes have been already SIGQUIT'd for an immediate shutdown or for a FatalError. A test case is added, able to stress the scenario of a shutdown triggered during the early phases of crash recovery with a custom restore command waiting for a shutdown by scanning the server logs. The CI is stable with this test (several runs done); the buildfarm will be the final judge regarding its stability. No backpatch is done for now, to be on the conservative side. This feels also a slightly risky change for v19, which is close to release, and letting this change bake on HEAD a bit would not hurt. This problem has been originally reported by Justin Pryzby back in 2023, without being fixed, and still existed on HEAD. Reported-by: Justin Pryzby <[email protected]> Author: Ayush Tiwari <[email protected]> Co-authored-by: Michael Paquier <[email protected]> Discussion: https://postgr.es/m/CAJTYsWXCLi00cJ6-_OkMJB4UjyV6EZjc86QwmgE9=ih_c0r...@mail.gmail.com Discussion: https://postgr.es/m/ZWlrdQarrZvLsgIk@pryzbyj2023 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/12f866cd5f72d65ed4435b8203540efde45ba01d Modified Files -------------- src/backend/postmaster/postmaster.c | 14 +++++- src/test/recovery/meson.build | 1 + src/test/recovery/t/058_shutdown_crash_restart.pl | 61 +++++++++++++++++++++++ src/test/recovery/t/wait_for_shutdown | 22 ++++++++ 4 files changed, 96 insertions(+), 2 deletions(-)
