Hi, XLogReaderFree() does not free decoded records that were too large for the decode buffer. XLogReadRecordAlloc() allocates those separately, and only XLogReleasePreviousRecord() and ResetDecoder() free them. The record last returned by XLogReadRecord() stays in the decode queue until the next read, so if it is larger than the decode buffer (64kB by default), it leaks when the reader is freed.
Most callers free that memory soon after anyway. The checkpointer does not. CheckPointTwoPhase() calls XlogReadTwoPhaseData() for each prepared transaction whose PREPARE record is before the redo pointer, and that function creates and frees a reader in checkpointer_context, which is reset only after an error. So each large PREPARE record read there stays allocated until the checkpointer exits. The attached SQL file reproduces it. It prepares five transactions with 20000 subtransactions each (80220-byte PREPARE records), runs CHECKPOINT after each one, and logs the checkpointer's memory contexts. "Grand total ... used" on master 8628df8891a: step master patched 1 331136 248392 2 413880 248392 3 496624 248392 4 579368 248392 5 662112 248392 That is 82744 bytes per transaction without the patch. 15.19 shows the same growth. The patch calls ResetDecoder() in XLogReaderFree() before the decode buffer is freed. ResetDecoder() frees only the separately allocated records, so a decode buffer supplied by the caller is not touched. The leak goes back to 3f1ce97346 in 15, so I think it should be backpatched. With assertions enabled, the regress, recovery, pg_waldump, pg_rewind, pg_walinspect, test_decoding and pg_walsummary suites give the same results with and without the patch (the injection-point tests were skipped). The leak was found and fixed by Perfloop. I am the contact. Feel free to rework the patch however you prefer. Thanks, Tomás
From c35d167d40146b9c10a018ebe36012ba3e27e720 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tom=C3=A1s=20Senart?= <[email protected]> Date: Mon, 2 Nov 2026 23:05:54 +0000 Subject: [PATCH v1] Free oversized decoded WAL records in XLogReaderFree() A decoded record that does not fit in the reader's decode buffer is allocated separately (XLogReadRecordAlloc() with allow_oversized). Such records are freed only by XLogReleasePreviousRecord() or ResetDecoder(). XLogReaderFree() called neither, so any oversized record still in the decode queue was leaked when the reader was destroyed. The last record returned by XLogReadRecord() stays in the queue until the next read, so this happens whenever the last record read before XLogReaderFree() is larger than the decode buffer. The leak lasts for the life of the process in the checkpointer. CheckPointTwoPhase() calls XlogReadTwoPhaseData() for each prepared transaction whose PREPARE record is before the redo pointer, and that function allocates and frees a reader in checkpointer_context, which is reset only after an error. With five prepared transactions, each with an 80220-byte PREPARE record and followed by a CHECKPOINT, the checkpointer's used memory grew by 82744 bytes per transaction. With this fix it stays flat. Fix by calling ResetDecoder() in XLogReaderFree() before the decode buffer is freed. ResetDecoder() frees only records that were allocated separately, so a caller-supplied decode buffer is not affected. This problem exists since 3f1ce97346, in PostgreSQL 15. Backpatch-through: 15 --- src/backend/access/transam/xlogreader.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index e05a831b6e..f5051ba217 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -178,6 +178,9 @@ XLogReaderFree(XLogReaderState *state) if (state->seg.ws_file >= 0) state->routine.segment_close(state); + /* Free any queued decoded records. */ + ResetDecoder(state); + if (state->decode_buffer && state->free_decode_buffer) pfree(state->decode_buffer); @@ -1728,8 +1731,7 @@ WALRead(XLogReaderState *state, */ /* - * Private function to reset the state, forgetting all decoded records, if we - * are asked to move to a new read position. + * Forget all decoded records, freeing any oversized ones. */ static void ResetDecoder(XLogReaderState *state)
xlogreader-leak-repro.sql
Description: application/sql
