Hi,
I put many queries into one transaction and made ReorderBuffer spill
data to disk, and sent SIGKILL to postgres before the end of the
transaction.
After starting up postgres again, I observed the files spilled to
data wasn't deleted.
I think these files should be deleted because its transaction was no
more valid, so no one can use these files.
Below is a reproduction instructions.
------------------------------------------------
1. Create table and publication at publiser.
@pub =# CREATE TABLE t1(
id INT PRIMARY KEY,
name TEXT);
@pub =# CREATE PUBLICATION pub FOR TABLE t1;
2. Create table and subscription at subscriber.
@sub =# CREATE TABLE t1(
id INT PRIMARY KEY,
name TEXT
);
@sub =# CREATE SUBSCRIPTION sub
CONNECTION 'host=[hostname] port=[port] dbname=[dbname]'
PUBLICATION pub;
3. Put many queries into one transaction.
@pub =# BEGIN;
INSERT INTO t1
SELECT
i,
'aaaaaaaaaa'
FROM
generate_series(1, 1000000) as i;
4. Then we can see spilled files.
@pub $ ls -1 ${PGDATA}/pg_replslot/sub/
state
xid-561-lsn-0-1000000.snap
xid-561-lsn-0-2000000.snap
xid-561-lsn-0-3000000.snap
xid-561-lsn-0-4000000.snap
xid-561-lsn-0-5000000.snap
xid-561-lsn-0-6000000.snap
xid-561-lsn-0-7000000.snap
xid-561-lsn-0-8000000.snap
xid-561-lsn-0-9000000.snap
5. Kill publisher's postgres process before COMMIT.
@pub $ kill -s SIGKILL [pid of postgres]
6. Start publisher's postgres process.
@pub $ pg_ctl start -D ${PGDATA}
7. After a while, we can see the files remaining.
(Immediately after starting publiser, we can not see these files.)
@pub $ pg_ctl start -D ${PGDATA}
When I configured with '--enable-cassert', below assertion error
was appeared.
TRAP: FailedAssertion("!(txn->final_lsn != 0)", File:
"reorderbuffer.c", Line: 2576)
------------------------------------------------
Attached patch sets final_lsn to the last ReorderBufferChange if
final_lsn == 0.
--
Atsushi Torikoshi
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
diff --git a/src/backend/replication/logical/reorderbuffer.c
b/src/backend/replication/logical/reorderbuffer.c
index 0f607ba..51d5b71 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -1093,6 +1093,20 @@ ReorderBufferCleanupTXN(ReorderBuffer *rb,
ReorderBufferTXN *txn)
ReorderBufferCleanupTXN(rb, subtxn);
}
+ /*
+ * If this transaction encountered crash during transaction,
+ * txn->final_lsn remains initial value.
+ * To properly remove entries which were spilled to disk, we need valid
+ * final_lsn.
+ * So we set final_lsn to the lsn of the last ReorderBufferChange.
+ */
+ if (txn->final_lsn == 0)
+ {
+ ReorderBufferChange *last_change =
+ dlist_tail_element(ReorderBufferChange, node, &txn->changes);
+ txn->final_lsn = last_change->lsn;
+ }
+
/* cleanup changes in the toplevel txn */
dlist_foreach_modify(iter, &txn->changes)
{
