On Wed, Apr 28, 2021 at 5:01 PM Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Wed, Apr 28, 2021 at 4:51 PM Masahiko Sawada <sawada.m...@gmail.com> wrote: > > > > On Wed, Apr 28, 2021 at 6:39 PM Amit Kapila <amit.kapil...@gmail.com> wrote: > > @@ -1369,7 +1369,7 @@ ReorderBufferIterTXNNext(ReorderBuffer *rb, > ReorderBufferIterTXNState *state) > * Update the total bytes processed before releasing the current set > * of changes and restoring the new set of changes. > */ > - rb->totalBytes += rb->size; > + rb->totalBytes += entry->txn->total_size; > if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->file, > &state->entries[off].segno)) > > I have not tested this but won't in the above change you need to check > txn->toptxn for subtxns? >
Now, I am able to reproduce this issue:
Create table t1(c1 int);
select pg_create_logical_replication_slot('s', 'test_decoding');
Begin;
insert into t1 values(1);
savepoint s1;
insert into t1 select generate_series(1, 100000);
commit;
postgres=# select count(*) from pg_logical_slot_peek_changes('s1', NULL, NULL);
count
--------
100005
(1 row)
postgres=# select * from pg_stat_replication_slots;
slot_name | spill_txns | spill_count | spill_bytes | stream_txns |
stream_count | stream_bytes | total_txns | total_bytes |
stats_reset
-----------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+----------------------------------
s1 | 0 | 0 | 0 | 0 |
0 | 0 | 2 | 13200672 | 2021-04-29
14:33:55.156566+05:30
(1 row)
select * from pg_stat_reset_replication_slot('s1');
Now reduce the logical decoding work mem to allow spilling.
postgres=# set logical_decoding_work_mem='64kB';
SET
postgres=# select count(*) from pg_logical_slot_peek_changes('s1', NULL, NULL);
count
--------
100005
(1 row)
postgres=# select * from pg_stat_replication_slots;
slot_name | spill_txns | spill_count | spill_bytes | stream_txns |
stream_count | stream_bytes | total_txns | total_bytes |
stats_reset
-----------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+----------------------------------
s1 | 1 | 202 | 13200000 | 0 |
0 | 0 | 2 | 672 | 2021-04-29
14:35:21.836613+05:30
(1 row)
You can notice that after we have allowed spilling the 'total_bytes'
stats is showing a different value. The attached patch fixes the issue
for me. Let me know what do you think about this?
--
With Regards,
Amit Kapila.
use_total_size_v3.patch
Description: Binary data
