Hi, On 2021-05-21 11:01:03 -0400, Tom Lane wrote: > It was a good thing I went through this code, too, because I noticed > one serious bug (attcompression not checked in equalTupleDescs) and > another thing that looks like a bug:
Grepping for attcompression while trying to understand the issue Tom reported I found a substantial, but transient, memory leak: During VACUUM FULL reform_and_rewrite_tuple() detoasts the old value if it was compressed with a different method, while in TopTransactionContext. There's nothing freeing that until TopTransactionContext ends - obviously not great for a large relation being VACUUM FULLed. SET default_toast_compression = 'lz4'; DROP TABLE IF EXISTS wide CASCADE; CREATE TABLE wide(data text not null); INSERT INTO wide(data) SELECT random() || (SELECT string_agg(i::text, '') data FROM generate_series(1, 100000) g(i)) FROM generate_series(1, 1000); \c SET client_min_messages = 'log'; SET log_statement_stats = on; VACUUM FULL wide; ... DETAIL: ! system usage stats: ! 0.836638 s user, 0.375344 s system, 1.268705 s elapsed ! [2.502369 s user, 0.961681 s system total] ! 18052 kB max resident size ! 0/1789088 [0/3530048] filesystem blocks in/out ! 0/277 [0/205655] page faults/reclaims, 0 [0] swaps ! 0 [0] signals rcvd, 0/0 [0/0] messages rcvd/sent ! 22/1 [55/6] voluntary/involuntary context switches LOCATION: ShowUsage, postgres.c:4886 VACUUM Time: 1269.029 ms (00:01.269) \c ALTER TABLE wide ALTER COLUMN data SET COMPRESSION pglz; SET client_min_messages = 'log'; SET log_statement_stats = on; VACUUM FULL wide; ... DETAIL: ! system usage stats: ! 19.816867 s user, 0.493233 s system, 20.320711 s elapsed ! [19.835995 s user, 0.493233 s system total] ! 491588 kB max resident size ! 0/656032 [0/656048] filesystem blocks in/out ! 0/287363 [0/287953] page faults/reclaims, 0 [0] swaps ! 0 [0] signals rcvd, 0/0 [0/0] messages rcvd/sent ! 1/24 [13/26] voluntary/involuntary context switches Note the drastically different "max resident size". This is with huge pages (removing s_b from RSS), but it's visible even without. Random fun note: time for VACUUM FULL wide with recompression: pglz->lz4: 3.2s lz4->pglz: 20.3s Greetings, Andres Freund
