Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-09-24 Thread Peter Geoghegan
> On Thu, Mar 18, 2021 at 6:51 AM Tom Dearman wrote: > Is this a known issue, are they any ways around it, and if it is an > issue is there a plan to fix it if a fix is possible? On second thought I do think that the improvements to 14 will fix this for you. See the test case here: https://www.p

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Peter Geoghegan
On Fri, Jul 16, 2021 at 9:19 AM Tom Dearman wrote: > Other indexes do bloat, but the percentage bloat is a lot less I have to imagine that the remaining problems have a lot to do with the fact that this is a partial index -- the partial index naturally gets vacuumed much less frequently than what

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Tom Dearman
Other indexes do bloat, but the percentage bloat is a lot less, presumably because this is a partial index where the partial column has a high degree of changes ie maybe 100 genuinely ‘live’ rows in a table of 300 million where every row has gone through a state where it would have been in the i

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Francisco Olarte
Tom-Tom: On Fri, Jul 16, 2021 at 5:43 PM Tom Lane wrote: > Tom Dearman writes: > > We have change autovacuum so that it runs more frequently > > autovacuum_vacuum_scale_factor=0.035, the reason we have a partial index on > > the status is that in a table of 300 million entries, only about 100

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Tom Lane
Tom Dearman writes: > We have change autovacuum so that it runs more frequently > autovacuum_vacuum_scale_factor=0.035, the reason we have a partial index on > the status is that in a table of 300 million entries, only about 100 or so > would have status=‘IN_PROGRESS’ so we think this should be

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Michael Lewis
Have you tried setting autovacuum to run quite aggressively, perhaps just on this table? Have you tried an index on the status column only, rather than partial?

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Tom Dearman
We have change autovacuum so that it runs more frequently autovacuum_vacuum_scale_factor=0.035, the reason we have a partial index on the status is that in a table of 300 million entries, only about 100 or so would have status=‘IN_PROGRESS’ so we think this should be a nice small index and many

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Tom Dearman
Hi, We upgraded to 13 a couple of months ago on production but are still having an issue with bloated partial indexes which have an impact on our partial queries especially towards the end of a quarter when our quarterly-partitioned tables are getting big. I have built 14 (on macOS catalina, 1

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-03-18 Thread Peter Geoghegan
On Thu, Mar 18, 2021 at 6:51 AM Tom Dearman wrote: > Is this a known issue, are they any ways around it, and if it is an > issue is there a plan to fix it if a fix is possible? It's not exactly a known issue per se, but I think the problem here is related to the fact that you have lots of duplica

Frequetly updated partial index leads to bloat on index for Postresql 11

2021-03-18 Thread Tom Dearman
Hi, We have a partial index on a column of the form: CREATE TABLE table_p2021q1 ( pk_id BIGINT, col1 BIGINT NOT NULL, status character varying(255) NOT NULL, ...other columns PRIMARY KEY (pk_id) ); CREATE INDEX table_p2021q1_ix04 ON table_p2021q1 (col1) WHERE status = 'IN_PROGRESS'; (t