Re: tstzrange on large table gives poor estimate of expected rows

2022-01-25 Thread Tom Dearman
Thanks for your help. It is true we could get rid of it but we still want to use the functional index on the date range as we understand it is supposed to be a better look up - we also have other date range look ups on tables that seem to be degrading. I have found a solution to the problem. The

tstzrange on large table gives poor estimate of expected rows

2022-01-24 Thread Tom Dearman
Hi, We have a fairly big table (22 million rows) which has a start and end timestamp with time zone and other columns. Some of the columns plus start timestamp make a primary key. The end timestamp is exactly one day ahead of the start timestamp for each row and there are approximately 1 row

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Tom Dearman
index. In some of our partitions we might have 2000 old rows that do hang around for a long time and another 100 or so ‘real’ partial index entries so 2200 in total but the number of rows would be 300 million so it is a lot less than 1%. > On 16 Jul 2021, at 16:43, Tom Lane wrote: >

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Tom Dearman
We have change autovacuum so that it runs more frequently autovacuum_vacuum_scale_factor=0.035, the reason we have a partial index on the status is that in a table of 300 million entries, only about 100 or so would have status=‘IN_PROGRESS’ so we think this should be a nice small index and many

Re: Frequetly updated partial index leads to bloat on index for Postresql 11

2021-07-16 Thread Tom Dearman
? Thanks, Tom > On 18 Mar 2021, at 16:30, Peter Geoghegan wrote: > > On Thu, Mar 18, 2021 at 6:51 AM Tom Dearman wrote: >> Is this a known issue, are they any ways around it, and if it is an >> issue is there a plan to fix it if a fix is possible? > > It's not

Frequetly updated partial index leads to bloat on index for Postresql 11

2021-03-18 Thread Tom Dearman
Hi, We have a partial index on a column of the form: CREATE TABLE table_p2021q1 ( pk_id BIGINT, col1 BIGINT NOT NULL, status character varying(255) NOT NULL, ...other columns PRIMARY KEY (pk_id) ); CREATE INDEX table_p2021q1_ix04 ON table_p2021q1 (col1) WHERE status = 'IN_PROGRESS'; (t

logical decoder lsn order between transactions

2018-03-27 Thread Tom Dearman
Hi, We have written a decoder plugin to use streaming replication in an aggregation framework and we write the changes in chunks using OutputPluginWriter from pg_decode_change. In order to ensure we don't process a message twice we add the lsn of each message to our aggregated value and check a

Java client fails to connect to database with replication slots registered if targetServerType=master

2018-03-07 Thread Tom Dearman
Hi, We are trying to use logical decoding on our db (9.4.17), I have written a decoder based on the examples one, it runs ok on my local db, but we found that in another environment we we unable to get a db connection. It turns out to be related to having targetServerType=master in our jdbc co