Re: Choosing an index on partitioned tables.

2021-09-06 Thread Tim Uckun
Hi Brent. I looked at timescaledb. It does make partitioning on date ranges automatic which is awesome and as you said it does add a couple of extra features on top of postgres but their cloud offering are much more expensive than buying a generic postgres instance from AWS. A generic t3.medium o

Choosing an index on partitioned tables.

2021-09-06 Thread Tim Uckun
I have a series of tables which are going to be queries mostly on two columns. A timestamp table and a metric type column. My plan is to partition by date ranges which means the primary key has to include the timestamp column and the id column As far as I know there is no way to specify an index

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Peter Geoghegan
On Mon, Sep 6, 2021 at 7:52 AM Daniel Westermann (DWE) wrote: > >Try running vacuum with index cleanup = on. > > Thank you, Peter Thanks for testing! -- Peter Geoghegan

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Peter Geoghegan
On Mon, Sep 6, 2021 at 8:59 AM Tom Lane wrote: > Assuming that that choice was made appropriately, I think the advice you > propose here will just cause people to waste lots of cycles on VACUUM > runs that have only marginal effects. Right. The advice that they should receive (if any) is to tune

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Peter Geoghegan
On Mon, Sep 6, 2021 at 9:21 AM Laurenz Albe wrote: > #define BYPASS_THRESHOLD_PAGES 0.02/* i.e. 2% of rel_pages */ > > So up to an additional 2% of all pages can have the all-visible bit > unset with "index_cleanup = auto". > > That is probably not worth worrying, right? I don't think it's w

Re: Query takes around 15 to 20 min over 20Lakh rows

2021-09-06 Thread Matthias Apitz
El día lunes, septiembre 06, 2021 a las 11:45:34p. m. +0530, Shubham Mittal escribió: > 20 Lakh is the current no of rows in the task table.. on which the query is > executed.. Ahh, I never came accross this (Indian) unit 'lakh' and now understand that we're are talking about https://en.wikipedi

Re: Query takes around 15 to 20 min over 20Lakh rows

2021-09-06 Thread Josef Šimánek
po 6. 9. 2021 v 20:14 odesílatel Matthias Apitz napsal: > > > What does the term 'over 20Lakh rows' mean? Thanks AFAIK in India (and surrounding areas) 20 Lakh = 20 * 100 000 = 2 000 000 > matthias > -- > Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 > Publ

Re: Query takes around 15 to 20 min over 20Lakh rows

2021-09-06 Thread Shubham Mittal
20 Lakh is the current no of rows in the task table.. on which the query is executed.. On Mon, Sep 6, 2021, 11:44 PM Matthias Apitz wrote: > > What does the term 'over 20Lakh rows' mean? Thanks > > matthias > -- > Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ > +49-176-3890

Re: Query takes around 15 to 20 min over 20Lakh rows

2021-09-06 Thread Matthias Apitz
What does the term 'over 20Lakh rows' mean? Thanks matthias -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub August 13, 1961: Better a wall than a war. And, while the GDR was still existing, no German troup

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Laurenz Albe
On Mon, 2021-09-06 at 11:59 -0400, Tom Lane wrote: > Laurenz Albe writes: > > It is not an incompatibility that warrants a mention in the release notes, > > but perhaps somthing in > > https://www.postgresql.org/docs/14/indexes-index-only-scans.html > > and/or > > https://www.postgresql.org/docs/1

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Tom Lane
Laurenz Albe writes: > It is not an incompatibility that warrants a mention in the release notes, > but perhaps somthing in > https://www.postgresql.org/docs/14/indexes-index-only-scans.html > and/or > https://www.postgresql.org/docs/14/routine-vacuuming.html#VACUUM-FOR-VISIBILITY-MAP > could be a

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Laurenz Albe
On Mon, 2021-09-06 at 07:46 -0700, Peter Geoghegan wrote: > It's a non-hot update, and so there is a single dead index tuple. You're > seeing > the new optimization that makes vacuum skip indexes in marginal cases.  > > Try running vacuum with index cleanup = on.  It occurs to me that this new d

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Daniel Westermann (DWE)
>It's a non-hot update, and so there is a single dead index tuple. You're >seeing the new optimization that makes vacuum skip indexes in >marginal cases. >Try running vacuum with index cleanup = on. Thank you, Peter

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Peter Geoghegan
It's a non-hot update, and so there is a single dead index tuple. You're seeing the new optimization that makes vacuum skip indexes in marginal cases. Try running vacuum with index cleanup = on. Peter Geoghegan (Sent from my phone)

Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-06 Thread Daniel Westermann (DWE)
Hi, while playing with PostgreSQL 14 Beta 3 I noticed a change when it comes to the visibility map and vacuum. Test case: gbench=# create table t1 ( a int, b text ) with ( fillfactor = 100 ); CREATE TABLE pgbench=# insert into t1 select a, a::text from generate_series(1,100) a; INSERT 0 100

Re: update non-indexed value is slow if some non-related index/fk are enabled

2021-09-06 Thread David G. Johnston
On Monday, September 6, 2021, Philippe Doussot wrote: > I whas hopping the same optimisation as you: Write in place. > > > How exactly would you expect “update-in-place” to work given the nature of MVCC? David J.

Re: pg_upgrade - fe_sendauth: no password supplied

2021-09-06 Thread Tom Lane
"Nick Renders" writes: > [ pg_upgrade fails with ] > connection to database failed: fe_sendauth: no password supplied > could not connect to source postmaster started with the command: > "/Library/PostgreSQL/11/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D > "/Volumes/Postgres_Da

PostgreSQL : column value filtering in Logical Replication

2021-09-06 Thread PALAYRET Jacques
Hello, Would it be possible to have a " Column value filtering in Logical Replication ", on the publication side ? Il would not be a " Column Filtering " neither a " row filtering ". It would be the possibility to send NULL (instead of the column value, for a publication table), when a where

Re: update non-indexed value is slow if some non-related index/fk are enabled

2021-09-06 Thread Philippe Doussot
On 05/09/2021 17:21, Tom Lane wrote: "David G. Johnston" writes: On Friday, September 3, 2021, Philippe Doussot wrote: I don't understand why disabling all index from the table speed up the update because the boolean column is not indexed Index entries point to physical records. You just del

Re: update non-indexed value is slow if some non-related index/fk are enabled

2021-09-06 Thread Philippe Doussot
On 06/09/2021 10:21, rob stone wrote: On Sun, 2021-09-05 at 11:21 -0400, Tom Lane wrote: "David G. Johnston" writes: On Friday, September 3, 2021, Philippe Doussot < philippe.dous...@up.coop> wrote: I don't understand why disabling all index from the table speed up the update because the boo

Re: update non-indexed value is slow if some non-related index/fk are enabled

2021-09-06 Thread rob stone
On Sun, 2021-09-05 at 11:21 -0400, Tom Lane wrote: > "David G. Johnston" writes: > > On Friday, September 3, 2021, Philippe Doussot < > > philippe.dous...@up.coop> > > wrote: > > > I don't understand why disabling all index from the table speed > > > up the > > > update because the boolean colu

pg_upgrade - fe_sendauth: no password supplied

2021-09-06 Thread Nick Renders
Hello, I have been trying to use the pg_upgrade command to update a PostgreSQL 11 environment to 13 on macOS 11. I have followed the steps in the documentation, but the command always fails when trying to connect to the original database. This is the command that is sent: /Library/Postgre

Re: How to observe plan_cache_mode transition from custom to generic plan?

2021-09-06 Thread Laurenz Albe
On Mon, 2021-09-06 at 03:07 -0400, Mladen Gogala wrote: > > On 9/6/21 2:26 AM, Laurenz Albe wrote: > > "Bind variables" just being an Oraclism for parameters, it is*not*  a > > mistake to use them in PostgreSQL. > > Actually, it is a mistake because they don't give you any performance > benefit

Re: How to observe plan_cache_mode transition from custom to generic plan?

2021-09-06 Thread Mladen Gogala
On 9/6/21 2:26 AM, Laurenz Albe wrote: "Bind variables" just being an Oraclism for parameters, it is*not* a mistake to use them in PostgreSQL. Actually, it is a mistake because they don't give you any performance benefit and can potentially worsen the performance. There is no cursor sharin