Re: prevent WAL replication to fill filesystem

2021-09-07 Thread Jehan-Guillaume de Rorthais
Le 8 septembre 2021 06:07:15 GMT+02:00, Ninad Shah a écrit : >These suggestions are appropriate. However, if you are ready to lose your >replica to keep production running, there are a couple of methods. > >1) Set archive_command to "/bin/true" >2) rename .ready files in archive_status to .don

Re: Is there any technical reason why "alter table .. set not null" can't use index?

2021-09-07 Thread hubert depesz lubaczewski
On Wed, Sep 08, 2021 at 07:09:31AM +0200, Alexander Kukushkin wrote: > Hi, > > On Wed, 8 Sep 2021, 06:59 hubert depesz lubaczewski, > wrote: > > > Hi, > > we needed recently to add not null constraint on some fields, and it > > struck me that it took long. > > Obviously - pg has to check the dat

Re: Is there any technical reason why "alter table .. set not null" can't use index?

2021-09-07 Thread Alexander Kukushkin
Hi, On Wed, 8 Sep 2021, 06:59 hubert depesz lubaczewski, wrote: > Hi, > we needed recently to add not null constraint on some fields, and it > struck me that it took long. > Obviously - pg has to check the data. But it seems that it can't use > index. > It can't use the index, but can use an al

Is there any technical reason why "alter table .. set not null" can't use index?

2021-09-07 Thread hubert depesz lubaczewski
Hi, we needed recently to add not null constraint on some fields, and it struck me that it took long. Obviously - pg has to check the data. But it seems that it can't use index. Made 4 test tables: create table test (a int4, b timestamptz); insert into test (a,b) select i, now() - i * '1 minute'::

Re: prevent WAL replication to fill filesystem

2021-09-07 Thread Ninad Shah
These suggestions are appropriate. However, if you are ready to lose your replica to keep production running, there are a couple of methods. 1) Set archive_command to "/bin/true" 2) rename .ready files in archive_status to .done. Again, I personally do not recommend this, so consider this as a la

Re: update non-indexed value is slow if some non-related index/fk are enabled

2021-09-07 Thread Philippe Doussot
On 06/09/2021 16:13, David G. Johnston wrote: On Monday, September 6, 2021, Philippe Doussot mailto:philippe.dous...@up.coop>> wrote: I whas hopping the same optimisation as you: Write in place. How exactly would you expect “update-in-place” to work given the nature of MVCC? David J.

Re: Query takes around 15 to 20 min over 20Lakh rows

2021-09-07 Thread Michael Lewis
Have you ever used this site to visualize the explain plan and spot bad estimates and slow nodes? https://explain.depesz.com/s/WE1R This stands out to me- *Subquery Scan on npiactionjoin (cost=10,165,289.40..10,167,192.01 rows=293 width=16) (actual time=118,413.432..118,806.684 rows=446,782 loops

Re: How to log bind values for statements that produce errors

2021-09-07 Thread Raul Kaubi
Hmm, actually, this same client executes several queries for that same session. For the first queries, I can clearly see the binds in postgresql.log. Only this third query that is being executed, it produces error and not binds are logged. Also, there may the case, where the third query gets its bi

Re: How to log bind values for statements that produce errors

2021-09-07 Thread Tom Lane
Raul Kaubi writes: > We have a problem with certain select statement, which produces error and we > would like to know the bind value that is given as parameter. > I have tried parameters: > log_min_duration_statement = 0 > log_parameter_max_length_on_error = -1 The other constraint on reporting

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Magnus Hagander
On Tue, Sep 7, 2021 at 12:15 PM Tim Uckun wrote: > > There is an image marked as official: https://hub.docker.com/_/postgres > > I presumed this was maintained by the postgres team. It is official *docker*, just not official *postgresql*. If you click their "maintained by" link you get to https:

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Tim Uckun
There is an image marked as official: https://hub.docker.com/_/postgres I presumed this was maintained by the postgres team. On Tue, Sep 7, 2021 at 9:59 PM Magnus Hagander wrote: > > On Tue, Sep 7, 2021 at 11:52 AM Tim Uckun wrote: > > > > It's weird that it's supported on AWS and many other

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Magnus Hagander
On Tue, Sep 7, 2021 at 11:52 AM Tim Uckun wrote: > > It's weird that it's supported on AWS and many other providers but not > in the official docker images. That'd be something to talk to the docker people about I guess? There are no official docker images published by *postgresql*. (And of cours

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Tim Uckun
It's weird that it's supported on AWS and many other providers but not in the official docker images. On Tue, Sep 7, 2021 at 9:16 PM Magnus Hagander wrote: > > On Tue, Sep 7, 2021 at 10:51 AM Tim Uckun wrote: > > > > To be fair Timescale also adds some other features which might be > > useful.

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Magnus Hagander
On Tue, Sep 7, 2021 at 10:51 AM Tim Uckun wrote: > > To be fair Timescale also adds some other features which might be > useful. For example they add some SQL enhancements like last value > and auto maintaining materialized views and such. The automatic > management of partitions is also pretty b

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Tim Uckun
To be fair Timescale also adds some other features which might be useful. For example they add some SQL enhancements like last value and auto maintaining materialized views and such. The automatic management of partitions is also pretty big plus in my opinion. You can get some of the equivalent f

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Tim Uckun
Thanks! That's great about the Btree deduplication feature in 13. On Tue, Sep 7, 2021 at 7:21 PM Laurenz Albe wrote: > > On Tue, 2021-09-07 at 15:44 +1200, Tim Uckun wrote: > > I have a series of tables which are going to be queries mostly on two > > columns. A timestamp table and a metric type

How to log bind values for statements that produce errors

2021-09-07 Thread Raul Kaubi
H Centos7, postgres 13 We have a problem with certain select statement, which produces error and we would like to know the bind value that is given as parameter. At the moment, the column is bigint type, and probably the bind that goes there is text, in postgresql logfile, the error is followin

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Laurenz Albe
On Tue, 2021-09-07 at 04:06 +, Brent Wood wrote: > From: Tim Uckun > > My plan is to partition by date ranges which means the primary key has > > to include the timestamp column and the id column  As far as I know > > there is no way to specify an index type for those columns. > > > > The metr

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Laurenz Albe
On Tue, 2021-09-07 at 15:44 +1200, Tim Uckun wrote: > I have a series of tables which are going to be queries mostly on two > columns. A timestamp table and a metric type column. > > My plan is to partition by date ranges which means the primary key has > to include the timestamp column and the id

Re: Behavior change in PostgreSQL 14Beta3 or bug?

2021-09-07 Thread Laurenz Albe
On Mon, 2021-09-06 at 12:11 -0700, Peter Geoghegan wrote: > On Mon, Sep 6, 2021 at 9:21 AM Laurenz Albe wrote: > > #define BYPASS_THRESHOLD_PAGES  0.02    /* i.e. 2% of rel_pages */ > > > > So up to an additional 2% of all pages can have the all-visible bit > > unset with "index_cleanup = auto".

Re: Choosing an index on partitioned tables.

2021-09-07 Thread Brent Wood
Hi Tim, I've had good success with TimescaleDB for large timesries databases (40b readings). https://www.timescale.com/ You turn your timestamp table into a Timescale hypertable and it looks after the indexing and partitioning automatically, with the table accessed like a normal postgres table