Re: WIP: WAL prefetch (another approach)

2021-12-10 Thread Ashutosh Sharma
Hi Thomas, I am unable to apply these new set of patches on HEAD. Can you please share the rebased patch or if you have any work branch can you please point it out, I will refer to it for the changes. -- With Regards, Ashutosh sharma. On Tue, Nov 23, 2021 at 3:44 PM Thomas Munro wrote: > On Mo

Re: Replication slot drop message is sent after pgstats shutdown.

2021-12-10 Thread Masahiko Sawada
On Wed, Sep 1, 2021 at 12:37 PM Andres Freund wrote: > > Hi, > > On 2021-09-01 10:05:18 +0900, Masahiko Sawada wrote: > > On Wed, Sep 1, 2021 at 2:39 AM Andres Freund wrote: > > > On 2021-08-31 18:34:12 +0900, Kyotaro Horiguchi wrote: > > > > At Tue, 31 Aug 2021 17:14:45 +0900, Masahiko Sawada >

Re: port conflicts when running tests concurrently on windows.

2021-12-10 Thread Peter Eisentraut
On 09.12.21 19:41, Andres Freund wrote: Withhttps://github.com/anarazel/postgres/commit/046203741803da863f6129739fd215f8a32ec357 all tests pass. pg_regress requires PG_REGRESS_SOCK_DIR because it checks for TMPDIR, but windows only has TMP and TEMP. This looks reasonable so far. The commit mes

RE: track_io_timing default setting

2021-12-10 Thread Jakub Wartak
> Can we change the default setting of track_io_timing to on? +1 for better observability by default. > I can't imagine a lot of people who care much about its performance impact > will be running the latest version of PostgreSQL on ancient/weird systems  > that have slow clock access. (And the

Re: parallel vacuum comments

2021-12-10 Thread Amit Kapila
On Thu, Dec 9, 2021 at 6:05 PM Masahiko Sawada wrote: > > On Thu, Dec 9, 2021 at 7:44 PM Amit Kapila wrote: > > Agreed with the above two points. > > I've attached updated patches that incorporated the above comments > too. Please review them. > I have made the following minor changes to the 000

Re: Column Filtering in Logical Replication

2021-12-10 Thread Peter Eisentraut
On 02.12.21 15:23, Alvaro Herrera wrote: I took the latest posted patch, rebased on current sources, fixed the conflicts, and pgindented. No further changes. Here's the result. All tests are passing for me. Some review comments that were posted have not been addressed yet; I'll look into that

Re: Column Filtering in Logical Replication

2021-12-10 Thread Alvaro Herrera
On 2021-Dec-10, Peter Eisentraut wrote: > I looked through this a bit. You had said that you are still going to > integrate past review comments, so I didn't look to deeply before you get to > that. Thanks for doing this! As it happens I've spent the last couple of days working on some of these

Re: Post-CVE Wishlist

2021-12-10 Thread Heikki Linnakangas
On 09/12/2021 17:24, Peter Eisentraut wrote: On 07.12.21 19:49, Jacob Champion wrote: = Implicit TLS = Reactions to implicit TLS were mixed, from "we should not do this" to "it might be nice to have the option, from a technical standpoint". Both a separate-port model and a shared-port model wer

Re: Added schema level support for publication.

2021-12-10 Thread vignesh C
On Fri, Dec 10, 2021 at 6:24 AM Alvaro Herrera wrote: > > I just noticed that this (commit 5a2832465fd8) added a separate catalog > to store schemas which are part of a publication, side-by-side with the > catalog to store relations which are part of a publication. This seems > a strange way to r

Re: Non-superuser subscription owners

2021-12-10 Thread Robert Haas
On Thu, Dec 9, 2021 at 11:15 PM Amit Kapila wrote: > Yeah, to me also (b) sounds better than (a). However, a few points > that we might want to consider in that regard are as follows: 1. > locking the subscription for each transaction will add new blocking > areas considering we acquire AccessExcl

Probable memory leak with ECPG and AIX

2021-12-10 Thread Guillaume Lelarge
Hello, Our customer thinks he has found a memory leak on ECPG and AIX. The code is quite simple. It declares a cursor, opens it, and fetches the only line available in the table many times. After some time, the client crashes with a segfault error. According to him, it consumed around 256MB. What

Re: Non-superuser subscription owners

2021-12-10 Thread Andrew Dunstan
On 12/10/21 09:09, Robert Haas wrote: > On Thu, Dec 9, 2021 at 11:15 PM Amit Kapila wrote: >> Yeah, to me also (b) sounds better than (a). However, a few points >> that we might want to consider in that regard are as follows: 1. >> locking the subscription for each transaction will add new block

Re: track_io_timing default setting

2021-12-10 Thread Tom Lane
Jeff Janes writes: > Can we change the default setting of track_io_timing to on? That adds a very significant amount of overhead on some platforms (gettimeofday is not cheap if it requires a kernel call). And I doubt the claim that the average Postgres user needs this, and doubt even more that t

Re: Remove pg_strtouint64(), use strtoull() directly

2021-12-10 Thread Tom Lane
Peter Eisentraut writes: > Therefore, remove pg_strtouint64(), and use strtoull() directly in all > call sites. Our experience with the variable size of "long" has left a sufficiently bad taste in my mouth that I'm not enthused about adding hard-wired assumptions that "long long" is identical to

Re: track_io_timing default setting

2021-12-10 Thread Laurenz Albe
On Fri, 2021-12-10 at 10:20 -0500, Tom Lane wrote: > Jeff Janes writes: > > Can we change the default setting of track_io_timing to on? > > That adds a very significant amount of overhead on some platforms > (gettimeofday is not cheap if it requires a kernel call).  And I > doubt the claim that t

Re: track_io_timing default setting

2021-12-10 Thread Tomas Vondra
On 12/10/21 17:22, Laurenz Albe wrote: On Fri, 2021-12-10 at 10:20 -0500, Tom Lane wrote: Jeff Janes writes: Can we change the default setting of track_io_timing to on? That adds a very significant amount of overhead on some platforms (gettimeofday is not cheap if it requires a kernel call).

RE: [EXTERNAL] Re: track_io_timing default setting

2021-12-10 Thread Godfrin, Philippe E
>-Original Message- >Sent: Friday, December 10, 2021 9:20 AM >To: Jeff Janes >Cc: PostgreSQL Hackers >Subject: [EXTERNAL] Re: track_io_timing default setting >Jeff Janes writes: > Can we change the default setting of track_io_timing to on? >That adds a very significant amount of overhe

RE: [EXTERNAL] Re: should we document an example to set multiple libraries in shared_preload_libraries?

2021-12-10 Thread Godfrin, Philippe E
On Wed, Dec 1, 2021 at 5:15 AM Tom Lane wrote: > > Justin Pryzby writes: > > +1 to document it, but it seems like the worse problem is allowing > > +the admin to > > write a configuration which causes the server to fail to start, > > without having issued a warning. > > > I think you could fix

Re: speed up verifying UTF-8

2021-12-10 Thread Heikki Linnakangas
On 20/10/2021 00:42, John Naylor wrote: I've decided I'm not quite comfortable with the additional complexity in the build system introduced by the SIMD portion of the previous patches. It would make more sense if the pure C portion were unchanged, but with the shift-based DFA plus the bitwise

RE: [EXTERNAL] Re: speed up verifying UTF-8

2021-12-10 Thread Godfrin, Philippe E
>-Original Message- >From: Heikki Linnakangas >Sent: Friday, December 10, 2021 12:34 PM >To: John Naylor ; Vladimir Sitnikov > >Cc: pgsql-hackers ; Amit Khandekar >; Thomas Munro ; Greg Stark > >Subject: [EXTERNAL] Re: speed up verifying UTF-8 > >On 20/10/2021 00:42, John Naylor wrote:

Re: Column Filtering in Logical Replication

2021-12-10 Thread Alvaro Herrera
On 2021-Sep-02, Alvaro Herrera wrote: > On 2021-Sep-02, Rahila Syed wrote: > > > After thinking about this, I think it is best to remove the entire table > > from publication, > > if a column specified in the column filter is dropped from the table. > > Hmm, I think it would be cleanest to give

Re: Column Filtering in Logical Replication

2021-12-10 Thread Alvaro Herrera
On 2021-Dec-10, Alvaro Herrera wrote: > I thought about this some more and realized that our earlier conclusions > were wrong or at least inconvenient. I think that the best behavior if > you drop a column from a table is to remove the column from the > publication column list, and do nothing els

Question about 001_stream_rep.pl recovery test

2021-12-10 Thread David Zhang
Hi Hackers, Inside the test script `src/test/recovery/t/001_stream_rep.pl`, a comment at line 30 says `my_backup` is "not mandatory",  30 # Take backup of standby 1 (not mandatory, but useful to check if  31 # pg_basebackup works on a standby).  32 $node_standby_1->backup($backup_name); howev

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-12-10 Thread Peter Geoghegan
On Tue, Nov 30, 2021 at 11:52 AM Peter Geoghegan wrote: > I haven't had time to work through any of your feedback just yet -- > though it's certainly a priority for. I won't get to it until I return > home from PGConf NYC next week. Attached is v3, which works through most of your (Andres') feedb

Re: Appetite for Frama-C annotations?

2021-12-10 Thread Colin Gilbert
Thank you all very much for your inputs! To summarise: I asked about the possibility of adding ACSL annotations to the codebase and the responses ranged from nonplussed on one end of the spectrum to some degree of enthusiasm on the other. It was suggested that libpsql would be a better initial targ

Re: A test for replay of regression tests

2021-12-10 Thread Andrew Dunstan
On 12/9/21 15:15, Thomas Munro wrote: > On Fri, Dec 10, 2021 at 2:12 AM Andrew Dunstan wrote: >> The new version appears to set an empty --bindir for pg_regress. Is that >> right? > It seems to be necessary to find eg psql, since --bindir='' means > "expect $PATH to contain the installed binarie

Re: A test for replay of regression tests

2021-12-10 Thread Andres Freund
Hi, On 2021-12-10 12:58:01 +1300, Thomas Munro wrote: > > What's the relation of this to the rest? > > Someone decided that allow_streaming should imply max_connections = > 10, but we need ~20 to run the parallel regression test schedule. > However, I can just as easily move that to a local adjus

range_agg with multirange inputs

2021-12-10 Thread Paul Jungwirth
Here is a patch adding range_agg(anymultirange). Previously range_agg only accepted anyrange. Here is a bug report from last month requesting this addition: https://www.postgresql.org/message-id/CAOC8YUcOtAGscPa31ik8UEMzgn8uAWA09s6CYOGPyP9_cBbWTw%40mail.gmail.com As that message points out, ra

isolationtester: add session name to application name

2021-12-10 Thread Andres Freund
Hi, When writing / debugging an isolation test it's sometimes useful to see which session holds what lock etc. I find it kind of painful to map pg_stat_activity / pg_locks / log output to the isolationtester spec. Sometimes its easy enough to infer identity based on a statement, but far from all t

Re: PATCH: generate fractional cheapest paths in generate_orderedappend_path

2021-12-10 Thread Tomas Vondra
On 12/10/21 00:51, Arne Roland wrote: Hi, thanks for the reply! From: Tomas Vondra Sent: Thursday, December 2, 2021 20:58 Subject: Re: PATCH: generate fractional cheapest paths in generate_orderedappend_path > [...] > Well, I mentioned three open questions in my first message, and I don'

Re: GiST operator class for bool

2021-12-10 Thread Tomas Vondra
On 12/6/21 22:35, Pavel Luzanov wrote: Hello, I don't see any changes in the documentation.[1] Should bool appear in the looong list of supported operator classes? You're right, I forgot to update the list of data types in the docs. Fixed, thanks for the report. regards -- Tomas Vondra

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-10 Thread Andres Freund
Hi, On 2021-11-29 18:51:37 -0800, Peter Geoghegan wrote: > One thing that's still unclear is what the new elevel should be for > the ereport messages that used to be either LOG (for VACUUM VERBOSE) > or DEBUG2 (for everything else) -- what should I change them to now? > For now I've done taken the

Re: Add client connection check during the execution of the query

2021-12-10 Thread Thomas Munro
On Tue, Oct 12, 2021 at 3:10 AM Maksim Milyutin wrote: > Good work. I have tested your patch on Linux and FreeBSD on three basic > cases: client killing, cable breakdown (via manipulations with firewall) > and silent closing client connection before completion of previously > started query in asyn

Re: [PATCH] Don't block HOT update by BRIN index

2021-12-10 Thread Tomas Vondra
On 12/6/21 02:47, Tom Lane wrote: Tomas Vondra writes: On 12/5/21 21:16, Tom Lane wrote: Oh, geez. *Please* let us not add another regression failure mode like the ones that afflict stats.sql. We do not need a doubling of that failure rate. I suggest just removing this test. Whooops. Agr

Re: Non-superuser subscription owners

2021-12-10 Thread Amit Kapila
On Fri, Dec 10, 2021 at 7:39 PM Robert Haas wrote: > > On Thu, Dec 9, 2021 at 11:15 PM Amit Kapila wrote: > > Yeah, to me also (b) sounds better than (a). However, a few points > > that we might want to consider in that regard are as follows: 1. > > locking the subscription for each transaction w

Re: Add client connection check during the execution of the query

2021-12-10 Thread Andres Freund
Hi, On 2021-12-11 17:41:34 +1300, Thomas Munro wrote: > --- a/src/backend/libpq/pqcomm.c > +++ b/src/backend/libpq/pqcomm.c > @@ -1959,22 +1959,36 @@ pq_settcpusertimeout(int timeout, Port *port) > bool > pq_check_connection(void) > { > -#if defined(POLLRDHUP) > - /* > - * POLLRDHUP is

Re: parallel vacuum comments

2021-12-10 Thread Andres Freund
Hi, On 2021-10-30 14:21:01 -0700, Andres Freund wrote: > Due to bug #17245: [1] I spent a considerably amount of time looking at vacuum > related code. And I found a few things that I think could stand improvement: While working on the fix for #17255 (more specifically some cleanup that Peter sug

Re: Add client connection check during the execution of the query

2021-12-10 Thread Thomas Munro
On Sat, Dec 11, 2021 at 6:11 PM Andres Freund wrote: > Yuck. Is there really no better way to deal with this? What kind of errors is > this trying to handle transparently? Afaics this still changes when we'd > e.g. detect postmaster death. The problem is that WaitEventSetWait() only reports the l

Re: Skipping logical replication transactions on subscriber side

2021-12-10 Thread Amit Kapila
On Fri, Dec 10, 2021 at 11:14 AM Masahiko Sawada wrote: > > On Thu, Dec 9, 2021 at 6:16 PM Amit Kapila wrote: > > > > On Thu, Dec 9, 2021 at 2:24 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Dec 9, 2021 at 11:47 AM Amit Kapila > > > wrote: > > > > > > > > I am thinking that we can start

Re: Probable memory leak with ECPG and AIX

2021-12-10 Thread Justin Pryzby
On Fri, Dec 10, 2021 at 03:40:50PM +0100, Guillaume Lelarge wrote: > Hello, > > Our customer thinks he has found a memory leak on ECPG and AIX. > > The code is quite simple. It declares a cursor, opens it, and fetches the > only line available in the table many times. After some time, the client

Re: Probable memory leak with ECPG and AIX

2021-12-10 Thread Guillaume Lelarge
Le sam. 11 déc. 2021 à 07:52, Justin Pryzby a écrit : > On Fri, Dec 10, 2021 at 03:40:50PM +0100, Guillaume Lelarge wrote: > > Hello, > > > > Our customer thinks he has found a memory leak on ECPG and AIX. > > > > The code is quite simple. It declares a cursor, opens it, and fetches the > > only