Re: seawasp failing, maybe in glibc allocator

2021-06-18 Thread Thomas Munro
On Sat, May 22, 2021 at 12:25 PM Andres Freund wrote: > On 2021-05-21 15:57:01 -0700, Andres Freund wrote: > > I found the LLVM commit to blame (c8fc5e3ba942057d6c4cdcd1faeae69a28e7b671). > > Contacting the author and reading the change to see if I can spit the > > issue myself. > > Hrmpf. It's a

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Fabien COELHO
There was a silent API breakage (same API, different behavior, how nice…) in llvm main that Andres figured out, which will have to be fixed at some point, so this is reminder that it is still a todo… If it were *our* todo, that would be one thing; but it isn't. Over on the other thread[1] we

Re: Centralizing protective copying of utility statements

2021-06-18 Thread Julien Rouhaud
On Fri, Jun 18, 2021 at 11:24:00AM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > On Fri, Jun 18, 2021 at 10:24:20AM -0400, Tom Lane wrote: > >> Maybe "if true, pstmt's node tree must not be modified" ? > > > Thanks, I find it way better! > > OK, pushed that way, and with a couple other com

Re: PG 14 release notes, first draft

2021-06-18 Thread Peter Geoghegan
On Mon, Jun 14, 2021 at 1:11 PM Bruce Momjian wrote: > FYI, the most recent PG 14 relnote doc build is at: > > https://momjian.us/pgsql_docs/release-14.html I just pushed a commit that makes the existing vacuum_index_cleanup reloption and INDEX_CLEANUP VACUUM parameter support disabling t

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-06-18 Thread Peter Geoghegan
On Thu, Jun 17, 2021 at 7:26 PM Peter Geoghegan wrote: > Thanks for the review! > > Attached is v3, which has all the changes that you suggested (plus the > doc stuff from Justin). Just pushed a version of that with much improved documentation. Thanks again -- Peter Geoghegan

Re: pgbench logging broken by time logic changes

2021-06-18 Thread Michael Paquier
On Sat, Jun 19, 2021 at 11:59:16AM +1200, Thomas Munro wrote: > Thanks for looking so far. It's the weekend here and I need to > unplug, but I'll test these changes and if all looks good push on > Monday. Thanks for the update. Have a good weekend. -- Michael signature.asc Description: PGP sig

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-18 Thread Mark Dilger
> On Jun 17, 2021, at 11:34 PM, Peter Smith wrote: > > I tried your patch. Thanks for the quick and thorough review! > (2) New column "disabled_by_error". > > I wondered if there was actually any need for this column. Isn't the > same information conveyed by just having "subenabled" = false,

Re: Speed up pg_checksums in cases where checksum already set

2021-06-18 Thread Greg Sabino Mullane
On Fri, Jun 18, 2021 at 1:57 AM Michael Paquier wrote: > This doc addition is a bit confusing, as it could mean that each file > has just one single checksum. We could be more precise, say: > "When enabling checksums, each relation file block with a changed > checksum is rewritten in place." >

Re: pgbench logging broken by time logic changes

2021-06-18 Thread Thomas Munro
On Fri, Jun 18, 2021 at 12:31 PM Michael Paquier wrote: > On Fri, Jun 18, 2021 at 12:49:42AM +1200, Thomas Munro wrote: > > Yeah I've been catching up with these threads. > > Thomas, do you want me to look more at this issue? I don't feel > comfortable with the idea of doing anything if you are p

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Thomas Munro
On Sat, Jun 19, 2021 at 9:46 AM Tom Lane wrote: > Fabien COELHO writes: > >> It'd sure be nice if seawasp stopped spamming the buildfarm failure log, > >> too. > > > There was a silent API breakage (same API, different behavior, how nice…) > > in llvm main that Andres figured out, which will have

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Fabien COELHO
Hello Tom, Could you please just shut down the animal until that's dealt with? The test is failing because there is a problem, and shuting down the test to improve a report does not in any way help to fix it, it just helps to hide it. Our buildfarm is run for the use of the Postgres projec

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Tom Lane
Fabien COELHO writes: >> Could you please just shut down the animal until that's dealt with? > The test is failing because there is a problem, and shuting down the test > to improve a report does not in any way help to fix it, it just helps to > hide it. Our buildfarm is run for the use of the

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Fabien COELHO
Hello Tom, So I'm not very confident that the noise will go away quickly, sorry. Could you please just shut down the animal until that's dealt with? Hmmm… Obviously I can. However, please note that the underlying logic of "a test is failing, let's just remove it" does not sound right to m

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Tom Lane
Fabien COELHO writes: >> It'd sure be nice if seawasp stopped spamming the buildfarm failure log, >> too. > There was a silent API breakage (same API, different behavior, how nice…) > in llvm main that Andres figured out, which will have to be fixed at some > point, so this is reminder that it

Re: Version reporting in pgbench

2021-06-18 Thread Fabien COELHO
Note that if no connections are available, then you do not get the version, which may be a little bit strange. Attached v3 prints out the local version in that case. Not sure whether it is worth the effort. I'm inclined to think that the purpose of that output is mostly to report the server v

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Fabien COELHO
It'd sure be nice if seawasp stopped spamming the buildfarm failure log, too. There was a silent API breakage (same API, different behavior, how nice…) in llvm main that Andres figured out, which will have to be fixed at some point, so this is reminder that it is still a todo… Not sure when

Re: Version reporting in pgbench

2021-06-18 Thread Tom Lane
Fabien COELHO writes: > Note that if no connections are available, then you do not get the > version, which may be a little bit strange. Attached v3 prints out the > local version in that case. Not sure whether it is worth the effort. I'm inclined to think that the purpose of that output is mos

Re: Version reporting in pgbench

2021-06-18 Thread Fabien COELHO
Hello Tom, Why not move the printVersion call right after the connection is created, at line 6374? I started with that, and one of the 001_pgbench_with_server.pl tests fell over --- it was expecting no stdout output before a "Perhaps you need to do initialization" failure. If you don't mind

Re: A few nuances about specifying the timeline with START_REPLICATION

2021-06-18 Thread Heikki Linnakangas
On 18/06/2021 22:55, Jeff Davis wrote: On Fri, 2021-06-18 at 21:48 +0300, Heikki Linnakangas wrote: On 18/06/2021 20:27, Jeff Davis wrote: We could teach it to look into the timeline history to find the correct file, though. That's how recovery_target_timeline behaves, and it would match my in

Re: Race condition in InvalidateObsoleteReplicationSlots()

2021-06-18 Thread Álvaro Herrera
Apologies, I inadvertently sent the version before I added a maximum number of iterations in the final loop. -- Álvaro Herrera Valdivia, Chile "La fuerza no está en los medios físicos sino que reside en una voluntad indomable" (Gandhi) >From 1492e9468ecd86167a1253c4a2e9b31139835649 Mon Sep

Re: Race condition in InvalidateObsoleteReplicationSlots()

2021-06-18 Thread Álvaro Herrera
On 2021-Jun-11, Álvaro Herrera wrote: > I tried hard to make this stable, but it just isn't (it works fine one > thousand runs, then I grab some coffee and run it once more and that one > fails. Why? that's not clear to me). Attached is the last one I have, > in case somebody wants to make it b

Re: pg_stats and range statistics

2021-06-18 Thread Tomas Vondra
On 6/18/21 6:22 PM, Egor Rogov wrote: Hi, Statistics for range types are not currently exposed in pg_stats view (i.e. STATISTIC_KIND_RANGE_LENGTH_HISTOGRAM and STATISTIC_KIND_BOUNDS_HISTOGRAM). Shouldn't they? If so, here is a patch for adding them. I think they should be exposed - I don'

Re: PoC: Using Count-Min Sketch for join cardinality estimation

2021-06-18 Thread Tomas Vondra
On 6/18/21 9:54 PM, John Naylor wrote: On Fri, Jun 18, 2021 at 3:43 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > Sorry, I'm not sure what you mean by "we set the number of MCVs to the > number of histograms" :-( > > When you say "MCV limit" you mean that we limit the n

Re: A few nuances about specifying the timeline with START_REPLICATION

2021-06-18 Thread Jeff Davis
On Fri, 2021-06-18 at 21:48 +0300, Heikki Linnakangas wrote: > On 18/06/2021 20:27, Jeff Davis wrote: > We could teach it to look into the timeline history to find the > correct > file, though. That's how recovery_target_timeline behaves, and it would match my intuition better if START_REPLICATIO

Re: PoC: Using Count-Min Sketch for join cardinality estimation

2021-06-18 Thread John Naylor
On Fri, Jun 18, 2021 at 3:43 PM Tomas Vondra wrote: > Sorry, I'm not sure what you mean by "we set the number of MCVs to the > number of histograms" :-( > > When you say "MCV limit" you mean that we limit the number of items to > statistics target, right? I agree plan time is one concern - but it

Re: PoC: Using Count-Min Sketch for join cardinality estimation

2021-06-18 Thread Tomas Vondra
On 6/18/21 7:03 PM, John Naylor wrote: On Wed, Jun 16, 2021 at 8:23 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > Not really, but to be fair even for the histograms it's only really > supported by "seems to work in practice" :-( Hmm, we cite a theoretical result in analyze

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-18 Thread Mark Dilger
> On Jun 17, 2021, at 9:47 PM, Amit Kapila wrote: > > (a) The patch > seem to be assuming that the error can happen only by the apply worker > but I think the constraint violation can happen via one of the table > sync workers as well You are right. Peter mentioned the same thing, and it is

Re: Supply restore_command to pg_rewind via CLI argument

2021-06-18 Thread Alexey Kondratov
Hi, On Fri, Jun 18, 2021 at 5:42 PM Andrey Borodin wrote: > > If we run 'pg_rewind --restore-target-wal' there must be restore_command in > config of target installation. But if the config is not within > $PGDATA\postgresql.conf pg_rewind cannot use it. > If we run postmaster with `-c > config

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Tom Lane
Alvaro Herrera writes: > Hello, this problem is still happening; serinus' configure output says > it's running a snapshot from 20210527, and Fabien mentioned downthread > that his compiler stopped complaining on 2021-06-05. Andres, maybe the > compiler in serinus is due for an update too? Yeah,

Re: A few nuances about specifying the timeline with START_REPLICATION

2021-06-18 Thread Heikki Linnakangas
On 18/06/2021 20:27, Jeff Davis wrote: A few questions about this comment in walsender.c, originating in commit abfd192b1b5b: /* * Found the requested timeline in the history. Check that * requested startpoint is on that timeline in our history. * * This is quite loose on purpose. We onl

Re: Version reporting in pgbench

2021-06-18 Thread Tom Lane
Fabien COELHO writes: > Why not move the printVersion call right after the connection is created, > at line 6374? I started with that, and one of the 001_pgbench_with_server.pl tests fell over --- it was expecting no stdout output before a "Perhaps you need to do initialization" failure. If you

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-18 Thread Alvaro Herrera
On 2021-Jun-03, Michael Paquier wrote: > Hi all, > > serinus has been complaining about the new gcd functions in 13~: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=serinus&dt=2021-06-03%2003%3A44%3A14 Hello, this problem is still happening; serinus' configure output says it's running

Re: Version reporting in pgbench

2021-06-18 Thread Fabien COELHO
Hello Tom, One point here is that printing the server version requires access to a connection, which printResults() hasn't got because we already closed all the connections by that point. I solved that by printing the banner during the initial connection that gets the scale factor, does vacuum

Re: Add version macro to libpq-fe.h

2021-06-18 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Jun-18, Tom Lane wrote: >> I guess one unresolved question is whether we want to mention these in >> the SGML docs. I vote "no", because it'll raise the maintenance cost >> noticeably. But I can see an argument on the other side. > Well, if we do want docs for t

Re: Add version macro to libpq-fe.h

2021-06-18 Thread Alvaro Herrera
On 2021-Jun-18, Tom Lane wrote: > Alvaro Herrera writes: > > So I'm +1 on adding this "feature macro". > > Concretely, how about the attached? Seems OK to me. We can just accumulate any similar ones in the future nearby. > (I also got rid of a recently-added > extra comma. While the compiler

Re: Pipeline mode and PQpipelineSync()

2021-06-18 Thread Alvaro Herrera
On 2021-Jun-16, Boris Kolpackov wrote: > Specifically, the documentation[1] > makes it sound like the use of PQpipelineSync() is optional (34.5.1.1 > "Issuing Queries"): Hmm. My intention here was to indicate that you should have PQpipelineSync *somewhere*, but that the server was free to start

Re: Add version macro to libpq-fe.h

2021-06-18 Thread Tom Lane
Alvaro Herrera writes: > So I'm +1 on adding this "feature macro". Concretely, how about the attached? (I also got rid of a recently-added extra comma. While the compilers we use might not warn about that, it seems unwise to assume that no user's compiler will.) I guess one unresolved question

A few nuances about specifying the timeline with START_REPLICATION

2021-06-18 Thread Jeff Davis
A few questions about this comment in walsender.c, originating in commit abfd192b1b5b: /* * Found the requested timeline in the history. Check that * requested startpoint is on that timeline in our history. * * This is quite loose on purpose. We only check that we didn't * fork off the reques

Version reporting in pgbench

2021-06-18 Thread Tom Lane
I see that commit 547f04e73 caused pgbench to start printing its version number. I think that's a good idea in general, but it appears to me that next to no thought went into the details (as perhaps evidenced by the fact that the commit message doesn't even mention it). I've got two beefs with ho

Re: PoC: Using Count-Min Sketch for join cardinality estimation

2021-06-18 Thread John Naylor
On Wed, Jun 16, 2021 at 8:23 PM Tomas Vondra wrote: > Not really, but to be fair even for the histograms it's only really > supported by "seems to work in practice" :-( Hmm, we cite a theoretical result in analyze.c, but I don't know if there is something better out there: * The following choi

Re: disfavoring unparameterized nested loops

2021-06-18 Thread John Naylor
On Fri, Jun 18, 2021 at 6:20 AM Ashutosh Bapat wrote: > I have not come across many papers which leverage this idea. Googling > "selectivity estimation confidence interval", does not yield many > papers. Although I found [1] to be using a similar idea. So may be > there's not merit in this idea,

pg_stats and range statistics

2021-06-18 Thread Egor Rogov
Hi, Statistics for range types are not currently exposed in pg_stats view (i.e. STATISTIC_KIND_RANGE_LENGTH_HISTOGRAM and STATISTIC_KIND_BOUNDS_HISTOGRAM). Shouldn't they? If so, here is a patch for adding them. The following is a simple example of what it looks like: CREATE TABLE test(r in

Re: pgbench bug candidate: negative "initial connection time"

2021-06-18 Thread Yugo NAGATA
Hello Horiguchi-san, Fabien, On Fri, 18 Jun 2021 15:58:48 +0200 (CEST) Fabien COELHO wrote: > > >>>/* must be something wrong */ > >>>pg_log_error("%s() failed: %m", SOCKET_WAIT_METHOD); > >>>goto done; > >> > >> Should say such like "thread %d aborted: %s() failed: ...

Re: Centralizing protective copying of utility statements

2021-06-18 Thread Tom Lane
Julien Rouhaud writes: > On Fri, Jun 18, 2021 at 10:24:20AM -0400, Tom Lane wrote: >> Maybe "if true, pstmt's node tree must not be modified" ? > Thanks, I find it way better! OK, pushed that way, and with a couple other comment tweaks from an additional re-reading. rega

Re: Centralizing protective copying of utility statements

2021-06-18 Thread Julien Rouhaud
On Fri, Jun 18, 2021 at 10:24:20AM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > On Thu, Jun 17, 2021 at 01:03:29PM -0400, Tom Lane wrote: > > + * readOnlyTree: treat pstmt's node tree as read-only > > > Maybe it's because I'm not a native english speaker, or because it's quite > > late her

Re: Add version macro to libpq-fe.h

2021-06-18 Thread Alvaro Herrera
On 2021-Jun-18, Boris Kolpackov wrote: > Tom Lane writes: > > > I think putting a version number as such in there is a truly > > horrid idea. However, I could get behind adding a boolean flag > > that says specifically whether the pipeline feature exists. > > Then you'd do something like > > >

Re: Centralizing protective copying of utility statements

2021-06-18 Thread Tom Lane
Julien Rouhaud writes: > On Thu, Jun 17, 2021 at 01:03:29PM -0400, Tom Lane wrote: > + * readOnlyTree: treat pstmt's node tree as read-only > Maybe it's because I'm not a native english speaker, or because it's quite > late here, but I don't find "treat as read-only" really clear. I don't have

Re: Add version macro to libpq-fe.h

2021-06-18 Thread Tom Lane
Boris Kolpackov writes: > Tom Lane writes: >> I think putting a version number as such in there is a truly >> horrid idea. However, I could get behind adding a boolean flag >> that says specifically whether the pipeline feature exists. > That would be even better, but I agree with what others h

Re: pgbench bug candidate: negative "initial connection time"

2021-06-18 Thread Fabien COELHO
/* must be something wrong */ pg_log_error("%s() failed: %m", SOCKET_WAIT_METHOD); goto done; Should say such like "thread %d aborted: %s() failed: ...". After having a lookg, there are already plenty such cases. I'd say not to change anything for beta, and think of it

Re: Add version macro to libpq-fe.h

2021-06-18 Thread Boris Kolpackov
Tom Lane writes: > I think putting a version number as such in there is a truly > horrid idea. However, I could get behind adding a boolean flag > that says specifically whether the pipeline feature exists. > Then you'd do something like > > #ifdef LIBPQ_HAS_PIPELINING > > rather than embeddin

Re: Refactor "mutually exclusive options" error reporting code in parse_subscription_options

2021-06-18 Thread Bharath Rupireddy
On Thu, Jun 10, 2021 at 6:36 PM Bharath Rupireddy wrote: > > On Thu, Jun 10, 2021 at 9:17 AM Bharath Rupireddy > wrote: > > > I don't know the answer; my guess is that all this has become obsolete > > > and the whole Assert and the dodgy comment can just be deleted. > > > > Hm. I get it. Unfortun

Re: pgbench bug candidate: negative "initial connection time"

2021-06-18 Thread Fabien COELHO
Hello, Doing this means we regard any state other than CSTATE_FINISHED as aborted. So, the current goto's to done in threadRun are effectively aborting a part or the all clients running on the thread. So for example the following place: pgbench.c:6713 /* must be something wrong */

Re: Asymmetric partition-wise JOIN

2021-06-18 Thread Alexander Pyhalov
Andrey Lepikhov писал 2021-05-27 07:27: Next version of the patch. For searching any problems I forced this patch during 'make check' tests. Some bugs were found and fixed. Hi. I've tested this patch and haven't found issues, but I have some comments. src/backend/optimizer/path/joinrels.c:

Supply restore_command to pg_rewind via CLI argument

2021-06-18 Thread Andrey Borodin
Hi hackers! Starting from v13 pg_rewind can use restore_command if it lacks necessary WAL segments. And this is awesome for HA clusters with many nodes! Thanks to everyone who worked on the feature! Here's some feedback on how to make things even better. If we run 'pg_rewind --restore-target-w

Re: Added schema level support for publication.

2021-06-18 Thread Ajin Cherian
On Thu, Jun 17, 2021 at 12:41 AM vignesh C wrote: > Thanks for reporting it, the attached patch is a rebased version of > the patch with few review comment fixes, I will reply with the comment > fixes to the respective mails. > I've applied the patch, it applies cleand and ran "make check" and t

Re: row filtering for logical replication

2021-06-18 Thread Amit Kapila
On Wed, Jun 9, 2021 at 5:33 AM Peter Smith wrote: > > On Mon, May 10, 2021 at 11:42 PM Euler Taveira wrote: > > > > On Mon, May 10, 2021, at 5:19 AM, Peter Smith wrote: > > > > AFAIK this is the latest patch available, but FYI it no longer applies > > cleanly on HEAD. > > > > Peter, the last patc

Re: disfavoring unparameterized nested loops

2021-06-18 Thread Ashutosh Bapat
> > The problem I have with this idea is that I really don't know how to > properly calculate what the risk_factor should be set to. It seems > easy at first to set it to something that has the planner avoid these > silly 1-row estimate nested loop mistakes, but I think what we'd set > the risk_fa

Re: fdatasync performance problem with large number of DB files

2021-06-18 Thread Fujii Masao
On 2021/06/04 23:39, Justin Pryzby wrote: You said switching to SIGHUP "would have zero effect"; but, actually it allows an admin who's DB took a long time in recovery/startup to change the parameter without shutting down the service. This mitigates the downtime if it crashes again. I think

RE: locking [user] catalog tables vs 2pc vs logical rep

2021-06-18 Thread osumi.takami...@fujitsu.com
On Friday, June 18, 2021 11:41 AM osumi.takami...@fujitsu.com wrote: > On Thursday, June 17, 2021 10:34 PM Simon Riggs > wrote: > > On Thu, Jun 17, 2021 at 12:57 PM Amit Kapila > > wrote: > > > On Thu, Jun 17, 2021 at 4:27 PM Amit Kapila > > > > > wrote: > > > > > > > > On Thu, Jun 17, 2021 a

Re: [PoC] Federated Authn/z with OAUTHBEARER

2021-06-18 Thread Heikki Linnakangas
On 08/06/2021 19:37, Jacob Champion wrote: We've been working on ways to expand the list of third-party auth methods that Postgres provides. Some example use cases might be "I want to let anyone with a Google account read this table" or "let anyone who belongs to this GitHub organization connect

Re: pgbench bug candidate: negative "initial connection time"

2021-06-18 Thread Kyotaro Horiguchi
At Thu, 17 Jun 2021 11:52:04 +0200 (CEST), Fabien COELHO wrote in > > Ok. I gave up to change the state in threadRun. Instead, I changed the > > condition at the end of bench, which enables to report abortion due to > > socket errors. > > > > +@@ -6480,7 +6490,7 @@ main(int argc, char **argv) >