Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-08-29 Thread Noah Misch
On Mon, May 25, 2020 at 12:00:33AM -0700, Noah Misch wrote: > On Mon, Apr 06, 2020 at 09:18:47PM -0700, Noah Misch wrote: > > On Mon, Apr 06, 2020 at 02:46:09PM -0400, Tom Lane wrote: > > > Noah Misch writes: > > > > On Wed, Mar 25, 2020 at 04:42:31PM -0400, Tom Lane wrote: > > > >> So I think wha

Rare link canary failure in dblink test

2020-08-29 Thread Thomas Munro
Hi, A couple of recent cases where an error "libpq is incorrectly linked to backend functions" broke the dblink test: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2020-08-22%2002:55:22 | REL9_6_STABLE | Windows https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=l

Re: Windows buildfarm members vs. new async-notify isolation test

2020-08-29 Thread Tom Lane
Thomas Munro writes: > I made a thing to watch out for low probability BF failures and it > told me that a similar failure in async-notify might have reappeared > on brolga: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=brolga&dt=2020-07-15%2008:30:11 > | REL_10_STABLE > [ etc ]

Re: Windows buildfarm members vs. new async-notify isolation test

2020-08-29 Thread Thomas Munro
On Thu, Dec 12, 2019 at 9:11 AM Tom Lane wrote: > Amit Kapila writes: > > I am convinced by your points. So +1 for your proposed patch. I have > > already reviewed it yesterday and it appears fine to me. > > OK, pushed. Thanks for reviewing! I made a thing to watch out for low probability BF

Re: Background writer and checkpointer in crash recovery

2020-08-29 Thread Tom Lane
Thomas Munro writes: > Once we had the checkpointer running, we could also consider making > the end-of-recovery checkpoint optional, or at least the wait for it > to complete. I haven't shown that in this patch, but it's just > different checkpoint request flags and possibly an end-of-recovery >

Re: Disk-based hash aggregate's cost model

2020-08-29 Thread Tomas Vondra
On Fri, Aug 28, 2020 at 06:32:38PM -0700, Jeff Davis wrote: On Thu, 2020-08-27 at 17:28 -0700, Peter Geoghegan wrote: We have a Postgres 13 open item for Disk-based hash aggregate, which is the only non-trivial open item. There is a general concern about how often we get disk-based hash aggregat

Background writer and checkpointer in crash recovery

2020-08-29 Thread Thomas Munro
(Forking this thread from the SLRU fsync one[1] to allow for a separate CF entry; it's unrelated, except for being another case of moving work off the recovery process's plate.) Hello hackers, Currently we don't run the bgwriter process during crash recovery. I've CCed Simon and Heikki who establ

Re: PROC_IN_ANALYZE stillborn 13 years ago

2020-08-29 Thread Tom Lane
Alvaro Herrera writes: > I pushed despite the objection because it seemed that downstream > discussion was largely favorable to the change, and there's a different > proposal to solve the bloat problem for analyze; and also: Note that this quasi-related patch has pretty thoroughly hijacked the CF

Re: Get rid of runtime handling of AlternativeSubPlan?

2020-08-29 Thread Tom Lane
I wrote: > Back in bd3daddaf232d95b0c9ba6f99b0170a0147dd8af, which introduced > AlternativeSubPlans, I wrote: > There is a lot more that could be done based on this infrastructure: in > particular it's interesting to consider switching to the hash plan if we > start > out using the non-hashe

Re: list of extended statistics on psql

2020-08-29 Thread Tomas Vondra
On Sat, Aug 29, 2020 at 06:43:47PM -0400, Alvaro Herrera wrote: On 2020-Aug-29, Tomas Vondra wrote: But if we want the output to show both what was requested and which types were actually built, that'll effectively double the number of columns needed :-( I was thinking it would be one column

Re: list of extended statistics on psql

2020-08-29 Thread Alvaro Herrera
On 2020-Aug-29, Tomas Vondra wrote: > But if we want the > output to show both what was requested and which types were actually > built, that'll effectively double the number of columns needed :-( I was thinking it would be one column per type showing either disabled or enabled or built. But an

Re: [EXTERNAL] Re: WIP: WAL prefetch (another approach)

2020-08-29 Thread Tomas Vondra
On Thu, Aug 27, 2020 at 04:28:54PM -0400, Stephen Frost wrote: Greetings, * Robert Haas (robertmh...@gmail.com) wrote: On Thu, Aug 27, 2020 at 2:51 PM Stephen Frost wrote: > > Hm? At least earlier versions didn't do prefetching for records with an fpw, and only for subsequent records affectin

Re: list of extended statistics on psql

2020-08-29 Thread Tomas Vondra
On Thu, Aug 27, 2020 at 11:26:17PM -0400, Alvaro Herrera wrote: On 2020-Aug-28, Tatsuro Yamada wrote: > IMO the per-type columns should show both the type being enabled as > well as it being built. Hmm. I'm not sure how to get the status (enabled or disabled) of extended stats. :( Could you ex

Re: list of extended statistics on psql

2020-08-29 Thread Tomas Vondra
On Thu, Aug 27, 2020 at 07:53:23PM -0400, Alvaro Herrera wrote: +1 for the general idea, and +1 for \dX being the syntax to use IMO the per-type columns should show both the type being enabled as well as it being built. (How many more stat types do we expect -- Tomas? I wonder if having one co

Re: new heapcheck contrib module

2020-08-29 Thread Mark Dilger
> On Aug 29, 2020, at 3:27 AM, Andrey M. Borodin wrote: > > > >> 29 авг. 2020 г., в 00:56, Robert Haas написал(а): >> >> On Fri, Aug 28, 2020 at 2:10 PM Andrey M. Borodin >> wrote: >>> I don't think so. ISTM It's the same problem of xmax>> just hidden behind detoasing. >>> Our regular he

Re: Ideas about a better API for postgres_fdw remote estimates

2020-08-29 Thread Andrey Lepikhov
On 8/29/20 9:22 PM, Stephen Frost wrote: I implemented some FDW + pg core machinery to reduce weight of the problem. The ANALYZE command on foreign table executes query on foreign server that extracts statistics tuple, serializes it into json-formatted string and returns to the caller. The c

Re: Ideas about a better API for postgres_fdw remote estimates

2020-08-29 Thread Tom Lane
Stephen Frost writes: > Isn't this going to create a version dependency that we'll need to deal > with..? What if a newer major version has some kind of improved ANALYZE > command, in terms of what it looks at or stores, and it's talking to an > older server? Yeah, this proposal is a nonstarter

Re: Clang UndefinedBehaviorSanitize (Postgres14) Detected undefined-behavior

2020-08-29 Thread Tom Lane
Peter Geoghegan writes: > I wonder if we should start using -fno-delete-null-pointer-checks: > https://lkml.org/lkml/2018/4/4/601 > This may not be strictly relevant to the discussion, but I was > reminded of it just now and thought I'd mention it. Hmm. gcc 8.3 defines this as: Assume that

Re: Ideas about a better API for postgres_fdw remote estimates

2020-08-29 Thread Stephen Frost
Greetings, * Andrey Lepikhov (a.lepik...@postgrespro.ru) wrote: > During the implementation of sharding related improvements i noticed that if > we use a lot of foreign partitions, we have bad plans because of vacuum > don't update statistics of foreign tables.This is done by the ANALYZE > command

Re: More aggressive vacuuming of temporary tables

2020-08-29 Thread Tom Lane
Michael Paquier writes: > I got to wonder lately if we should not have a static state like what > we do for MyXactFlags to track down if a utility query is a top-level > one or not as most of the time we just want to check the flag as > passed down by standard_ProcessUtility(). I have faced this

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-08-29 Thread Amit Kapila
On Fri, Aug 28, 2020 at 2:18 PM Dilip Kumar wrote: > > As discussed, I have added a another test case for covering the out of > order subtransaction rollback scenario. > +# large (streamed) transaction with out of order subtransaction ROLLBACKs +$node_publisher->safe_psql('postgres', q{ How abou

Re: new heapcheck contrib module

2020-08-29 Thread Andrey M. Borodin
> 29 авг. 2020 г., в 00:56, Robert Haas написал(а): > > On Fri, Aug 28, 2020 at 2:10 PM Andrey M. Borodin > wrote: >> I don't think so. ISTM It's the same problem of xmax> just hidden behind detoasing. >> Our regular heap_check was checking xmin\xmax invariants for tables, but >> failed to

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-08-29 Thread Dilip Kumar
On Sat, Aug 29, 2020 at 1:46 AM Robert Haas wrote: > > On Fri, Aug 28, 2020 at 1:29 PM Andres Freund wrote: > > It can break HOT chains, plain ctid chains etc, for example. Which, if > > earlier / follower tuples are removed can't be detected anymore at a > > later time. > > I think I need a more

Re: Allow ERROR from heap_prepare_freeze_tuple to be downgraded to WARNING

2020-08-29 Thread Dilip Kumar
On Fri, Aug 28, 2020 at 9:49 PM Robert Haas wrote: > > On Tue, Jul 21, 2020 at 9:21 AM Dilip Kumar wrote: > > In the previous version, the feature was enabled for cluster/vacuum > > full command as well. in the attached patch I have enabled it only > > if we are running vacuum command. It will