Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-07-24 Thread Andres Freund
Hi, On 2021-06-21 05:29:19 -0700, Andres Freund wrote: > On 2021-06-16 12:12:23 -0700, Andres Freund wrote: > > Could you share your testcase? I've been working on a series of patches > > to address this (I'll share in a bit), and I've run quite a few tests, > > and didn't hit any infinite loops.

Re: log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness

2021-07-24 Thread Fujii Masao
On 2021/07/25 7:50, Andres Freund wrote: Hi, I've been repeatedly confused by the the number of WAL files supposedly added. Even when 100s of new WAL files are created the relevant portion of log_checkpoints will only ever list zero or one added WAL file. The reason for that is that Checkpoi

Re: Removing "long int"-related limit on hash table sizes

2021-07-24 Thread Bruce Momjian
On Sat, Jul 24, 2021 at 06:25:53PM -0700, Andres Freund wrote: > > That's because they spill to disk where they did not before. The easy > > answer of "raise hash_mem_multiplier" doesn't help, because on Windows > > the product of work_mem and hash_mem_multiplier is clamped to 2GB, > > thanks to t

Re: Removing "long int"-related limit on hash table sizes

2021-07-24 Thread Andres Freund
Hi, On 2021-07-23 17:15:24 -0400, Tom Lane wrote: > Per the discussion at [1], users on Windows are seeing nasty performance > losses in v13/v14 (compared to prior releases) for hash aggregations that > required somewhat more than 2GB in the prior releases. Ugh :(. > That's because they spill t

Re: Avoiding data loss with synchronous replication

2021-07-24 Thread Andres Freund
Hi, On 2021-07-24 15:53:15 +0500, Andrey Borodin wrote: > Are there any other problems with blocking cancels? Unless you have commandline access to the server, it's not hard to get into a situation where you can't change the configuration setting because all connections are hanging, and you can't

Re: Avoiding data loss with synchronous replication

2021-07-24 Thread Andres Freund
Hi, On 2021-07-22 21:17:56 +, Bossart, Nathan wrote: > AFAICT there are a variety of ways that the aforementioned problem may > occur: > 1. Server restarts: As noted in the docs [2], "waiting transactions > will be marked fully committed once the primary database > recovers." I th

Re: something is wonky with pgbench pipelining

2021-07-24 Thread Andres Freund
Hi, Adding RMT. On 2021-07-21 16:55:08 -0700, Andres Freund wrote: > On 2021-07-20 14:57:15 -0400, Alvaro Herrera wrote: > > On 2021-Jul-20, Andres Freund wrote: > > > > > I think what's happening is that the first recvfrom() actually gets all 7 > > > connection results. The server doesn't have

Re: log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness

2021-07-24 Thread Andres Freund
Hi, On 2021-07-24 15:50:36 -0700, Andres Freund wrote: > As an example, here's the log output of a workload that has a replication slot > preventing WAL files from being recycled (and too small max_wal_size): > > 2021-07-24 15:47:42.524 PDT [2251649][checkpointer][:0] LOG: checkpoint > complete:

log_checkpoint's "WAL file(s) added" is misleading to the point of uselessness

2021-07-24 Thread Andres Freund
Hi, I've been repeatedly confused by the the number of WAL files supposedly added. Even when 100s of new WAL files are created the relevant portion of log_checkpoints will only ever list zero or one added WAL file. The reason for that is that CheckpointStats.ckpt_segs_added is only incremented in

Re: DROP INDEX CONCURRENTLY on partitioned index

2021-07-24 Thread Justin Pryzby
On Wed, Jul 14, 2021 at 09:18:12PM +0530, vignesh C wrote: > The patch does not apply on Head anymore, could you rebase and post a > patch. I'm changing the status to "Waiting for Author". I'm withdrawing this patch at least until the corresponding patch for CIC is progressed. https://commitfest.

Re: CREATE SEQUENCE with RESTART option

2021-07-24 Thread Bharath Rupireddy
On Sat, Jul 24, 2021 at 3:20 AM Cary Huang wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: tested, passed > Spec compliant: tested, passed > Documentation:tested, passed >

Re: Configure with thread sanitizer fails the thread test

2021-07-24 Thread Tom Lane
I wrote: > After a bit more thought, HEAD+v14 is also my preference. I'm not > eager to change this in stable branches, but it doesn't seem too > late for v14. Perusing the commit log, I realized that there's another reason why v14 is a good cutoff: thread_test.c was in a different place with an

Re: Use WaitLatch for {pre, post}_auth_delay instead of pg_usleep

2021-07-24 Thread Bharath Rupireddy
On Fri, Jul 23, 2021 at 4:40 AM Bossart, Nathan wrote: > I would suggest changing "attach from a debugger" to "attaching with a > debugger." Thanks. IMO, the following looks better: Waiting on connection startup before authentication to allow attaching a debugger to the process. Wai

Re: Maintain the pathkesy for subquery from outer side information

2021-07-24 Thread Tom Lane
Andy Fan writes: > On Sat, Jul 24, 2021 at 10:14 PM Tom Lane wrote: >> Why do you "expect" that? I think pushing the outer ORDER BY past a >> LIMIT is an unacceptable semantics change. > I don't mean push down a "ORDER BY" clause to subquery, I mean push > down an "interesting order" to subque

Re: Maintain the pathkesy for subquery from outer side information

2021-07-24 Thread Andy Fan
On Sat, Jul 24, 2021 at 10:14 PM Tom Lane wrote: > > Andy Fan writes: > > When I am working on the UnqiueKey stuff, I find the following cases. > > SELECT * FROM (SELECT * FROM t offset 0) v ORDER BY a; > > // root->query_keys = A. root->order_pathkeys = A > > // Current: subroot->query_pathkeys

Re: Maintain the pathkesy for subquery from outer side information

2021-07-24 Thread Tom Lane
Andy Fan writes: > When I am working on the UnqiueKey stuff, I find the following cases. > SELECT * FROM (SELECT * FROM t offset 0) v ORDER BY a; > // root->query_keys = A. root->order_pathkeys = A > // Current: subroot->query_pathkeys = NIL. > // Expected: subroot->_pathkeys = [A]. Why do

Re: visibility map corruption

2021-07-24 Thread Bruce Momjian
On Fri, Jul 23, 2021 at 09:01:18PM -0400, Bruce Momjian wrote: > On Fri, Jul 23, 2021 at 05:47:18PM -0700, Peter Geoghegan wrote: > > > I could perhaps see corruption happening if pg_control's oldest xid > > > value was closer to the current xid value than it should be, but I can't > > > see how ha

Maintain the pathkesy for subquery from outer side information

2021-07-24 Thread Andy Fan
When I am working on the UnqiueKey stuff, I find the following cases. SELECT * FROM (SELECT * FROM t offset 0) v ORDER BY a; // root->query_keys = A. root->order_pathkeys = A // Current: subroot->query_pathkeys = NIL. // Expected: subroot->_pathkeys = [A]. SELECT * FROM (SELECT * FROM t off

Re: Avoiding data loss with synchronous replication

2021-07-24 Thread Andrey Borodin
> 23 июля 2021 г., в 22:54, Bossart, Nathan написал(а): > > On 7/23/21, 4:33 AM, "Andrey Borodin" wrote: >> Thanks for you interest in the topic. I think in the thread [0] we almost >> agreed on general design. >> The only left question is that we want to threat pg_ctl stop and kill >> SIGT

Re: Avoiding data loss with synchronous replication

2021-07-24 Thread Andrey Borodin
23 июля 2021 г., в 22:54, Bossart, Nathan написал(а):On 7/23/21, 4:33 AM, "Andrey Borodin" wrote:Thanks for you interest in the topic. I think in the thread [0] we almost agreed on general design.The only left question is that we want to threat pg_ctl stop and kill SIGTERM differently to pg_termi

Re: Incorrect usage of strtol, atoi for non-numeric junk inputs

2021-07-24 Thread Michael Paquier
On Thu, Jul 22, 2021 at 02:32:35PM +0900, Michael Paquier wrote: > Okay, done those parts as per the attached. While on it, I noticed an > extra one for pg_dump --rows-per-insert. I am counting 25 translated > strings cut in total. > > Any objections to this first step? I have looked at that ov

Re: Logical replication error "no record found" /* shouldn't happen */

2021-07-24 Thread Michail Nikolaev
Hello. I saw this error multiple times trying to replicate the 2-3 TB server (version 11 to version 12). I was unable to find any explanation for this error. Thanks, Michail.

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-07-24 Thread Amit Kapila
On Fri, Jul 23, 2021 at 6:55 PM Robert Haas wrote: > > On Wed, Jul 21, 2021 at 11:55 PM Amit Kapila wrote: > > I see here we have a mix of opinions from various people. Dilip seems > > to be favoring the approach where we provide some option to the user > > for partitioned tables and automatic be