Re: support for SSE2 intrinsics

2022-08-03 Thread John Naylor
On Thu, Aug 4, 2022 at 12:38 PM Masahiko Sawada wrote: > > I also think it's a good start. There is a typo in the commit message: > > s/hepler/helper/ > > The rest looks good to me. Fixed, and pushed, thanks to you both! I'll polish a small patch I have that actually uses this. -- John Naylor ED

RE: Perform streaming logical transactions by background workers and parallel apply

2022-08-03 Thread wangw.f...@fujitsu.com
On Mon, Jul 25, 2022 at 21:50 PM Amit Kapila wrote: > Few comments on 0001: > == Thanks for your comments. > 1. > - substream bool > + substream char > > > - If true, the subscription will allow streaming of in-progress > - transactions

RE: Perform streaming logical transactions by background workers and parallel apply

2022-08-03 Thread wangw.f...@fujitsu.com
On Wed, Jul 27, 2022 at 16:03 PM Peter Smith wrote: > Here are some review comments for the patch v19-0004: Thanks for your kindly review and comments. To avoid making this thread too long, I will reply to all of your comments (0001-patch ~ 0004-patch) in this email. In addition, in order not to

RE: Perform streaming logical transactions by background workers and parallel apply

2022-08-03 Thread wangw.f...@fujitsu.com
On Thurs, Jul 28, 2022 at 13:20 PM Kuroda, Hayato/黒田 隼人 wrote: > > Dear Wang-san, > > Hi, I'm also interested in the patch and I started to review this. > Followings are comments about 0001. Thanks for your kindly review and comments. To avoid making this thread too long, I will reply to all of

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-08-03 Thread Bharath Rupireddy
On Sun, Jul 31, 2022 at 8:36 PM mahendrakar s wrote: > >> On Mon, 25 Jul 2022 at 16:42, Bharath Rupireddy >> wrote: >> Here's the v3 patch after rebasing. >> >> I just would like to reiterate the issue the patch is trying to solve: >> At times (when no space is left on the device or when the pr

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-03 Thread Noah Misch
On Tue, Aug 02, 2022 at 07:37:27AM -0700, Noah Misch wrote: > On Tue, Aug 02, 2022 at 10:14:22AM -0400, David Steele wrote: > > On 7/31/22 02:17, Noah Misch wrote: > > >On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: > > >>On 6/19/21 16:39, Noah Misch wrote: > > >>>On Tue, Feb 02, 202

Re: Cygwin cleanup

2022-08-03 Thread Thomas Munro
On Thu, Aug 4, 2022 at 5:23 PM Tom Lane wrote: > Thomas Munro writes: > > It may be madness to try to work around this, but I wonder if we could > > use a static local variable that we update with atomic compare > > exhange, inside PG_SIGNAL_HANDLER_ENTRY(), and > > PG_SIGNAL_HANDLER_EXIT() macro

Re: Correct comment in RemoveNonParentXlogFiles()

2022-08-03 Thread Kyotaro Horiguchi
At Thu, 04 Aug 2022 15:00:06 +0900 (JST), Kyotaro Horiguchi wrote in > At Wed, 3 Aug 2022 18:16:33 +0530, Ashutosh Sharma > wrote in > > Following comment in RemoveNonParentXlogFiles() says that we are trying to > > remove any WAL file whose segment number is >= the segment number of the > >

Re: Correct comment in RemoveNonParentXlogFiles()

2022-08-03 Thread Kyotaro Horiguchi
At Wed, 3 Aug 2022 18:16:33 +0530, Ashutosh Sharma wrote in > Following comment in RemoveNonParentXlogFiles() says that we are trying to > remove any WAL file whose segment number is >= the segment number of the > first WAL file on the new timeline. However, looking at the code, I can say > that

Re: [PATCH] Introduce array_shuffle() and array_sample()

2022-08-03 Thread Martin Kalcher
Patch update without merge conflicts. MartinFrom 0ecffcf3ed2eb59d045941b69bb86a34b93f3391 Mon Sep 17 00:00:00 2001 From: Martin Kalcher Date: Sun, 17 Jul 2022 18:06:04 +0200 Subject: [PATCH v3] Introduce array_shuffle() and array_sample() * array_shuffle() shuffles the elements of an array. * a

Re: doc: mentioned CREATE+ATTACH PARTITION as an alternative to CREATE TABLE..PARTITION OF

2022-08-03 Thread Robert Treat
On Mon, Jul 18, 2022 at 10:39 AM Andrew Dunstan wrote: > On 2022-07-18 Mo 10:33, Justin Pryzby wrote: > > It's easy to use CREATE TABLE..LIKE + ALTER..ATTACH PARTITION to avoid > > acquiring a strong lock when creating a new partition. > > But it's also easy to forget. > > > > commit 76c0d1198cf29

Re: support for SSE2 intrinsics

2022-08-03 Thread Masahiko Sawada
Hi, On Wed, Aug 3, 2022 at 2:01 PM John Naylor wrote: > > > On Tue, Aug 2, 2022 at 11:53 PM Nathan Bossart > wrote: > > I did a bit of cross-checking, and AFAICT this is a reasonable starting > > point. emmintrin.h appears to be sufficient for one of my patches that > > makes use of SSE2 instr

Re: Cygwin cleanup

2022-08-03 Thread Tom Lane
Thomas Munro writes: > It may be madness to try to work around this, but I wonder if we could > use a static local variable that we update with atomic compare > exhange, inside PG_SIGNAL_HANDLER_ENTRY(), and > PG_SIGNAL_HANDLER_EXIT() macros that do nothing on every other system. > On entry, if yo

Re: Cygwin cleanup

2022-08-03 Thread Thomas Munro
On Thu, Aug 4, 2022 at 4:16 PM Thomas Munro wrote: > On Thu, Aug 4, 2022 at 3:38 PM Justin Pryzby wrote: > > [train wreck] > > Oh my, so I'm getting the impression we might actually be totally > unstable on Cygwin. Which surprises me because ... wait a minute ... > lorikeet isn't even running mo

Re: Cleaning up historical portability baggage

2022-08-03 Thread Thomas Munro
On Thu, Aug 4, 2022 at 3:43 PM Andres Freund wrote: > > We retain a HAVE_SHM_OPEN macro, because it's clearer to readers than > > something like !defined(WIN32). > > I don't like these. I don't find them clearer - if we really just assume this > to be the case on windows, it's easier to understand

Re: Generalize ereport_startup_progress infrastructure

2022-08-03 Thread Bharath Rupireddy
On Wed, Aug 3, 2022 at 12:11 AM Robert Haas wrote: > > On Tue, Aug 2, 2022 at 3:25 AM Bharath Rupireddy > wrote: > > ereport_startup_progress infrastructure added by commit 9ce346e [1] > > will be super-useful for reporting progress of any long-running server > > operations, not just the startup

Re: Cygwin cleanup

2022-08-03 Thread Andres Freund
Hi, On 2022-08-04 16:16:06 +1200, Thomas Munro wrote: > Ok, that's slightly reassuring, so maybe we *can* fix this, but I'm > one step closer to what Tom said, re wasting developer time... It might be worth checking whether the cygwin installer, which at some point at least allowed installing pos

Re: Cleaning up historical portability baggage

2022-08-03 Thread Thomas Munro
On Thu, Aug 4, 2022 at 4:09 PM Tom Lane wrote: > Andres Freund writes: > >> XXX This can only be committed once prairedog is decommissioned, because > >> macOS 10.4 didn't have clock_gettime(). > > > Maybe put it later in the queue? > > clock_gettime is required by SUSv2 (1997), so I have to admi

Re: Cygwin cleanup

2022-08-03 Thread Thomas Munro
On Thu, Aug 4, 2022 at 3:38 PM Justin Pryzby wrote: > [train wreck] Oh my, so I'm getting the impression we might actually be totally unstable on Cygwin. Which surprises me because ... wait a minute ... lorikeet isn't even running most of the tests. So... we don't really know the degree to whic

Re: Refactoring postgres_fdw/connection.c

2022-08-03 Thread Amul Sul
On Thu, Jul 28, 2022 at 11:56 AM Fujii Masao wrote: > > > > On 2022/07/27 10:36, Kyotaro Horiguchi wrote: > > At Tue, 26 Jul 2022 18:33:04 +0900, Fujii Masao > > wrote in > >>> I'm not sure the two are similar with each other. The new function > >>> pgfdw_exec_pre_commit() looks like a merger o

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Dilip Kumar
On Thu, Aug 4, 2022 at 12:18 AM Justin Pryzby wrote: > > On Wed, Aug 03, 2022 at 11:26:43AM -0700, Andres Freund wrote: > > Hm. This looks more like an issue of DROP DATABASE not being interruptible. > > I > > suspect this isn't actually related to STRATEGY wal_log and could likely be > > reprodu

Re: Cleaning up historical portability baggage

2022-08-03 Thread Tom Lane
Andres Freund writes: >> XXX This can only be committed once prairedog is decommissioned, because >> macOS 10.4 didn't have clock_gettime(). > Maybe put it later in the queue? clock_gettime is required by SUSv2 (1997), so I have to admit that macOS 10.4 doesn't have a lot of excuse not to have i

Re: Cygwin cleanup

2022-08-03 Thread Andres Freund
Hi, On 2022-07-28 17:23:19 -0500, Justin Pryzby wrote: > On Fri, Jul 29, 2022 at 10:04:04AM +1200, Thomas Munro wrote: > > > > XXX This should use a canned Docker image with all the right packages > > > > installed > > > > > > Has anyone tried using non-canned images ? It sounds like this could

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-08-03 Thread Amit Kapila
On Mon, Aug 1, 2022 at 9:52 PM Önder Kalacı wrote: > > Hi, > > As far as I can see, the following is the answer to the only remaining open > discussion in this thread. Let me know if anything is missed. > >> (b) it appears to me that the patch decides >> >> which index to use the first time it op

Re: Eliminating SPI from RI triggers - take 2

2022-08-03 Thread Amit Langote
On Wed, Jul 13, 2022 at 8:59 PM Amit Langote wrote: > On Sat, Jul 9, 2022 at 1:15 AM Robert Haas wrote: > Thanks for taking a look at this. I'll try to respond to other points > in a separate email, but I wanted to clarify something about below: > > > I find my ego slightly wounded by the commen

Re: Cleaning up historical portability baggage

2022-08-03 Thread Andres Freund
Hi, Can we get a few more of these committed soon? It's all tests that I need to sync with the meson stuff, and I'd rather get it over with :). And it reduces the set of tests that need to be compared... Or is there a blocker (leaving the prairedog one aside)? On 2022-08-03 14:25:01 +1200, Thom

Re: Cygwin cleanup

2022-08-03 Thread Justin Pryzby
On Fri, Jul 29, 2022 at 10:04:04AM +1200, Thomas Munro wrote: > > > XXX We would never want this to run by default in CI, but it'd be nice > > > to be able to ask for it with ci-os-only! (See commented out line) > > > only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' > > > > Does

Re: PostgreSQL 15 minor fixes in protocol.sgml

2022-08-03 Thread Amit Kapila
On Wed, Aug 3, 2022 at 4:23 PM Alvaro Herrera wrote: > > On 2022-Aug-03, Amit Kapila wrote: > > > Thanks for the report and Thanks Michael for including me. I am just > > redirecting it to -hackers so that others involved in this feature > > also can share their views. > > I'm sorry, but our polic

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-08-03 Thread Thomas Munro
On Sun, Jul 31, 2022 at 11:17 AM Tom Lane wrote: > Thomas Munro writes: > > I noticed this is a 32 bit FBSD system. Is it running on UFS, perhaps > > on slow storage? Are soft updates enabled (visible as options in > > output of "mount")? > > It's an ancient (2006) mac mini with 5400RPM spinnin

Re: [PATCH] postgresql.conf.sample comment alignment.

2022-08-03 Thread Peter Smith
On Thu, Aug 4, 2022 at 11:09 AM Michael Paquier wrote: > > On Wed, Aug 03, 2022 at 12:58:04PM +0200, Alvaro Herrera wrote: > > On 2022-Aug-01, Tom Lane wrote: > >> One idea for avoiding confusion is to legislate that we won't > >> use tabs at all in this file (which we could enforce via > >> .gita

Re: Cleaning up historical portability baggage

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 21:52:04 -0400, Tom Lane wrote: > Andres Freund writes: > > Another potential cleanup is the fallback for strtoll/strtoull. > > +1, I suspect the alternate spellings are dead. Looks like that includes systems where there's no declaration for strtoll, strtoull. The test was intr

Re: [PATCH] postgresql.conf.sample comment alignment.

2022-08-03 Thread Julien Rouhaud
On Thu, Aug 04, 2022 at 10:09:27AM +0900, Michael Paquier wrote: > On Wed, Aug 03, 2022 at 12:58:04PM +0200, Alvaro Herrera wrote: > > On 2022-Aug-01, Tom Lane wrote: > >> One idea for avoiding confusion is to legislate that we won't > >> use tabs at all in this file (which we could enforce via > >

Re: Mingw task for Cirrus CI

2022-08-03 Thread Thomas Munro
I noticed that this says: [01:01:45.657] sqlda.pgc: In function 'dump_sqlda': [01:01:45.657] sqlda.pgc:45:33: warning: format '%d' expects argument of type 'int', but argument 3 has type 'long long int' [-Wformat=] [01:01:45.657] 45 | "name sqlda descriptor: '%s' value %I64d\n", [01:01:45.657] | ^

Re: Cleaning up historical portability baggage

2022-08-03 Thread Tom Lane
Andres Freund writes: > Another potential cleanup is the fallback for strtoll/strtoull. +1, I suspect the alternate spellings are dead. regards, tom lane

Re: Introduce wait_for_subscription_sync for TAP tests

2022-08-03 Thread Amit Kapila
On Wed, Aug 3, 2022 at 10:21 AM Amit Kapila wrote: > > Pushed this one and now I'll look at your other patch. > I have pushed the second patch as well after making minor changes in the comments. Alvaro [1] and Tom [2] suggest to back-patch this and they sound reasonable to me. Will you be able to

Re: Cleaning up historical portability baggage

2022-08-03 Thread Andres Freund
Hi, Another potential cleanup is the fallback for strtoll/strtoull. Some of the spellings were introduced because of "ancient HPUX": commit 06f66cff9e0b93db81db1595156b2aff8ba1786e Author: Tom Lane Date: 2018-05-19 14:22:18 -0400 Support platforms where strtoll/strtoull are spelled __strt

Re: A test for replay of regression tests

2022-08-03 Thread Justin Pryzby
On Thu, Aug 04, 2022 at 09:24:24AM +1200, Thomas Munro wrote: > On Thu, Aug 4, 2022 at 3:30 AM Justin Pryzby wrote: > > On Thu, Dec 09, 2021 at 12:10:23PM +1300, Thomas Munro wrote: > > > This adds 2 whole minutes to the recovery check, when running with the > > > Windows serial-only scripts on Ci

Re: [PATCH] postgresql.conf.sample comment alignment.

2022-08-03 Thread Michael Paquier
On Wed, Aug 03, 2022 at 12:58:04PM +0200, Alvaro Herrera wrote: > On 2022-Aug-01, Tom Lane wrote: >> One idea for avoiding confusion is to legislate that we won't >> use tabs at all in this file (which we could enforce via >> .gitattributes, I think). > > +1. That's not the first time this 4- or

Re: enable/disable broken for statement triggers on partitioned tables

2022-08-03 Thread Amit Langote
On Thu, Aug 4, 2022 at 3:01 AM Alvaro Herrera wrote: > On 2022-Aug-02, Amit Langote wrote: > > Regarding the patch, I agree that storing the recurse flag rather than > > overwriting subtype might be better. > > > > + boolexecTimeRecursion; /* set by ATPrepCmd if ATExecCmd must > > +

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Tom Lane
"Jonathan S. Katz" writes: > I did rule out wanting to do the "xid + $X" check after reviewing some > of the output. I think that both $X could end up varying, and it really > feels like a bandaid. It is that. I wouldn't feel comfortable with $X less than 100 or so, which is probably sloppy en

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Jonathan S. Katz
On 8/3/22 2:08 PM, Peter Geoghegan wrote: On Wed, Aug 3, 2022 at 1:47 PM Tom Lane wrote: Again, this seems to me to be breaking the test's real-world applicability for a (false?) sense of stability. I agree. A lot of the VACUUM test flappiness issues we've had to deal with in the past now se

Re: [doc] fix a potential grammer mistake

2022-08-03 Thread Junwang Zhao
On Thu, Aug 4, 2022 at 12:42 AM Robert Treat wrote: > > On Wed, Aug 3, 2022 at 11:15 AM Junwang Zhao wrote: > > > > Attachment is a corrected version based on Tom's suggestion. > > > > Thanks. > > > > On Wed, Aug 3, 2022 at 9:56 PM Tom Lane wrote: > > > > > > Erikjan Rijkers writes: > > > > I d

Re: postgres_fdw: batch inserts vs. before row triggers

2022-08-03 Thread Matthias van de Meent
On Wed, 3 Aug 2022 at 23:57, Tom Lane wrote: > > Matthias van de Meent writes: > > I don't have a current version of the SQL spec, but one preliminary > > version of SQL:2012 I retrieved via the wiki details that all BEFORE > > triggers on INSERT/UPDATE/DELETE statements are all executed before >

Re: Unstable tests for recovery conflict handling

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 16:33:46 -0400, Robert Haas wrote: > On Wed, Aug 3, 2022 at 1:57 PM Andres Freund wrote: > > The reason nothing might get logged in some cases is that > > e.g. ResolveRecoveryConflictWithLock() tells > > ResolveRecoveryConflictWithVirtualXIDs() to *not* report the waiting: > >

Re: postgres_fdw: batch inserts vs. before row triggers

2022-08-03 Thread Tom Lane
Matthias van de Meent writes: > I don't have a current version of the SQL spec, but one preliminary > version of SQL:2012 I retrieved via the wiki details that all BEFORE > triggers on INSERT/UPDATE/DELETE statements are all executed before > _any_ of that statements' affected data is modified. >

Re: postgres_fdw: batch inserts vs. before row triggers

2022-08-03 Thread Matthias van de Meent
On Tue, 19 Apr 2022 at 14:00, Tomas Vondra wrote: > > On 4/19/22 11:16, Etsuro Fujita wrote: > > On Sun, Apr 17, 2022 at 6:20 PM Etsuro Fujita > > wrote: > >> Here > >> is an example using HEAD: > >> > >> create extension postgres_fdw; > >> create server loopback foreign data wrapper postgres_fd

Re: Proposal: Support custom authentication methods using hooks

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 17:21:58 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2022-08-03 16:28:08 -0400, Stephen Frost wrote: > > > Again, server-side only is not interesting and not a direction that > > > makes sense to go in because it doesn't provide any way to hav

Re: Checking pgwin32_is_junction() errors

2022-08-03 Thread Thomas Munro
On Thu, Aug 4, 2022 at 9:28 AM Andrew Dunstan wrote: > On 2022-08-01 Mo 16:06, Andrew Dunstan wrote: > > I'll try it out on fairywren/drongo. > They are happy with patches 2, 3, and 4. Thanks for testing! If there are no objections, I'll go ahead and commit these later today.

Re: Checking pgwin32_is_junction() errors

2022-08-03 Thread Andrew Dunstan
On 2022-08-01 Mo 16:06, Andrew Dunstan wrote: > On 2022-08-01 Mo 01:09, Thomas Munro wrote: >> On Thu, Jul 28, 2022 at 9:31 PM Thomas Munro wrote: >>> There's one curious change in the draft patch attached: you can't >>> unlink() a junction point, you have to rmdir() it. Previously, things >>>

Re: A test for replay of regression tests

2022-08-03 Thread Thomas Munro
On Thu, Aug 4, 2022 at 3:30 AM Justin Pryzby wrote: > On Thu, Dec 09, 2021 at 12:10:23PM +1300, Thomas Munro wrote: > > This adds 2 whole minutes to the recovery check, when running with the > > Windows serial-only scripts on Cirrus CI (using Andres's CI patches). > > For Linux it adds ~20 seconds

Re: Proposal: Support custom authentication methods using hooks

2022-08-03 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2022-08-03 16:28:08 -0400, Stephen Frost wrote: > > Again, server-side only is not interesting and not a direction that > > makes sense to go in because it doesn't provide any way to have > > trust established in both directions, which is

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Peter Geoghegan
On Wed, Aug 3, 2022 at 1:47 PM Tom Lane wrote: > Again, this seems to me to be breaking the test's real-world applicability > for a (false?) sense of stability. I agree. A lot of the VACUUM test flappiness issues we've had to deal with in the past now seem like problems with VACUUM itself, the t

Re: Clarifying Commitfest policies

2022-08-03 Thread Matthias van de Meent
On Wed, 3 Aug 2022 at 20:04, Jacob Champion wrote: > > [was: CF app: add "Returned: Needs more interest"] > > On Wed, Aug 3, 2022 at 10:09 AM Julien Rouhaud wrote: > > I'm afraid that > > patches will still be left alone to rot and there still be no clear rules on > > what to do and when, reminde

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Andres Freund
On 2022-08-03 16:46:57 -0400, Tom Lane wrote: > Robert Haas writes: > > Or we could disable autovacuum on the new cluster, which I think is a > > better solution. I like it when things match exactly; it makes me feel > > that the universe is well-ordered. > > Again, this seems to me to be breakin

Re: Proposal: Support custom authentication methods using hooks

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 16:28:08 -0400, Stephen Frost wrote: > Again, server-side only is not interesting and not a direction that > makes sense to go in because it doesn't provide any way to have > trust established in both directions, which is what all modern > authentication methods do (certificates,

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Tom Lane
Robert Haas writes: > Or we could disable autovacuum on the new cluster, which I think is a > better solution. I like it when things match exactly; it makes me feel > that the universe is well-ordered. Again, this seems to me to be breaking the test's real-world applicability for a (false?) sense

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Peter Geoghegan
On Wed, Aug 3, 2022 at 1:34 PM Tom Lane wrote: > That doesn't seem like it'd be all that thorough: we expect VACUUM > to skip pages whenever possible. I'm also a bit concerned about > the expense, though admittedly this test is ridiculously expensive > already. I bet the SKIP_PAGES_THRESHOLD stu

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Robert Haas
On Wed, Aug 3, 2022 at 4:20 PM Andres Freund wrote: > > I don't really like this approach. Imagine that the code got broken in > > such a way that relfrozenxid and relminmxid were set to a value chosen > > at random - say, the contents of 4 bytes of unallocated memory that > > contained random gar

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Tom Lane
Peter Geoghegan writes: > It couldn't hurt to do that as well, in passing (at the same time as > testing that newrelfrozenxid >= oldrelfrozenxid directly). But > deliberately running VACUUM afterwards seems like a good idea. We > really ought to expect VACUUM to catch cases where > relfrozenxid/re

Re: Unstable tests for recovery conflict handling

2022-08-03 Thread Robert Haas
On Wed, Aug 3, 2022 at 1:57 PM Andres Freund wrote: > The reason nothing might get logged in some cases is that > e.g. ResolveRecoveryConflictWithLock() tells > ResolveRecoveryConflictWithVirtualXIDs() to *not* report the waiting: > /* > * Prevent ResolveRecoveryCo

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Peter Geoghegan
On Wed, Aug 3, 2022 at 1:20 PM Andres Freund wrote: > > I don't really like this approach. Imagine that the code got broken in > > such a way that relfrozenxid and relminmxid were set to a value chosen > > at random - say, the contents of 4 bytes of unallocated memory that > > contained random gar

Re: Proposal: Support custom authentication methods using hooks

2022-08-03 Thread Stephen Frost
Greetings, * samay sharma (smilingsa...@gmail.com) wrote: > On Tue, Aug 2, 2022 at 2:48 PM Jacob Champion > wrote: > > [dev hat] That said, I plan to do some additional dev work on top of > > this over the next couple of months. The ideal case would be to provide > > a server-only extension that

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-03 Thread Nathan Bossart
On Wed, Aug 03, 2022 at 11:06:58AM -0700, Andres Freund wrote: > On 2022-08-02 16:43:57 -0700, Nathan Bossart wrote: >> >> +#ifdef USE_SSE2 >> >> +pg_attribute_no_sanitize_alignment() >> >> +#endif >> > >> > What's the deal with this annotation? Needs a comment. >> >> Will do. c.h suggests that

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 09:59:40 -0400, Robert Haas wrote: > On Tue, Aug 2, 2022 at 3:51 PM Tom Lane wrote: > > > The test does look helpful and it would catch regressions. Loosely > > > quoting Robert on a different point upthread, we don't want to turn off > > > the alarm just because it's spuriously

Re: Smoothing the subtrans performance catastrophe

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 15:36:40 -0400, Robert Haas wrote: > On Wed, Aug 3, 2022 at 3:18 PM Andres Freund wrote: > > In contrast to e.g. clog or multixact we don't need to access a lot of old > > While we can't put a useful hard cap on the number of potential subtrans > > entries (we can only throw sub

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Tom Lane
Andres Freund writes: > I agree very much with that - just am doubtful that "lacks interest" is a good > way of dealing with it, unless we just want to treat it as a nicer sounding > "rejected". I think there is a difference. "Lacks interest" suggests that there is a path forward for the patch,

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 12:06:03 -0700, Jacob Champion wrote: > On 8/3/22 11:41, Andres Freund wrote: > > What patches are we concretely talking about?> > > My impression is that a lot of the patches floating from CF to CF have > > gotten > > sceptical feedback and at best a minor amount of work to add

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Peter Geoghegan
On Wed, Aug 3, 2022 at 6:59 AM Robert Haas wrote: > I don't really like this approach. Imagine that the code got broken in > such a way that relfrozenxid and relminmxid were set to a value chosen > at random - say, the contents of 4 bytes of unallocated memory that > contained random garbage. Well

Re: Smoothing the subtrans performance catastrophe

2022-08-03 Thread Robert Haas
On Wed, Aug 3, 2022 at 3:18 PM Andres Freund wrote: > In contrast to e.g. clog or multixact we don't need to access a lot of old > While we can't put a useful hard cap on the number of potential subtrans > entries (we can only throw subxid->parent mappings away once no existing > snapshot might ne

Re: Smoothing the subtrans performance catastrophe

2022-08-03 Thread Andres Freund
Hi, On 2022-08-01 17:42:49 +0100, Simon Riggs wrote: > The reason for the slowdown is clear: when we overflow we check every > xid against subtrans, producing a large stream of lookups. Some > previous hackers have tried to speed up subtrans - this patch takes a > different approach: remove as man

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Jacob Champion
On 8/3/22 11:41, Andres Freund wrote: > What patches are we concretely talking about?> > My impression is that a lot of the patches floating from CF to CF have gotten > sceptical feedback and at best a minor amount of work to address that has been > done. - https://commitfest.postgresql.org/38/248

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Tom Lane
Andres Freund writes: > My impression is that a lot of the patches floating from CF to CF have gotten > sceptical feedback and at best a minor amount of work to address that has been > done. That I think is a distinct issue: nobody wants to take on the unpleasant job of saying "no, we don't want

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Peter Geoghegan
On Tue, Aug 2, 2022 at 12:32 PM Tom Lane wrote: > Hmmm ... now that you mention it, I see nothing in 002_pg_upgrade.pl > that attempts to turn off autovacuum on either the source server or > the destination. So one plausible theory is that autovac moved the > numbers since we checked. It's very

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Justin Pryzby
On Wed, Aug 03, 2022 at 11:26:43AM -0700, Andres Freund wrote: > Hm. This looks more like an issue of DROP DATABASE not being interruptible. I > suspect this isn't actually related to STRATEGY wal_log and could likely be > reproduced in older versions too. I couldn't reproduce it with file_copy, b

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 10:52:36 -0700, Jacob Champion wrote: > The situation I'm looking at, though, is where we have a dozen patches > floating forward that multiple CFMs in a row feel should be returned, > but they won't because claiming "they have feedback" is clearly unfair > to the author. I think

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Andres Freund
Hi, On 2022-08-03 12:01:18 -0500, Justin Pryzby wrote: > Now, I've reproduced the problem under valgrind, but it doesn't show anything > useful Yea, that looks like an issue on a different level. > > pryzbyj@pryzbyj:~$ while :; do psql -h /tmp template1 -c "DROP DATABASE a" -c > "CREATE DATABA

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-03 Thread Andres Freund
Hi, On 2022-08-02 16:43:57 -0700, Nathan Bossart wrote: > >> +/* > >> + * pg_linearsearch_uint32 > >> + * > >> + * Returns the address of the first element in 'base' that equals 'key', > >> or > >> + * NULL if no match is found. > >> + */ > >> +#ifdef USE_SSE2 > >> +pg_attribute_no_sanitize_align

Clarifying Commitfest policies

2022-08-03 Thread Jacob Champion
[was: CF app: add "Returned: Needs more interest"] On Wed, Aug 3, 2022 at 10:09 AM Julien Rouhaud wrote: > I'm afraid that > patches will still be left alone to rot and there still be no clear rules on > what to do and when, reminder for CFM and such, and that this new status would > never be use

Re: enable/disable broken for statement triggers on partitioned tables

2022-08-03 Thread Alvaro Herrera
On 2022-Aug-02, Amit Langote wrote: > Regarding the patch, I agree that storing the recurse flag rather than > overwriting subtype might be better. > > + boolexecTimeRecursion; /* set by ATPrepCmd if ATExecCmd must > + * recurse to children */ > > Migh

Re: Unstable tests for recovery conflict handling

2022-08-03 Thread Andres Freund
Hi, On 2022-07-26 13:03:54 -0700, Andres Freund wrote: > On 2022-07-26 14:30:30 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2022-07-26 13:57:53 -0400, Tom Lane wrote: > > >> So this is not a case of RecoveryConflictInterrupt doing the wrong thing: > > >> the startup process hasn't d

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Jacob Champion
On Wed, Aug 3, 2022 at 10:09 AM Julien Rouhaud wrote: > First of all, I didn't want to imply that rejecting a patch should be > pleasant, > sorry if that sounded that way. No worries, I don't think it really sounded that way. :D > It's not that I'm opposed to adding that status, I just don't se

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-03 Thread Nathan Bossart
Here is a new patch set. 0001 is the currently-proposed patch from the other thread [0] for determining SSE2 support. 0002 introduces the optimized linear search function. And 0003 makes use of the new function for the [sub]xip lookups in XidInMVCCSnapshot(). [0] https://postgr.es/m/CAFBsxsGkt

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Julien Rouhaud
Hi, On Wed, Aug 03, 2022 at 08:58:49AM -0700, Jacob Champion wrote: > On Tue, Aug 2, 2022 at 8:00 PM Julien Rouhaud wrote: > > I'm personally fine with the current statutes, as closing a patch with RwF > > explaining that there was no interest is still a feedback, > > Making that explanation each

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Justin Pryzby
On Wed, Aug 03, 2022 at 11:02:00AM -0500, Justin Pryzby wrote: > But I reproduced the first problem with a handful of tries interrupting the > while loop: > > 2022-08-03 10:39:50.129 CDT client backend[5530] [unknown] PANIC: could not > open critical system index 2662 ... > So far, I haven't suc

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Dilip Kumar
On Wed, Aug 3, 2022 at 9:32 PM Justin Pryzby wrote: > > On Wed, Aug 03, 2022 at 04:45:23PM +0530, Dilip Kumar wrote: > > Another version of the patch which closes the smgr at the end using > > smgrcloserellocator() and I have also added a commit message. > > Thanks for providing a patch. > This se

Re: [doc] fix a potential grammer mistake

2022-08-03 Thread Robert Treat
On Wed, Aug 3, 2022 at 11:15 AM Junwang Zhao wrote: > > Attachment is a corrected version based on Tom's suggestion. > > Thanks. > > On Wed, Aug 3, 2022 at 9:56 PM Tom Lane wrote: > > > > Erikjan Rijkers writes: > > > I don't think these "were"s are wrong but arguably changing them to > > > "h

Re: fix typos

2022-08-03 Thread Robert Haas
On Tue, Aug 2, 2022 at 4:32 AM Erik Rijkers wrote: > > The part of the sentence inside parentheses is not clear to me, before > > or after the patch: > > > > Dropping an extension causes its component objects, and other > > explicitly > > dependent routines (see , > > - the depends on

Re: support for SSE2 intrinsics

2022-08-03 Thread Nathan Bossart
On Wed, Aug 03, 2022 at 12:00:39PM +0700, John Naylor wrote: > Thanks for checking! Here's a concrete patch for testing. LGTM -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Robert Haas
On Wed, Aug 3, 2022 at 10:13 AM Tom Lane wrote: > If you have a different solution that you can implement by, say, > tomorrow, then go for it. But I want to see some fix in there > within about 24 hours, because 15beta3 wraps on Monday and we > will need at least a few days to see if the buildfar

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Justin Pryzby
On Wed, Aug 03, 2022 at 04:45:23PM +0530, Dilip Kumar wrote: > Another version of the patch which closes the smgr at the end using > smgrcloserellocator() and I have also added a commit message. Thanks for providing a patch. This seems to fix the second problem with accessing freed memory. But I

Re: [PATCH] CF app: add "Returned: Needs more interest"

2022-08-03 Thread Jacob Champion
On Tue, Aug 2, 2022 at 8:00 PM Julien Rouhaud wrote: > I'm personally fine with the current statutes, as closing a patch with RwF > explaining that there was no interest is still a feedback, Hi Julien, Making that explanation each time we intend to close a patch "needs interest" takes a lot of t

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Dilip Kumar
On Wed, 3 Aug 2022 at 9:22 PM, Robert Haas wrote: > On Wed, Aug 3, 2022 at 7:15 AM Dilip Kumar wrote: > > Another version of the patch which closes the smgr at the end using > > smgrcloserellocator() and I have also added a commit message. > > Hmm, but didn't we decide against doing it that way

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-08-03 Thread Robert Haas
On Wed, Aug 3, 2022 at 7:15 AM Dilip Kumar wrote: > Another version of the patch which closes the smgr at the end using > smgrcloserellocator() and I have also added a commit message. Hmm, but didn't we decide against doing it that way intentionally? The comment you're deleting says "If we didn't

Re: A test for replay of regression tests

2022-08-03 Thread Justin Pryzby
On Thu, Dec 09, 2021 at 12:10:23PM +1300, Thomas Munro wrote: > This adds 2 whole minutes to the recovery check, when running with the > Windows serial-only scripts on Cirrus CI (using Andres's CI patches). > For Linux it adds ~20 seconds to the total of -j8 check-world. > Hopefully that's time wel

Re: [doc] fix a potential grammer mistake

2022-08-03 Thread Junwang Zhao
Attachment is a corrected version based on Tom's suggestion. Thanks. On Wed, Aug 3, 2022 at 9:56 PM Tom Lane wrote: > > Erikjan Rijkers writes: > > I don't think these "were"s are wrong but arguably changing them to > > "have" helps non-native speakers (like myself), as it doesn't change the

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Jonathan S. Katz
> On Aug 3, 2022, at 10:14 AM, Tom Lane wrote: > > Robert Haas writes: >>> On Tue, Aug 2, 2022 at 3:51 PM Tom Lane wrote: >>> I also think that ">=" is a sufficient requirement. > >> I don't really like this approach. Imagine that the code got broken in >> such a way that relfrozenxid and r

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Tom Lane
Robert Haas writes: > On Tue, Aug 2, 2022 at 3:51 PM Tom Lane wrote: >> I also think that ">=" is a sufficient requirement. > I don't really like this approach. Imagine that the code got broken in > such a way that relfrozenxid and relminmxid were set to a value chosen > at random - say, the con

Re: pg15b2: large objects lost on upgrade

2022-08-03 Thread Robert Haas
On Tue, Aug 2, 2022 at 3:51 PM Tom Lane wrote: > > The test does look helpful and it would catch regressions. Loosely > > quoting Robert on a different point upthread, we don't want to turn off > > the alarm just because it's spuriously going off. > > I think the weakened check is OK (and possibly

Re: [doc] fix a potential grammer mistake

2022-08-03 Thread Tom Lane
Erikjan Rijkers writes: > I don't think these "were"s are wrong but arguably changing them to > "have" helps non-native speakers (like myself), as it doesn't change the > meaning significantly as far as I can see. I think it does --- it changes the meaning from passive to active. I don't nece

Re: automatically generating node support functions

2022-08-03 Thread Tom Lane
Amit Kapila writes: > I have a question related to commit 964d01ae90. Today, after getting > the latest code, when I compiled it on my windows machine, it lead to > a compilation error because the outfuncs.funcs.c was not regenerated. > I did the usual steps which I normally perform after getting

  1   2   >