Re: Perform streaming logical transactions by background workers and parallel apply

2023-05-03 Thread Amit Kapila
On Tue, May 2, 2023 at 9:46 AM Amit Kapila wrote: > > On Tue, May 2, 2023 at 9:06 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Friday, April 28, 2023 2:18 PM Masahiko Sawada > > wrote: > > > > > > > > > > > Alexander, does the proposed patch fix the problem you are facing? > > > > Sawada-San, an

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-05-03 Thread John Naylor
On Wed, May 3, 2023 at 10:04 AM Peter Geoghegan wrote: > > That's not the only reason, though. I also genuinely don't have the > foggiest notion what was behind what you said. In particular, this > part still makes zero sense to me: > > "Claim that others are holding you back, and then try to move

Re: Support logical replication of DDLs

2023-05-03 Thread Alvaro Herrera
Patch 0001 adds a new event trigger type that can be fired, but it's missing documentation and its own tests. (I think part of the docs are in 0002, but that seems to be only the changes to the supported operations table, without any other explanation for it in sect1 event-trigger-definition, and

Re: Add PQsendSyncMessage() to libpq

2023-05-03 Thread Alvaro Herrera
On 2023-May-02, Robert Haas wrote: > On Mon, May 1, 2023 at 8:42 PM Michael Paquier wrote: > > Another thing that may matter in terms of extensibility? Would a > > boolean argument really be the best design? Could it be better to > > have instead one API with a bits32 and some flags controlling

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-05-03 Thread Amit Kapila
On Tue, May 2, 2023 at 4:52 PM Drouvot, Bertrand wrote: > > > As per your suggestion, changing the insert ordering (like in V8 attached) > makes it now work on the failing environment too. > I think it is better to use wait_for_replay_catchup() to wait for standby to catch up. I have changed tha

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-05-03 Thread Alvaro Herrera
On 2023-May-02, Julien Rouhaud wrote: > On Tue, May 02, 2023 at 12:55:18PM +0200, Alvaro Herrera wrote: > > A point on preserving physical replication slots: because we change WAL > > format from one major version to the next (adding new messages or > > changing format for other messages), we can

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-05-03 Thread Drouvot, Bertrand
Hi, On 5/3/23 12:29 PM, Amit Kapila wrote: On Tue, May 2, 2023 at 4:52 PM Drouvot, Bertrand wrote: As per your suggestion, changing the insert ordering (like in V8 attached) makes it now work on the failing environment too. I think it is better to use wait_for_replay_catchup() to wait fo

Re: pg_stat_io for the startup process

2023-05-03 Thread Melih Mutlu
Hi, Andres Freund , 27 Nis 2023 Per, 19:27 tarihinde şunu yazdı: > Huh - do you have a link to the failure? That's how I would expect it to be > done. I think I should have registered it in the beginning of PerformWalRecovery() and not just before the main redo loop since HandleStartupProcInter

Re: Logging parallel worker draught

2023-05-03 Thread Robert Haas
On Tue, May 2, 2023 at 6:57 AM Amit Kapila wrote: > We can output this at the LOG level to avoid running the server at > DEBUG1 level. There are a few other cases where we are not able to > spawn the worker or process and those are logged at the LOG level. For > example, "could not fork autovacuum

Re: issue with meson builds on msys2

2023-05-03 Thread Andrew Dunstan
On 2023-04-27 Th 18:18, Andres Freund wrote: Hi, On 2023-04-26 09:59:05 -0400, Andrew Dunstan wrote: Still running into this, and I am rather stumped. This is a blocker for buildfarm support for meson: Here's a simple illustration of the problem. If I do the identical test with a non-meson bu

Re: Rename 'lpp' to 'lp' in heapam.c

2023-05-03 Thread Robert Haas
On Tue, May 2, 2023 at 10:18 PM David Rowley wrote: > I don't really agree that one is any more correct than the other. I > also don't think we should be making changes like this as doing this > may give some false impression that we have some standard to follow > here that a local variable of a g

Re: Move defaults toward ICU in 16?

2023-05-03 Thread Jonathan S. Katz
On 4/17/23 2:33 PM, Tom Lane wrote: Jeff Davis writes: Is now a reasonable time to check it in and see what breaks? It looks like there are quite a few buildfarm members that specify neither -- with-icu nor --without-icu. I see you just pinged buildfarm-members again, so I'd think it's polite

Re: pg_stat_io not tracking smgrwriteback() is confusing

2023-05-03 Thread Jonathan S. Katz
On 4/27/23 11:36 AM, Melanie Plageman wrote: Thanks for the review! On Wed, Apr 26, 2023 at 10:22 PM Kyotaro Horiguchi wrote: At Wed, 26 Apr 2023 17:08:14 -0400, Melanie Plageman wrote in On Mon, Apr 24, 2023 at 9:29 PM Melanie Plageman wrote: I've yet to cook up a client backend test ca

Re: psql: Add role's membership options to the \du+ command

2023-05-03 Thread Jonathan S. Katz
On 4/13/23 8:44 AM, Pavel Luzanov wrote: P.S. If no objections I plan to add this patch to Open Items for v16 https://wiki.postgresql.org/wiki/PostgreSQL_16_Open_Items [RMT hat] I don't see why this is an open item as this feature was not committed for v16. Open items typically revolve aroun

Re: psql: Add role's membership options to the \du+ command

2023-05-03 Thread David G. Johnston
On Wed, May 3, 2023 at 9:00 AM Jonathan S. Katz wrote: > On 4/13/23 8:44 AM, Pavel Luzanov wrote: > > > P.S. If no objections I plan to add this patch to Open Items for v16 > > https://wiki.postgresql.org/wiki/PostgreSQL_16_Open_Items > > [RMT hat] > > I don't see why this is an open item as this

Re: psql: Add role's membership options to the \du+ command

2023-05-03 Thread Tom Lane
"David G. Johnston" writes: > On Wed, May 3, 2023 at 9:00 AM Jonathan S. Katz > wrote: >> I don't see why this is an open item as this feature was not committed >> for v16. Open items typically revolve around committed features. > The argument is that updating the psql \d views to show the newly

Re: psql: Add role's membership options to the \du+ command

2023-05-03 Thread Jonathan S. Katz
On 5/3/23 12:25 PM, Tom Lane wrote: "David G. Johnston" writes: On Wed, May 3, 2023 at 9:00 AM Jonathan S. Katz wrote: I don't see why this is an open item as this feature was not committed for v16. Open items typically revolve around committed features. The argument is that updating the p

Re: SQL/JSON revisited

2023-05-03 Thread Matthias Kurz
On Wed, 5 Apr 2023 at 09:53, Alvaro Herrera wrote: > > Okay, I've marked the CF entry as committed then. > This was marked as commited in the 2023-03 commitfest, however there are still patches missing (for example the JSON_TABLE one). However, I can not see an entry in the current 2023-07 Commi

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound

2023-05-03 Thread Peter Geoghegan
On Wed, May 3, 2023 at 12:30 AM John Naylor wrote: > I went to go find the phrase that I thought I was reacted to, and ... > nothing. I am also baffled. My comment was inexcusable. I'd quite like to drop this topic, and get on with the work at hand. But before I do that, I ask you to consider o

Re: SQL/JSON revisited

2023-05-03 Thread Alvaro Herrera
On 2023-May-03, Matthias Kurz wrote: > On Wed, 5 Apr 2023 at 09:53, Alvaro Herrera wrote: > > > Okay, I've marked the CF entry as committed then. > > This was marked as commited in the 2023-03 commitfest, however there are > still patches missing (for example the JSON_TABLE one). > However, I c

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-05-03 Thread Muhammad Malik
Hi, Could you please share repro steps for running these benchmarks? I am doing performance testing in this area and want to use the same benchmarks. Thanks, Muhammad From: Andres Freund Sent: Friday, October 28, 2022 7:54 PM To: pgsql-hack...@postgresql.org ; T

Re: issue with meson builds on msys2

2023-05-03 Thread Andres Freund
Hi, On 2023-05-03 09:20:28 -0400, Andrew Dunstan wrote: > On 2023-04-27 Th 18:18, Andres Freund wrote: > > Hi, > > > > On 2023-04-26 09:59:05 -0400, Andrew Dunstan wrote: > > > Still running into this, and I am rather stumped. This is a blocker for > > > buildfarm support for meson: > > > > > >

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-05-03 Thread Andres Freund
Hi, On 2023-05-03 18:25:51 +, Muhammad Malik wrote: > Could you please share repro steps for running these benchmarks? I am doing > performance testing in this area and want to use the same benchmarks. The email should contain all the necessary information. What are you missing? c=16;psql

Re: SQL/JSON revisited

2023-05-03 Thread Matthias Kurz
On Wed, 3 May 2023 at 20:17, Alvaro Herrera wrote: > > I would suggest to start a new thread with updated patches, and then a new > commitfest entry can be created with those. > Whoever starts that new thread, please link link it here, I am keen to follow it ;) Thanks a lot! Thanks a lot for all

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-05-03 Thread Muhammad Malik
> I use a script like: > c=16;psql -c 'DROP TABLE IF EXISTS copytest_0; CREATE TABLE copytest_0(data > text not null);' && time /srv/dev/build/m-opt/src/bin/pgbench/pgbench -n -P1 > -c$c -j$c -t$((1024/$c)) -f ~/tmp/copy.sql && psql -c 'TRUNCATE copytest_0' > >[1] COPY (SELECT repeat(random():

Re: issue with meson builds on msys2

2023-05-03 Thread Andrew Dunstan
On 2023-05-03 We 09:20, Andrew Dunstan wrote: On 2023-04-27 Th 18:18, Andres Freund wrote: Hi, On 2023-04-26 09:59:05 -0400, Andrew Dunstan wrote: Still running into this, and I am rather stumped. This is a blocker for buildfarm support for meson: Here's a simple illustration of the proble

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-05-03 Thread Nathan Bossart
Here's a new version of the patch. Besides adding comments and a commit message, I made sure to decrement the reference count for pltargs in the PG_CATCH block (which means that pltargs likely needs to be volatile). I'm not too wild about moving the chunk of code for pltargs like this, but I have

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-05-03 Thread Tom Lane
Nathan Bossart writes: > Here's a new version of the patch. Besides adding comments and a commit > message, I made sure to decrement the reference count for pltargs in the > PG_CATCH block (which means that pltargs likely needs to be volatile). Hmm, actually I think these changes should allow yo

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-05-03 Thread Nathan Bossart
On Wed, May 03, 2023 at 04:33:32PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Here's a new version of the patch. Besides adding comments and a commit >> message, I made sure to decrement the reference count for pltargs in the >> PG_CATCH block (which means that pltargs likely needs to be

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-05-03 Thread Andres Freund
Hi, On 2023-05-03 19:29:46 +, Muhammad Malik wrote: > > I use a script like: > > > c=16;psql -c 'DROP TABLE IF EXISTS copytest_0; CREATE TABLE copytest_0(data > > text not null);' && time /srv/dev/build/m-opt/src/bin/pgbench/pgbench -n > > -P1 -c$c -j$c -t$((1024/$c)) -f ~/tmp/copy.sql && ps

Re: Overhauling "Routine Vacuuming" docs, particularly its handling of freezing

2023-05-03 Thread Peter Geoghegan
Hi Samay, On Tue, May 2, 2023 at 11:40 PM samay sharma wrote: > Thanks for taking the time to do this. It is indeed difficult work. Thanks for the review! I think that this is something that would definitely benefit from a perspective such as yours. > There are things I like about the changes y

Re: issue with meson builds on msys2

2023-05-03 Thread Andrew Dunstan
On 2023-05-03 We 14:26, Andres Freund wrote: Hi, On 2023-05-03 09:20:28 -0400, Andrew Dunstan wrote: On 2023-04-27 Th 18:18, Andres Freund wrote: Hi, On 2023-04-26 09:59:05 -0400, Andrew Dunstan wrote: Still running into this, and I am rather stumped. This is a blocker for buildfarm support

Re: Overhauling "Routine Vacuuming" docs, particularly its handling of freezing

2023-05-03 Thread Peter Geoghegan
On Wed, May 3, 2023 at 2:59 PM Peter Geoghegan wrote: > Coming up with a new user-facing name for xidStopLimit is already on > my TODO list (it's surprisingly hard). I have used that name so far > because it unambiguously refers to the exact thing that I want to talk > about when discussing the wo

Re: Direct I/O

2023-05-03 Thread Thomas Munro
On Wed, Apr 19, 2023 at 7:35 AM Greg Stark wrote: > On Mon, 17 Apr 2023 at 17:45, Thomas Munro wrote: > > (2) without a page cache, you really need to size your shared_buffers > > adequately and we can't do that automatically. > > Well I'm more optimistic... That may not always be impossible.

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-05-03 Thread vignesh C
On Wed, 3 May 2023 at 15:59, Amit Kapila wrote: > > On Tue, May 2, 2023 at 4:52 PM Drouvot, Bertrand > wrote: > > > > > > As per your suggestion, changing the insert ordering (like in V8 attached) > > makes it now work on the failing environment too. > > > > I think it is better to use wait_for_

"CREATE RULE ... ON SELECT": redundant?

2023-05-03 Thread Ian Lawrence Barwick
Hi Commit b23cd185f [1] forbids manual creation of ON SELECT rule on a table, and updated the main rules documentation [2], but didn't update the corresponding CREATE RULE page [3]. [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b23cd185fd5410e5204683933f848d4583e34b35 [2] ht

Re: "CREATE RULE ... ON SELECT": redundant?

2023-05-03 Thread Tom Lane
Ian Lawrence Barwick writes: > While poking around at an update for that, unless I'm missing something it is > now not possible to use "CREATE RULE ... ON SELECT" for any kind of relation, > given that it's disallowed on views / material views already. What makes you think it's disallowed on view

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-05-03 Thread Hayato Kuroda (Fujitsu)
Dear Alvaro, Thanks for giving suggestion! > A point on preserving physical replication slots: because we change WAL > format from one major version to the next (adding new messages or > changing format for other messages), we can't currently rely on physical > slots working across different majo

Re: "CREATE RULE ... ON SELECT": redundant?

2023-05-03 Thread Ian Lawrence Barwick
2023年5月4日(木) 12:51 Tom Lane : > > Ian Lawrence Barwick writes: > > While poking around at an update for that, unless I'm missing something it > > is > > now not possible to use "CREATE RULE ... ON SELECT" for any kind of > > relation, > > given that it's disallowed on views / material views alre

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-05-03 Thread Amit Kapila
On Thu, May 4, 2023 at 8:37 AM vignesh C wrote: > > Thanks for posting the updated patch, I had run this test in a loop of > 100 times to verify that there was no failure because of race > conditions. The 100 times execution passed successfully. > > One suggestion: > "wal file" should be changed t

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-05-03 Thread Nathan Bossart
On Wed, May 03, 2023 at 01:58:38PM -0700, Nathan Bossart wrote: > With this change, pltdata isn't modified in the PG_TRY section, and the > only modification of pltargs happens after all elogs. It might be worth > keeping pltargs volatile in case someone decides to add an elog() in the > future, b

Re: PL/Python: Fix return in the middle of PG_TRY() block.

2023-05-03 Thread Nathan Bossart
On Wed, May 03, 2023 at 09:54:13PM -0700, Nathan Bossart wrote: > Here's a new patch that removes the volatile marker from pltdata. Gah, right after I sent that, I realized we can remove one more volatile marker. Sorry for the noise. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Order changes in PG16 since ICU introduction

2023-05-03 Thread Jeff Davis
On Fri, 2023-04-28 at 14:35 -0700, Jeff Davis wrote: > On Thu, 2023-04-27 at 14:23 +0200, Daniel Verite wrote: > > This should be pg_strcasecmp(...) == 0 > > Good catch, thank you! Fixed in updated patches. Rebased patches. === 0001: do not convert C to en-US-u-va-posix I plan to commit this so

Re: Add two missing tests in 035_standby_logical_decoding.pl

2023-05-03 Thread Drouvot, Bertrand
Hi, On 5/4/23 6:43 AM, Amit Kapila wrote: On Thu, May 4, 2023 at 8:37 AM vignesh C wrote: Thanks for posting the updated patch, I had run this test in a loop of 100 times to verify that there was no failure because of race conditions. The 100 times execution passed successfully. One suggesti

Re: "CREATE RULE ... ON SELECT": redundant?

2023-05-03 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> Now, this is certainly syntax that's deprecated in favor of using Tom> CREATE OR REPLACE VIEW, but I'm very hesitant to remove it. ISTR Tom> that ancient pg_dump files used it in cases involving circular Tom> dependencies. I thought they used CREATE RULE

Re: "CREATE RULE ... ON SELECT": redundant?

2023-05-03 Thread Andrew Gierth
> "Andrew" == Andrew Gierth writes: Andrew> I thought they used CREATE RULE on a table? Andrew> In fact here is an example from a pg 9.5 pg_dump output (with Andrew> cruft removed): And checking other versions, 9.6 is the same, it's only with pg 10 that it switches to creating a dummy vi

Re: ssl tests aren't concurrency safe due to get_free_port()

2023-05-03 Thread Peter Eisentraut
On 25.04.23 12:27, Peter Eisentraut wrote: On 20.11.22 16:10, Andrew Dunstan wrote: On 2022-11-19 Sa 15:16, Andres Freund wrote: Hi, On 2022-11-19 10:56:33 -0500, Andrew Dunstan wrote: Perhaps we should just export a directory in configure instead of this guessing game? I think the obvious

Re: Autogenerate some wait events code and documentation

2023-05-03 Thread Drouvot, Bertrand
Hi, On 5/1/23 1:59 AM, Michael Paquier wrote: On Fri, Apr 28, 2023 at 02:29:13PM +0200, Drouvot, Bertrand wrote: On 4/27/23 8:13 AM, Michael Paquier wrote: But do we need to merge more data than necessary? We could do things in the simplest fashion possible while making the docs and code use

Re: Autogenerate some wait events code and documentation

2023-05-03 Thread Drouvot, Bertrand
Hi, On 5/2/23 4:50 AM, Thomas Munro wrote: [patch] This is not a review of the perl/make/meson glue/details, but I just wanted to say thanks for working on this Bertrand & Michael, at a quick glance that .txt file looks like it's going to be a lot more fun to maintain! Thanks Thomas! Yeah an