Re: Refactoring postmaster's code to cleanup after child exit

2024-08-18 Thread Heikki Linnakangas
On 18/08/2024 11:00, Alexander Lakhin wrote: 10.08.2024 00:13, Heikki Linnakangas wrote: Committed the patches up to and including this one, with tiny comment changes. I've noticed that on the current HEAD server.log contains binary data (an invalid process name) after a child crash. For exam

Re: Speed up Hash Join by teaching ExprState about hashing

2024-08-18 Thread David Rowley
Thanks for having a look. On Sat, 17 Aug 2024 at 23:21, Tels wrote: > Is it nec. to rotate existing_hash here before checking for isnull? > Because in case of isnull, isn't the result of the rotate thrown away? Yeah, I think that it's worthwhile moving that to after the isnull check so as not to

Re: Conflict detection and logging in logical replication

2024-08-18 Thread Amit Kapila
On Mon, Aug 19, 2024 at 11:54 AM shveta malik wrote: > > On Mon, Aug 19, 2024 at 11:37 AM Amit Kapila wrote: > > > > On Mon, Aug 19, 2024 at 9:08 AM shveta malik wrote: > > > > > > On Sun, Aug 18, 2024 at 2:27 PM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > Attach the V16 patch which addre

Re: Fsync (flush) all inserted WAL records

2024-08-18 Thread Michael Paquier
On Wed, Aug 07, 2024 at 06:00:45PM +0300, Aleksander Alekseev wrote: > Assuming the function has value, as you claim, I see no reason not to > expose it similarly to pg_current_wal_*(). On top of that you will > have to test-cover it anyway. The easiest way to do it will be to have > an SQL-wrapper

Re: Use read streams in pg_visibility

2024-08-18 Thread Michael Paquier
On Tue, Aug 13, 2024 at 03:22:27PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > I am working on using the read stream in pg_visibility. There are two > places to use it: > > 1- collect_visibility_data() > > This one is straightforward. I created a read stream object if > 'include_pd' is true becaus

Re: Thread-safe nl_langinfo() and localeconv()

2024-08-18 Thread Thomas Munro
Here is a slightly better version of patch 0003. I removed some unnecessary refactoring, making the patch smaller. FTR I wrote a small program[1] for CI to test the assumptions about Windows in 0001. I printed out the addresses of the objects, to confirm that different threads were looking at di

Re: Normalize queries starting with SET for pg_stat_statements

2024-08-18 Thread Michael Paquier
On Tue, Aug 13, 2024 at 10:54:34AM -0400, Greg Sabino Mullane wrote: > Now that I've spent some time away from this, I'm reconsidering why we are > going through all the trouble of semi-jumbling SET statements. Maybe we > just keep it simple and everything becomes "SET myvar = $1" or even "SET > my

Re: Conflict detection and logging in logical replication

2024-08-18 Thread shveta malik
On Mon, Aug 19, 2024 at 11:37 AM Amit Kapila wrote: > > On Mon, Aug 19, 2024 at 9:08 AM shveta malik wrote: > > > > On Sun, Aug 18, 2024 at 2:27 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Attach the V16 patch which addressed the comments we agreed on. > > > I will add a doc patch to explain

Re: Create syscaches for pg_extension

2024-08-18 Thread Michael Paquier
On Tue, Aug 13, 2024 at 05:38:55PM +0300, Alexander Korotkov wrote: > +1 from me too I won't hide that I've wanted that in the past.. -- Michael signature.asc Description: PGP signature

Re: Apply PGDLLIMPORT markings to some GUC variables

2024-08-18 Thread Michael Paquier
On Tue, Aug 13, 2024 at 03:00:00PM -0400, Robert Haas wrote: > This seems correct to me. It is not the first time that this happens in recent history. Would it be worth automating that? I would guess a TAP test that takes a copy of the headers, applies the changes while filtering the few excepti

Improve pg_re_throw: check if sigjmp_buf is valid and report error

2024-08-18 Thread Xiaoran Wang
https://www.postgresql.org/message-id/canncwrjtse6wkkus_y8wj2phvrvaqpxmk_qtepsf_+nvpyx...@mail.gmail.com As the problem discussed in the above thread, I also run into that. Besides updating the doc, I would like to report a error for it. If the code in PG_TRY contains any non local control flow o

Re: Conflict detection and logging in logical replication

2024-08-18 Thread Amit Kapila
On Mon, Aug 19, 2024 at 9:08 AM shveta malik wrote: > > On Sun, Aug 18, 2024 at 2:27 PM Zhijie Hou (Fujitsu) > wrote: > > > > Attach the V16 patch which addressed the comments we agreed on. > > I will add a doc patch to explain the log format after the 0001 is RFC. > > > > Thank You for addressin

Re: Pgoutput not capturing the generated columns

2024-08-18 Thread Peter Smith
Hi, Here are my review comments for v27-0001. == contrib/test_decoding/expected/generated_columns.out contrib/test_decoding/sql/generated_columns.sql +-- By default, 'include-generated-columns' is enabled, so the values for the generated column 'b' will be replicated even if it is not explici

Re: Improving the notation for ecpg.addons rules

2024-08-18 Thread Tom Lane
Michael Paquier writes: > It looks like %replace_line expects all its elements to have one space > between each token, still this is not enforced with a check across its > hardcoded elements? Yeah, I was wondering about that. I wouldn't do it exactly like that, but with a check that the entry ge

Re: Improving the notation for ecpg.addons rules

2024-08-18 Thread Michael Paquier
On Sun, Aug 18, 2024 at 01:13:36PM -0400, Tom Lane wrote: > While I've not done it in the attached, perhaps it would be > but I think that might be a step too far. IMO it's not adding much > readability, and it seems like introducing an unnecessary dependency > on exactly how the gram.y alternativ

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-08-18 Thread Tom Lane
Robert Haas writes: > I feel like what you're really complaining about here is that libpq is > not properly versioned. We've just been calling it libpq.so.5 forever > instead of bumping the version number when we change stuff. Maybe we > should start doing that, because that's exactly what version

Re: Conflict detection and logging in logical replication

2024-08-18 Thread shveta malik
On Mon, Aug 19, 2024 at 9:07 AM shveta malik wrote: > > On Sun, Aug 18, 2024 at 2:27 PM Zhijie Hou (Fujitsu) > wrote: > > > > Attach the V16 patch which addressed the comments we agreed on. > > I will add a doc patch to explain the log format after the 0001 is RFC. > > > > Thank You for addressin

Re: Document that PG_TRY block cannot have a return statement

2024-08-18 Thread Xiaoran Wang
LGTM! Serpent 于2024年8月15日周四 15:01写道: > Hi, > > What about this wording: > > The code that might throw ereport(ERROR) cannot contain any non local > control flow other than ereport(ERROR) e.g.: return, goto, break, continue. > In other words once PG_TRY() is executed, either PG_CATCH() or > PG_FI

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-08-18 Thread Robert Haas
On Sat, Aug 17, 2024 at 5:32 AM Jelte Fennema-Nio wrote: > Not introducing new APIs definitely is useful to clients and the > community. Before users can use a new API, their libpq wrapper needs > to expose this new function that calls it through FFI. First of all > this requires work from client

RE: Conflict detection and logging in logical replication

2024-08-18 Thread Zhijie Hou (Fujitsu)
On Friday, August 16, 2024 7:47 PM Michail Nikolaev wrote: > > I think you might misunderstand the behavior of CheckAndReportConflict(), > > even if it found a conflict, it still inserts the tuple into the index which > > means the change is anyway applied. > > > In the above conditions where a

Re: Conflict detection and logging in logical replication

2024-08-18 Thread shveta malik
On Sun, Aug 18, 2024 at 2:27 PM Zhijie Hou (Fujitsu) wrote: > > Attach the V16 patch which addressed the comments we agreed on. > I will add a doc patch to explain the log format after the 0001 is RFC. > Thank You for addressing comments. Please see this scenario: create table tab1(pk int primar

Re: Logical Replication of sequences

2024-08-18 Thread Peter Smith
Here are my review comments for the latest patchset v20240817-0001. No changes. No comments. v20240817-0002. No changes. No comments. v20240817-0003. See below. v20240817-0004. See below. v20240817-0005. No changes. No comments. // v20240817-0003 and 0004. (This is a repeat of the same comm

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2024-08-18 Thread Michael Paquier
On Sat, Aug 10, 2024 at 10:50:55AM -0700, Noah Misch wrote: > On Sat, Jul 27, 2024 at 07:24:33AM +0900, Michael Paquier wrote: >> I've just applied now the remaining pieces down to 17. > > Comparing commit c9e2457 to the patch in zqgvzsbw5tgkq...@paquier.xyz, the > commit lacks the slru.c portion.

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Tom Lane
Thomas Munro writes: > On Mon, Aug 19, 2024 at 12:51 PM Tom Lane wrote: >> I'm not sure why the CI script should trigger that though. It >> does do a couple of "port" calls without "sudo", but not in places >> where the state should be only partially upgraded. > Oooh, I think I see where we mis

Re: Feature Request: Extending PostgreSQL's Identifier Length Limit

2024-08-18 Thread Bruce Momjian
On Thu, Jul 18, 2024 at 11:45:45AM +0200, Álvaro Herrera wrote: > On 2024-Jul-18, David HJ wrote: > > > As a long-time PostgreSQL user, I've increasingly run into issues with the > > 63-byte limit for identifiers, particularly table names. This limit, while > > historically sufficient, is becoming

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Thomas Munro
On Mon, Aug 19, 2024 at 12:51 PM Tom Lane wrote: > I'm not sure why the CI script should trigger that though. It > does do a couple of "port" calls without "sudo", but not in places > where the state should be only partially upgraded. Oooh, I think I see where we missed a sudo: if [ -n "$(port

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Tom Lane
I wrote: > Interesting. Now that I've finished "sudo port upgrade outdated", > my laptop is back to a state where unprivileged "port outdated" > is successful. I confirmed on another machine that, immediately after "sudo port selfupdate" from 2.9.3 to 2.10.1, I get $ port outdated sqlite error:

Some refactoring for fixed-numbered stats template in injection_points

2024-08-18 Thread Michael Paquier
Hi all, While reading again the code of injection_stats_fixed.c that holds the template for fixed-numbered pgstats, I got an idea to make the code a bit more elegant. Rather than using a single routine to increment the counters, we could use a series of routines with its internals hidden in a mac

Re: Injection Points remaining stats

2024-08-18 Thread Michael Paquier
On Mon, Aug 12, 2024 at 10:25:13AM -0400, Yogesh Sharma wrote: > Attaching a patch to add remaining cached and loaded stats as mentioned in > commit f68cd847fa40ead44a786b9c34aff9ccc048004b message. Existing TAP tests > were updated to handle new stats. This patch has been tested on HEAD using > "m

Re: Emitting JSON to file using COPY TO

2024-08-18 Thread jian he
On Mon, Apr 1, 2024 at 8:00 PM jian he wrote: > rebased. minor cosmetic error message change. I think all the issues in this thread have been addressed. From b96dfe41f0935b08b1190f399e29ee2450169529 Mon Sep 17 00:00:00 2001 From: jian he Date: Sat, 17 Aug 2024 11:08:25 +0800 Subject: [PATCH v11

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Tom Lane
Thomas Munro writes: > On Mon, Aug 19, 2024 at 10:55 AM Tom Lane wrote: >> What this smells like is that MacPorts has to do some kind of database >> update as a result of its major version change, and there are code >> paths that are not expecting that to get invoked. It makes sense >> that unpr

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Thomas Munro
On Mon, Aug 19, 2024 at 10:55 AM Tom Lane wrote: > Thomas Munro writes: > > I still don't know what's happening. In case it helps someone else > > see it, the error comes from "sudo port unsetrequested installed". > > But in any case, switching to 2.10.1 seems to do the trick. See > > attached.

Re: possible issue in postgres_fdw batching

2024-08-18 Thread Tom Lane
Tomas Vondra writes: > Consider this simplified example: > CREATE TABLE t (a INT); > CREATE FOREIGN TABLE f (a INT) SERVER loopback > OPTIONS (table_name 't'); > CREATE FUNCTION counter() RETURNS int LANGUAGE sql AS > $$ SELECT COUNT(*) FROM f $$; > And n

possible issue in postgres_fdw batching

2024-08-18 Thread Tomas Vondra
Hi, There's a pgsql-bugs report [1] about a behavior change with batching enabled in postgres_fdw. While I ultimately came to the conclusion the behavior change is not a bug, I find it annoying. But I'm not 100% sure about the "not a bug" conclusion, and and the same time I can't think of a way to

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Tom Lane
Thomas Munro writes: > I still don't know what's happening. In case it helps someone else > see it, the error comes from "sudo port unsetrequested installed". > But in any case, switching to 2.10.1 seems to do the trick. See > attached. Interesting. Now that I've finished "sudo port upgrade ou

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Thomas Munro
On Mon, Aug 19, 2024 at 10:01 AM Thomas Munro wrote: > Oh, it already is a cache miss and thus a fresh installation, in > Tomas's example. I can reproduce that in my own Github account by > making a trivial change to ci_macports_packages.sh to I get a cache > miss too. It appears to install macp

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Tom Lane
Thomas Munro writes: > Oh, it already is a cache miss and thus a fresh installation, in > Tomas's example. I can reproduce that in my own Github account by > making a trivial change to ci_macports_packages.sh to I get a cache > miss too. It appears to install macports just fine, and then a later

Re: Drop database command will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2024-08-18 Thread Tomas Vondra
On 8/16/24 13:26, Tomas Vondra wrote: > Hi Ayush, > > ... > > So this fix seems reasonable. > I've pushed this to all affected branches, except for 11 which is EOL. I thought about adding a test, but I couldn't think of a TAP test where this would really fit, and it didn't seem very practical

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-08-18 Thread Alexander Korotkov
On Sat, Aug 10, 2024 at 6:57 PM Dmitry Koval wrote: > > Probably > > QueryCompletion struct fits this purpose best from the existing > > parameters. Attached draft patch implements returning oid of newly > > created relation as part of QueryCompletion. Thoughts? > > I agree, returning the oid of

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Thomas Munro
On Mon, Aug 19, 2024 at 7:52 AM Thomas Munro wrote: > The macOS major version and hash of the MacPorts package install > script are in the cache key for that (see 64c39bd5), so a change to > that script would make a totally fresh installation, and hopefully > work. I will look into that, but it w

Re: CI cpluspluscheck failures

2024-08-18 Thread Thomas Munro
On Mon, Aug 19, 2024 at 12:10 AM Thomas Munro wrote: > I'm not sure why the first one started happening at the commit > (aa2d6b15) that caused the second one, I feel like I'm missing > something... What I was missing is that the LLVM warnings were already present, but not causing failures because

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Thomas Munro
On Mon, Aug 19, 2024 at 2:07 AM Peter Eisentraut wrote: > [14:01:37.431] Error: /opt/local/bin/port: Failed to initialize > MacPorts, sqlite error: attempt to write a readonly database (8) while > executing query: CREATE INDEX registry.snapshot_file_id ON > snapshot_files(id) Hmmm. Basically the

Re: Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Tomas Vondra
On 8/18/24 16:07, Peter Eisentraut wrote: > The Cirrus CI for REL_16_STABLE and REL_15_STABLE for macOS is > apparently broken right now.  Here is a log example: > > [13:57:11.305] sh src/tools/ci/ci_macports_packages.sh \ > [13:57:11.305]   ccache \ > [13:57:11.305]   icu \ > [13:57:11.305]   ker

Improving the notation for ecpg.addons rules

2024-08-18 Thread Tom Lane
I've gotten annoyed by the notation used for ecpg.addons rules, in which all the tokens of the gram.y rule to be modified are just concatenated. This is unreadable and potentially ambiguous: ECPG: fetch_argsABSOLUTE_PSignedIconstopt_from_incursor_name addon The attached draft patch changes thing

Re: The xversion-upgrade test fails to stop server

2024-08-18 Thread Alexander Lakhin
Hello Andrew, 04.06.2024 14:56, Andrew Dunstan wrote: but I can't see ../snapshot_too_old/output_iso/regression.diff and .../snapshot_too_old/output_iso/log/postmaster.log in the log. will do. I've discovered a couple of other failures where the interesting log files are not saved. [1

Re: Meson far from ready on Windows

2024-08-18 Thread walther
Andres Freund: That's not necessarily true. The nix package manager and thus NixOS track all dependencies for a piece of software. If any of the dependencies are updated, all dependents are rebuilt, too. So the security concern doesn't apply here. There is a "static overlay", which builds ever

Cirrus CI for macOS branches 16 and 15 broken

2024-08-18 Thread Peter Eisentraut
The Cirrus CI for REL_16_STABLE and REL_15_STABLE for macOS is apparently broken right now. Here is a log example: [13:57:11.305] sh src/tools/ci/ci_macports_packages.sh \ [13:57:11.305] ccache \ [13:57:11.305] icu \ [13:57:11.305] kerberos5 \ [13:57:11.305] lz4 \ [13:57:11.305] meson

Re: macOS prefetching support

2024-08-18 Thread Peter Eisentraut
On 17.08.24 00:01, Thomas Munro wrote: On Sat, Aug 17, 2024 at 6:58 AM Peter Eisentraut wrote: What to do about the order of the symbols and include files. I threw something into src/include/port/darwin.h, but I'm not sure if that's good. Alternatively, we could not use __darwin__ but instead

CI cpluspluscheck failures

2024-08-18 Thread Thomas Munro
Hi, The CompilerWarnings cspluspluscheck started failing recently. 1. LLVM library version issue. See commit message for details. 2. pg_verify_backup.h now uses simplehash.h, which references pg_fatal(), which nobody declared. I'm not sure why the first one started happening at the commit (aa

Re: Remove dependence on integer wrapping

2024-08-18 Thread Alexander Lakhin
18.08.2024 00:52, Joseph Koshakow wrote: The largest possible (theoretical) value for `nbuckets` is `1073741824`, the largest power of 2 that fits into an `int`. So, the largest possible value for `nbuckets << 1` is `2147483648`. This can fully fit in a `uint32`, so the simple fix for this case i

RE: Conflict detection and logging in logical replication

2024-08-18 Thread Zhijie Hou (Fujitsu)
On Friday, August 16, 2024 5:25 PM shveta malik wrote: > > On Fri, Aug 16, 2024 at 12:19 PM Amit Kapila > wrote: > > > > On Fri, Aug 16, 2024 at 11:48 AM shveta malik > wrote: > > > > > > On Fri, Aug 16, 2024 at 10:46 AM shveta malik > wrote: > > > > > > > > 3) > > > > For update_exists(), we

RE: Conflict detection and logging in logical replication

2024-08-18 Thread Zhijie Hou (Fujitsu)
On Friday, August 16, 2024 2:49 PM Amit Kapila wrote: > > > > > > > > One more comment: > > > > 5) > > For insert/update_exists, the sequence is: > > Key .. ; existing local tuple .. ; remote tuple ... > > > > For rest of the conflicts, sequence is: > > Existing local tuple

RE: Conflict detection and logging in logical replication

2024-08-18 Thread Zhijie Hou (Fujitsu)
On Friday, August 16, 2024 2:31 PM Amit Kapila wrote: > > On Fri, Aug 16, 2024 at 10:46 AM shveta malik > wrote: > > > > On Thu, Aug 15, 2024 at 12:47 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Thanks. I have checked and merged the changes. Here is the V15 patch > > > which addressed above

Re: Refactoring postmaster's code to cleanup after child exit

2024-08-18 Thread Alexander Lakhin
Hello Heikki, 10.08.2024 00:13, Heikki Linnakangas wrote: Committed the patches up to and including this one, with tiny comment changes. I've noticed that on the current HEAD server.log contains binary data (an invalid process name) after a child crash. For example, while playing with -ftapv,