Re: Incorrect CHUNKHDRSZ in nodeAgg.c

2025-01-08 Thread Jeff Davis
On Mon, 2025-01-06 at 12:34 -0800, Jeff Davis wrote: > If we separate out 4, we can use the Bump allocator for 2 & 3. Attached POC patch, which reduces memory usage by ~15% for a simple distinct query on an integer key. Performance is the same or perhaps a hair faster. It's not many lines of code

Re: New GUC autovacuum_max_threshold ?

2025-01-08 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 02:48:10PM +0100, Frédéric Yhuel wrote: > For what it's worth, although I would have preferred the sub-linear growth > thing, I'd much rather have this than nothing. +1, this is how I feel, too. But I also don't want to add something that folks won't find useful. > And I

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-08 Thread Tom Lane
Guillaume Lelarge writes: > I tend to agree it might be better than Benoit's patch on the index > messages, though I'm afraid that DEBUG1 level won't work for many users. > DEBUGx are for developers, not users. A better log level for me would be > LOG. Please, no. That would result in spamming t

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-08 Thread Jacob Champion
On Wed, Jan 8, 2025 at 11:37 AM Peter Eisentraut wrote: > I don't know what you mean by > "accept in the code". I would agree with "tolerate some inconsistency" > in the code but not with, like, create alias names for all the interface > names. "Tolerate inconsistency" was what I had in mind. So

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-01-08 Thread Tom Lane
Abhishek Chanda writes: > Attached is a patch to add support for logging secrets used in TLS > connection after psql is initialized. This adds a new env var > SSLKEYLOGFILE on the client side that points to a text file where keys > will be logged. I wonder if this poses a security risk, ie someth

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-08 Thread Melanie Plageman
On Wed, Jan 8, 2025 at 6:35 AM Daniel Gustafsson wrote: > > > On 7 Jan 2025, at 21:14, Tom Lane wrote: > > > I might be wrong, but I had the idea that our docs website has a > > capability to provide such redirects. You'd probably need to ask > > about that on the pgsql-www list, unless somebody

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-08 Thread Andres Freund
Hi, On 2025-01-08 15:10:03 +0200, Heikki Linnakangas wrote: > On 08/01/2025 04:10, Andres Freund wrote: > > I instead opted to somewhat rearrange the shutdown sequence: > > - I don't love the naming of the various PMState values (existing and new), > >but a larger renaming should probably be d

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-08 Thread Peter Eisentraut
On 07.01.25 23:24, Jacob Champion wrote: On Thu, Jan 2, 2025 at 2:11 AM Peter Eisentraut wrote: There is a mix of using "URL" and "URI" throughout the patch. I tried to look up in the source material (RFCs) what the correct use would be, but even they are mixing it in nonobvious ways. Maybe t

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-08 Thread Sami Imseih
> I think DEBUG is perfectly appropriate for this sort of message. I am curious what are the thoughts on introducing a CREATE INDEX VERBOSE which can provide this info? similar to users scripting VACUUM VERBOSE to log more details about the vacuum operation including parallel usage. I have not co

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-08 Thread Guillaume Lelarge
Le mer. 8 janv. 2025 à 21:44, Sami Imseih a écrit : > > I think DEBUG is perfectly appropriate for this sort of message. > > I am curious what are the thoughts on introducing a > CREATE INDEX VERBOSE which can provide this info? > similar to users scripting VACUUM VERBOSE to log > more details ab

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-08 Thread Guillaume Lelarge
Le mer. 8 janv. 2025 à 21:35, Tom Lane a écrit : > Guillaume Lelarge writes: > > I tend to agree it might be better than Benoit's patch on the index > > messages, though I'm afraid that DEBUG1 level won't work for many users. > > DEBUGx are for developers, not users. A better log level for me wo

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-08 Thread Andres Freund
Hi, On 2025-01-08 14:48:21 +, Bertrand Drouvot wrote: > On Wed, Jan 08, 2025 at 03:10:03PM +0200, Heikki Linnakangas wrote: > > On 08/01/2025 04:10, Andres Freund wrote: > > > 0002: Make logging of postmaster signalling child processes more > > > consistent > > > > > >I found it somew

Re: Sample rate added to pg_stat_statements

2025-01-08 Thread Ilia Evdokimov
On 07.01.2025 22:29, Sami Imseih wrote: You are right. This is absolutely unexpected for users. Thank you for the review. To fix this, I suggest storing a random number in the [0, 1) range in a separate variable, which will be used for comparisons in any place. We will compare 'sample_rate' and

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-08 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 03:23:45PM -0500, Tom Lane wrote: > I just acquired an M4 Pro, which may also be too small to show any > effect, but perhaps running the test there would at least give us > more confidence that there's not a bad effect. Which test case(s) > would you recommend trying? Than

Re: AIO v2.2

2025-01-08 Thread Andres Freund
Hi, On 2025-01-07 14:59:58 -0500, Robert Haas wrote: > On Tue, Jan 7, 2025 at 11:11 AM Andres Freund wrote: > > The difference between a handle and a reference is useful right now, to have > > some separation between the functions that can be called by anyone (taking a > > PgAioHandleRef) and onl

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-08 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 03:07:24PM -0500, Andres Freund wrote: > Out of curiosity, have you measured whether this has a positive effect without > pg_stat_statements? I think it'll e.g. also affect lwlocks, as they also use > perform_spin_delay(). AFAICT TAS_SPIN() is only used for s_lock(), which

Re: Parameter NOT NULL to CREATE DOMAIN not the same as CHECK (VALUE IS NOT NULL)

2025-01-08 Thread Bruce Momjian
On Wed, Jan 8, 2025 at 04:24:34PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > I think this needs some serious research. > > We've discussed this topic before. The spec's definition of IS [NOT] > NULL for composite values is bizarre to say the least. I think > there's been an intentional

Re: Add CASEFOLD() function.

2025-01-08 Thread Jeff Davis
On Thu, 2024-12-19 at 09:51 -0800, Jeff Davis wrote: > But there's a problem: full case folding doesn't preserve the normal > form, so even if the input is NFC normalized, the output might not > be. > If we solve this problem, then we can just say that CASEFOLD() > preserves the normal form, consis

Adding support for SSLKEYLOGFILE in the frontend

2025-01-08 Thread Abhishek Chanda
Hello folks, Attached is a patch to add support for logging secrets used in TLS connection after psql is initialized. This adds a new env var SSLKEYLOGFILE on the client side that points to a text file where keys will be logged. If a user runs psql multiple times with the same SSLKEYLOGFILE, new e

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-08 Thread Sami Imseih
> Of course, a patch for that would be a few orders of magnitude > larger than what you've got here :-(. But if you're looking > for a framework for reporting these sorts of details, I'd > much rather go in that direction than follow the model of > VACUUM VERBOSE. VACUUM VERBOSE is a kluge with l

RE: Conflict detection for update_deleted in logical replication

2025-01-08 Thread Zhijie Hou (Fujitsu)
On Thursday, January 9, 2025 9:48 AM Masahiko Sawada wrote: Hi, > > On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada > wrote: > > > > Hi, > > > > > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila > > > wrote: > > > > On We

Re: SCRAM pass-through authentication for postgres_fdw

2025-01-08 Thread Peter Eisentraut
This patch is surprisingly compact and straightforward for providing such complex functionality. I have one major code comment that needs addressing: In src/interfaces/libpq/fe-auth-scram.c, there is: + memcpy(ClientKey, state->conn->scram_client_key_binary, SCRAM_MAX_KEY_LEN); Here y

RE: Conflict detection for update_deleted in logical replication

2025-01-08 Thread Zhijie Hou (Fujitsu)
On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada wrote: Hi, > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila > wrote: > > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada > wrote: > > > > > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond > wrote: > > > > > > > > > > > > [3] Test with pgbench r

Re: Conflict detection for update_deleted in logical replication

2025-01-08 Thread vignesh C
On Tue, 7 Jan 2025 at 18:04, Zhijie Hou (Fujitsu) wrote: > > On Friday, January 3, 2025 1:53 PM Amit Kapila > wrote: > > > > On Wed, Dec 25, 2024 at 8:13 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Attach the new version patch set which addressed all other comments. > > > > > > > Some more m

Re: Support for NO INHERIT to INHERIT state change with named NOT NULL constraints

2025-01-08 Thread Alvaro Herrera
On 2024-Nov-25, Robert Haas wrote: > In a simple implementation of ALTER TABLE this would be true, but I > don't see why it should need to be true in ours. It should be possible > to notice that there's an existing NOT NULL constraint and use that as > evidence that the new one can be added withou

pg_settings.unit and DefineCustomXXXVariable

2025-01-08 Thread Luca Ferrari
Hi all, I need to define a few GUCs, and for that purpose I'm using DefineCustomXXXVariable functions that provide hooks for assignment, show and check. However it is not clear to me if it is possible to populate the unit column in pg_settings for the custom defined variables, and if so, how. Any s

Re: IANA timezone abbreviations versus timezone_abbreviations

2025-01-08 Thread Aleksander Alekseev
Hi Tom, > This mess was brought up in pgsql-bugs [1], but the solution > I propose here is invasive enough that I think it needs > discussion on -hackers. > > [...] I tested and reviewed the patch. It fixes the originally reported bug and looks good to me. > The only other way I can envision to

Re: Conflict detection for update_deleted in logical replication

2025-01-08 Thread Amit Kapila
On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote: > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond wrote: > > > > Here is further performance test analysis with v16 patch-set. > > > > > > In the test scenarios already shared on -hackers [1], where pgbench was run > > only on the publisher no

Re: pure parsers and reentrant scanners

2025-01-08 Thread Peter Eisentraut
On 27.12.24 10:19, Heikki Linnakangas wrote: On 26/12/2024 20:27, Peter Eisentraut wrote: On 22.12.24 22:43, Andreas Karlsson wrote: On 12/19/24 9:57 PM, Peter Eisentraut wrote: Here is an updated patch set on top of what has been committed so far, with all the issues you pointed out addressed

Re: Adding NetBSD and OpenBSD to Postgres CI

2025-01-08 Thread Nazir Bilal Yavuz
Hi, Thanks for the review! On Tue, 17 Dec 2024 at 19:21, Andres Freund wrote: > > Hi, > > On 2024-11-12 11:38:11 +0300, Nazir Bilal Yavuz wrote: > > On Fri, 1 Nov 2024 at 21:44, Andres Freund wrote: > > > > +CCACHE_DIR: /tmp/ccache_dir > > > > + > > > > +PATH: /usr/sbin:$PATH > > > > +

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-08 Thread Daniel Gustafsson
> On 7 Jan 2025, at 21:14, Tom Lane wrote: > I might be wrong, but I had the idea that our docs website has a > capability to provide such redirects. You'd probably need to ask > about that on the pgsql-www list, unless somebody who knows the > answer notices this thread. There is functionality

Re: Coccinelle for PostgreSQL development [1/N]: coccicheck.py

2025-01-08 Thread Mats Kindahl
On Wed, Jan 8, 2025 at 11:42 AM Andrew Dunstan wrote: > > On 2025-01-07 Tu 2:44 PM, Mats Kindahl wrote: > > I got some time over during the holidays, so I spent some of it > > doing something I've been thinking about for a while. > > > > For those of you that are not aware of it: Coccinelle is a

ecpg command does not warn COPY ... FROM STDIN;

2025-01-08 Thread Ryo Kanbayashi
Hi hackers, I found a code validation bug in master branch. Now, ecpg does not support 'EXEC SQL COPY ... FROM STDIN ... ;' and code for warning it exits. https://github.com/postgres/postgres/blob/7b27f5fd36cb3270e8ac25aefd73b552663d1392/src/interfaces/ecpg/preproc/ecpg.addons#L242-L245 --- ECPG

Re: Virtual generated columns

2025-01-08 Thread Vik Fearing
On 08/01/2025 20:19, Marcos Pegoraro wrote: Em qua., 8 de jan. de 2025 às 13:14, Peter Eisentraut escreveu: Here is a new patch version where I have gathered various pieces of feedback and improvement suggestions that are scattered over this thread.  I hope I got them all.  I wil

Re: AIO v2.2

2025-01-08 Thread Andres Freund
Hi, On 2025-01-07 22:09:56 +0200, Heikki Linnakangas wrote: > On 07/01/2025 18:11, Andres Freund wrote: > > > I didn't quite understand the point of the prepare callbacks. For example, > > > when AsyncReadBuffers() calls smgrstartreadv(), the > > > shared_buffer_readv_prepare() callback will be ca

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-08 Thread Tom Lane
Nathan Bossart writes: > AFAICT TAS_SPIN() is only used for s_lock(), which doesn't appear to be > used by LWLocks. But I did retry my test from upthread without > pg_stat_statements and was surprised to find a reproducible 4-6% > regression. On what hardware? I just spent an hour beating on my

Re: Parameter NOT NULL to CREATE DOMAIN not the same as CHECK (VALUE IS NOT NULL)

2025-01-08 Thread Bruce Momjian
On Fri, Jan 3, 2025 at 01:39:44PM +, PG Doc comments form wrote: > The following documentation comment has been logged on the website: > > Page: https://www.postgresql.org/docs/17/sql-createdomain.html > Description: > > The manual claims: > > The syntax NOT NULL in this command is a Postgr

Re: Coccinelle for PostgreSQL development [4/N]: correcting palloc() use

2025-01-08 Thread Peter Eisentraut
On 07.01.25 20:49, Mats Kindahl wrote: This is the first example semantic patch and shows how to capture and fix a common problem. If you use an palloc() to allocate memory for an object (or an array of objects) and by mistake type something like:     StringInfoData *info = palloc(sizeof(St

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-01-08 Thread Euler Taveira
On Thu, Aug 15, 2024, at 5:53 PM, Doruk Yilmaz wrote: > Hello again, > > On Tue, Aug 13, 2024 at 12:48 AM Euler Taveira wrote: > > I'm curious about your use case. Is it just because the internal function > > has a > > different signature or your tool is capable of apply logical replication > >

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-08 Thread Andres Freund
Hi, On 2025-01-08 16:01:19 -0600, Nathan Bossart wrote: > On Wed, Jan 08, 2025 at 03:07:24PM -0500, Andres Freund wrote: > > Out of curiosity, have you measured whether this has a positive effect > > without > > pg_stat_statements? I think it'll e.g. also affect lwlocks, as they also use > > perf

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-08 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 05:25:24PM -0500, Andres Freund wrote: > On 2025-01-08 16:01:19 -0600, Nathan Bossart wrote: >> But I did retry my test from upthread without pg_stat_statements and was >> surprised to find a reproducible 4-6% regression. > > Uh, huh. I assume this was readonly pgbench with

Re: Sample rate added to pg_stat_statements

2025-01-08 Thread Sami Imseih
> Unfortunately, these changes do not achieve the intended sampling goal. > I looked into this more deeply: while the sampled-out queries do not > appear in pg_stat_statements, an entry is still allocated in the hash > table after normalization, which, in my view, should not happen when > sampling

<    1   2