SQL function to access to `creating_extension`

2025-04-18 Thread Yurii Rashkovskii
Hi, I propose to introduce `pg_creating_extension()` function that would return the OID of the extension being currently created (or `null` if none is). The core motivation for it is to be able to establish data provenance in tables created by extensions to be used in `pg_extension_config_dump` c

Re: disabled SSL log_like tests

2025-04-18 Thread Tom Lane
I wrote: > What I think happened here is that a previous backend hadn't exited > yet when we start the test, and when its report does come out, > connect_fails prematurely stops waiting. (In the above, evidently > the child process we want to wait for is 599, but we're fooled by > a delayed report

Re: disabled SSL log_like tests

2025-04-18 Thread Tom Lane
I wrote: > * I was more than slightly surprised to find that there are a bunch of > other connect_fails callers that are testing log_like or log_unlike > and thereby risking the same type of race condition. Some of those > tests are relatively new and perhaps just haven't failed *yet*, > but I won

Re: disabled SSL log_like tests

2025-04-18 Thread Jacob Champion
On Fri, Apr 18, 2025 at 12:46 PM Tom Lane wrote: > * The commented-out tests in 001_ssltests.pl contained hard-wired > expectations as to certificate serial numbers, which are obsolete now. > I just replaced them with "\d+", but if anyone feels like that's not > good enough, we could continue to c

Re: disabled SSL log_like tests

2025-04-18 Thread Tom Lane
Andrew Dunstan writes: > On 2025-04-17 Th 10:56 AM, Tom Lane wrote: >> However, I wonder whether Andres' work at 8b886a4e3 could be used >> to improve this, either directly or as inspiration? > I don't think so - these tests are about checking log file contents, not > a psql problem. Well, I wa

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-18 Thread Sami Imseih
> I can see it being confusing. What about this wording to make it more > clear when the field is > updated? here are both of the changes in v4. -- Sami Imseih Amazon Web Services (AWS) v4-0001-Clarify-when-aborted-rows-are-tracked-for-tuple-r.patch Description: Binary data

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-18 Thread Sami Imseih
On Thu, Apr 17, 2025 at 11:13 PM David Rowley wrote: > > On Sat, 12 Apr 2025 at 07:33, Sami Imseih wrote: > > What do you think of the attached? > > I looked at the v3 patch and I'm having trouble getting excited about it. > > I'd say this part is misleading: > > @@ -3956,7 +3961,8 @@ description

Re: [BUG] temporary file usage report with extended protocol and unnamed portals

2025-04-18 Thread Sami Imseih
> [...] LOG: temporary file: path "base/pgsql_tmp/pgsql_tmp525566.0", > size 2416640 > [..] STATEMENT: SELECT 1 > but it should be: > [...] LOG: temporary file: path "base/pgsql_tmp/pgsql_tmp538230.0", > size 2416640 > [...] STATEMENT: SELECT * FROM foo ORDER BY a OFFSET $1 LIMIT 2 hmm, loo

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-18 Thread Masahiko Sawada
On Tue, Apr 8, 2025 at 10:14 PM Zhijie Hou (Fujitsu) wrote: > > > -- > Approach 2 > -- > > Instead of disallowing the use of two-phase and failover together, a more > flexible strategy could be only restrict failover for slots with two-phase > enabled when there's a possibility of

Re: pending_since assertion failure on skink

2025-04-18 Thread Bertrand Drouvot
Hi, On Sat, Apr 12, 2025 at 10:33:33AM -0400, Andres Freund wrote: > I suspect that this is related to > > commit 039549d70f6aa2daa3714a13752a08fa8ca2fb05 +1 > We might just have to give up on that assertion, I guess? Probably. I'll have a look at it next week. Regards, -- Bertrand Drouvot

Re: NUMA shared memory interleaving

2025-04-18 Thread Bertrand Drouvot
Hi, On Thu, Apr 17, 2025 at 01:58:44AM +1200, Thomas Munro wrote: > On Wed, Apr 16, 2025 at 9:14 PM Jakub Wartak > wrote: > > 2. Should we also interleave DSA/DSM for Parallel Query? (I'm not an > > expert on DSA/DSM at all) > > I have no answers but I have speculated for years about a very > sp

Re: NUMA shared memory interleaving

2025-04-18 Thread Bertrand Drouvot
Hi, On Wed, Apr 16, 2025 at 10:05:04AM -0400, Robert Haas wrote: > On Wed, Apr 16, 2025 at 5:14 AM Jakub Wartak > wrote: > > Normal pgbench workloads tend to be not affected, as each backend > > tends to touch just a small partition of shm (thanks to BAS > > strategies). Some remaining questions

Re: ALTER COLUMN SET DATA TYPE does not change the generation expression's collation

2025-04-18 Thread Tom Lane
jian he writes: >> ATPrepAlterColumnType forbids us to ALTER COLUMN SET DATA TYPE USING (expr) >> for generated columns. >> however we can still change the generated column type from non-text to text >> or text type from one collation to another collation. I don't really understand why we allow S

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-18 Thread Jacob Champion
On Tue, Apr 15, 2025 at 2:38 PM Jelte Fennema-Nio wrote: > libpq_append_conn_error(conn, "no custom OAuth flows are available, > and libpq-oauth could not be loaded library could not be loaded. Try > installing the libpq-oauth package from the same source that you > installed libpq from"); Thanks

Re: Changing shared_buffers without restart

2025-04-18 Thread Thomas Munro
On Thu, Nov 21, 2024 at 8:55 PM Peter Eisentraut wrote: > On 19.11.24 14:29, Dmitry Dolgov wrote: > >> I see that memfd_create() has a MFD_HUGETLB flag. It's not very clear how > >> that interacts with the MAP_HUGETLB flag for mmap(). Do you need to > >> specify > >> both of them if you want hu

Re: Changing shared_buffers without restart

2025-04-18 Thread Konstantin Knizhnik
On 25/02/2025 11:52 am, Dmitry Dolgov wrote: On Fri, Oct 18, 2024 at 09:21:19PM GMT, Dmitry Dolgov wrote: TL;DR A PoC for changing shared_buffers without PostgreSQL restart, via changing shared memory mapping layout. Any feedback is appreciated. Hi Dmitry, I am sorry that I have not participa

Re: Changing shared_buffers without restart

2025-04-18 Thread Ni Ku
Hi Ashutosh / Dmitry, Thanks for the information and discussions, it's been very helpful. I also have a related question about how ftruncate() is used in the patch. In my testing I also see that when using ftruncate to shrink a shared segment, the memory is freed immediately after the call, even

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-18 Thread Jacob Champion
On Thu, Apr 17, 2025 at 5:47 PM Jacob Champion wrote: > With those, I have no more TODOs and I believe this is ready for a > final review round. Some ABI self-review. These references to conn->errorMessage also need the indirection treatment, which I'm working on now: > if (actx->errctx) >

Re: Missing comma in libpq.sgml

2025-04-18 Thread Daniel Gustafsson
> On 17 Apr 2025, at 13:30, Tatsuo Ishii wrote: > > In the "Asynchronous Command Processing" section of libpq.sgml, > there's a line which misses a comma at the end. Attached patch should > fix it. I am going to push attached patch into master and > REL_17_STABLE branches if there's no objection

Re: jsonapi: scary new warnings with LTO enabled

2025-04-18 Thread Tom Lane
Jacob Champion writes: > On Wed, Apr 16, 2025 at 4:04 PM Tom Lane wrote: >> Looking through all of the callers of freeJsonLexContext, quite >> a lot of them use local JsonLexContext structs, and probably some >> of them are more performance-critical than these. So that raises >> the question of

Re: jsonapi: scary new warnings with LTO enabled

2025-04-18 Thread Jacob Champion
On Thu, Apr 17, 2025 at 8:20 AM Tom Lane wrote: > I confirm this silences those warnings on my Fedora 41 box. Instead of doing lex = calloc(...); /* (error out on NULL return) */ makeJsonLexContextCstringLen(lex, ...); we need to do lex = makeJsonLexContextCstringLen(NULL, ...)

SQL functions: avoid making a tuplestore unnecessarily

2025-04-18 Thread Tom Lane
(I'm not proposing this for v18, but I wanted to get the patch written while functions.c is still fresh in mind.) The attached patch changes functions.c so that we only make a tuplestore object if we're actually going to return multiple rows in it, that is if it's a set-returning function and we c

Re: Built-in Raft replication

2025-04-18 Thread Alastair Turner
Hi Konstantin On Wed, 16 Apr 2025 at 15:07, Konstantin Osipov wrote: > * Alastair Turner [25/04/16 15:58]: > > > > > If you use build-in failover you have to resort to 3 big Postgres > > > machines because you need 2/3 majority. Of course, you can install > > > MySQL-stype arbiter - host that h

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-04-18 Thread Robert Haas
On Tue, Apr 15, 2025 at 2:13 PM Nikhil Kumar Veldanda wrote: > Addressing Compressed Datum Leaks problem (via CTAS, INSERT INTO ... SELECT > ...) > > As compressed datums can be copied to other unrelated tables via CTAS, > INSERT INTO ... SELECT, or CREATE TABLE ... EXECUTE, I’ve introduced a > m

Re: magical eref alias names

2025-04-18 Thread Robert Haas
On Fri, Jan 3, 2025 at 8:44 AM Robert Haas wrote: > > Actually, I noticed that we are failing to honor that in the places > > where we inject "*SELECT*" and "*SELECT* %d" names, because that > > code puts those names into RTE->alias not only RTE->eref. > > I experimented with the attached patch to

Re: ALTER COLUMN SET DATA TYPE does not change the generation expression's collation

2025-04-18 Thread jian he
On Wed, Mar 26, 2025 at 1:01 PM jian he wrote: > > hi. > > ATPrepAlterColumnType forbids us to ALTER COLUMN SET DATA TYPE USING (expr) > for generated columns. > however we can still change the generated column type from non-text to text > or text type from one collation to another collation. > >

Re: [BUG] temporary file usage report with extended protocol and unnamed portals

2025-04-18 Thread Frédéric Yhuel
On 4/18/25 10:49, Frédéric Yhuel wrote: Hi, It seems there's a bug in the logging of temporary file usage when the extended protocol is used with unnamed portals. FWIW, the attached patch seems to fix the problem. Best regards, FrédéricFrom afb228f07f847c467ba05dbe204861e7be2ffc32 Mon Se

Re: Changing shared_buffers without restart

2025-04-18 Thread Andres Freund
Hi, On April 18, 2025 11:17:21 AM GMT+02:00, Thomas Munro wrote: > Doesn't that achieve the goal with fewer steps, using only >portable* POSIX stuff, and keeping all pointers stable? I understand >that pointer stability may not be required (I can see roughly how that >argument is constructed),

Re: Changing shared_buffers without restart

2025-04-18 Thread Thomas Munro
On Fri, Apr 18, 2025 at 9:17 PM Thomas Munro wrote: > On Fri, Apr 18, 2025 at 7:25 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Thanks for sharing. I need to do more thorough tests, but after a quick > > look I'm not sure about that. ftruncate will take care about the memory, > > but AFAICT

RE: Parallel heap vacuum

2025-04-18 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, Thanks for updating the patch. I have been reviewing and below are comments for now. 01. Sorry if I forgot something, but is there a reason that parallel_vacuum_compute_workers() is mandatory? My fresh eye felt that the function can check and regards zero if the API is NULL.

Re: Align memory context level numbering in pg_log_backend_memory_contexts()

2025-04-18 Thread Fujii Masao
On 2025/04/18 18:23, David Rowley wrote: On Fri, 18 Apr 2025 at 20:54, Fujii Masao wrote: Shouldn't the example output of pg_log_backend_memory_contexts() in the documentation also be updated to use 1-based numbering for consistency? Patch attached. Yeah. I failed to notice we had an examp

Re: Make prep_status() message translatable

2025-04-18 Thread Fujii Masao
On 2025/04/17 13:12, Fujii Masao wrote: I've updated the patch to reflect this comment. Barring any objections, I'm thinking to commit this patch. Pushed. Thanks! Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Changing shared_buffers without restart

2025-04-18 Thread Thomas Munro
On Fri, Apr 18, 2025 at 7:25 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Thu, Apr 17, 2025 at 03:22:28PM GMT, Ashutosh Bapat wrote: > > > > In an offlist chat Thomas Munro mentioned that just ftruncate() would > > be enough to resize the shared memory without touching address maps > > us

Re: Align memory context level numbering in pg_log_backend_memory_contexts()

2025-04-18 Thread David Rowley
On Fri, 18 Apr 2025 at 20:54, Fujii Masao wrote: > Shouldn't the example output of pg_log_backend_memory_contexts() in > the documentation also be updated to use 1-based numbering for consistency? > Patch attached. Yeah. I failed to notice we had an example of the output. Want to take care of it

Re: Align memory context level numbering in pg_log_backend_memory_contexts()

2025-04-18 Thread Fujii Masao
On 2025/04/18 6:11, David Rowley wrote: On Fri, 18 Apr 2025 at 00:25, Rahila Syed wrote: Regarding v2 patch, - int level = 0; Retaining the level variable will enhance the code readability, IMO. When I read that, I suspected it might have been left

[BUG] temporary file usage report with extended protocol and unnamed portals

2025-04-18 Thread Frédéric Yhuel
Hi, It seems there's a bug in the logging of temporary file usage when the extended protocol is used with unnamed portals. For example, with the attached Java / pgJDBC programs, we get the following logs: [...] LOG: temporary file: path "base/pgsql_tmp/pgsql_tmp525566.0", size 2416640 [

Re: Align memory context level numbering in pg_log_backend_memory_contexts()

2025-04-18 Thread David Rowley
On Fri, 18 Apr 2025 at 00:25, Rahila Syed wrote: > Regarding v2 patch, > - int level = 0; > > Retaining the level variable will enhance the code readability, IMO. When I read that, I suspected it might have been leftover from a refactor during the develop

Re: using index to speedup add not null constraints to a table

2025-04-18 Thread jian he
On Wed, Feb 5, 2025 at 4:24 PM jian he wrote: > > rebased new patch attached. > I also did some cosmetic changes. comments refined. > make sure using index_scan mechanism to fast check column not-null can > only be used via btree index. > isolation tests are simplified. I realized that my previou

Re: Changing shared_buffers without restart

2025-04-18 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 02:21:07PM GMT, Konstantin Knizhnik wrote: > > 1. Performance of Postgres CLOCK page eviction algorithm depends on number > of shared buffers. My first native attempt just to mark unused buffers as > invalid cause significant degrade of performance Thanks for sharing! Ri

Re: Add estimated hit ratio to Memoize in EXPLAIN to explain cost adjustment

2025-04-18 Thread David Rowley
On Wed, 16 Apr 2025 at 10:09, Ilia Evdokimov wrote: > I've prepared the updated patches as discussed, including the addition > of estimated lookups in the EXPLAIN output for Memoize. Please let me > know if you have any feedback or further suggestions. While this is fresh, as I might forget befor

Re: Fixup some appendPQExpBuffer() calls

2025-04-18 Thread David Rowley
On Thu, 17 Apr 2025 at 19:50, Daniel Gustafsson wrote: > > On 17 Apr 2025, at 01:44, David Rowley wrote: > > 1) Commit the attached to master > > 2) Do nothing. > > > > I'd like to do #1. > > I vote for #1 as well. Thanks for the judgment sense check. I suspect anyone who thought #2 would have

Re: Changing shared_buffers without restart

2025-04-18 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 03:22:28PM GMT, Ashutosh Bapat wrote: > > In an offlist chat Thomas Munro mentioned that just ftruncate() would > be enough to resize the shared memory without touching address maps > using mmap and munmap(). > > ftruncate man page seems to concur with him > >If th

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-18 Thread David Rowley
On Sat, 12 Apr 2025 at 07:33, Sami Imseih wrote: > What do you think of the attached? I looked at the v3 patch and I'm having trouble getting excited about it. I'd say this part is misleading: @@ -3956,7 +3961,8 @@ description | Waiting for a newly initialized WAL file to reach durable storage

Re: Changing shared_buffers without restart

2025-04-18 Thread Konstantin Knizhnik
On 18/04/2025 12:26 am, Dmitry Dolgov wrote: On Thu, Apr 17, 2025 at 02:21:07PM GMT, Konstantin Knizhnik wrote: 1. Performance of Postgres CLOCK page eviction algorithm depends on number of shared buffers. My first native attempt just to mark unused buffers as invalid cause significant degrade