RE: BUG #18815: Logical replication worker Segmentation fault

2025-04-07 Thread Zhijie Hou (Fujitsu)
On Tue, Apr 8, 2025 at 2:00 PM Amit Kapila wrote: > > On Mon, Apr 7, 2025 at 3:28 PM Zhijie Hou (Fujitsu) > wrote: > > > > > What is the > > > behavior of conflict reporting code in case of exclusion constraints? > > > > Under logical replication context, since we do not detect conflicts > > for

Re: Add pg_buffercache_evict_all() and pg_buffercache_mark_dirty[_all]() functions

2025-04-07 Thread Andres Freund
Hi, On 2025-04-08 03:15:57 +0300, Nazir Bilal Yavuz wrote: > On Tue, 8 Apr 2025 at 02:29, Andres Freund wrote: > > On 2025-04-07 19:37:50 +0300, Nazir Bilal Yavuz wrote: > > > > > > > + relation_close(rel, AccessExclusiveLock); > > > > > > > > > > > > Hm. Why are we dropping the lock here ear

Re: An incorrect check in get_memoize_path

2025-04-07 Thread Richard Guo
On Mon, Apr 7, 2025 at 9:54 PM Andrei Lepikhov wrote: > On 4/7/25 09:50, Richard Guo wrote: > > Consider the join to t3. It is a unique join, and not all of its > > restriction clauses are parameterized. Despite this, the check still > > passes. > At the same time I think term 'Incorrect' is no

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread David Rowley
On Tue, 8 Apr 2025 at 17:41, Ashutosh Bapat wrote: > Thanks for listing all the patterns. Creating four different iterators > is going to affect functionality and might require duplicate code. But > each of the patterns is using exactly one BMS operation on em_relids > and relids being used as sea

Re: pg16 && GSSAPI && Heimdal/Macos

2025-04-07 Thread Nico Williams
On Tue, Apr 08, 2025 at 02:18:40AM -0400, Tom Lane wrote: > Nico Williams writes: > > Heimdal in the master branch sure does; I'm the author if > > gss_store_cred_into() and gss_store_cred_into2(). Idk when we'll do an > > 8.0 release though. We've run out of steam. > > Yeah, this is what makes

Re: pg16 && GSSAPI && Heimdal/Macos

2025-04-07 Thread Tom Lane
Nico Williams writes: > On Wed, Dec 06, 2023 at 10:57:15PM -0500, Tom Lane wrote: >> The immediate reason for dropping that support is that Heimdal doesn't >> have gss_store_cred_into(), without which we can't support delegated >> credentials. AFAICT, Apple's version doesn't have that either. >

Re: pg16 && GSSAPI && Heimdal/Macos

2025-04-07 Thread Nico Williams
On Wed, Dec 06, 2023 at 10:57:15PM -0500, Tom Lane wrote: > The immediate reason for dropping that support is that Heimdal doesn't > have gss_store_cred_into(), without which we can't support delegated > credentials. AFAICT, Apple's version doesn't have that either. > We could argue about how impo

Re: pg_dump/restore failure (dependency?) on BF serinus

2025-04-07 Thread Andres Freund
Hi, On 2025-04-08 00:11:55 -0400, Tom Lane wrote: > Andres Freund writes: > >> # Running: pg_restore --create -j2 -d postgres > >> /home/bf/bf-build/serinus/HEAD/pgsql.build/testrun/pg_upgrade/002_pg_upgrade/data/tmp_test_KTss/regression.dump > >> pg_restore: error: could not execute query: ERRO

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread Ashutosh Bapat
On Tue, Apr 8, 2025 at 8:30 AM David Rowley wrote: > > On Tue, 8 Apr 2025 at 04:54, Ashutosh Bapat > wrote: > > - foreach(lc2, cur_ec->ec_members) > > + setup_eclass_member_iterator(&it, cur_ec, rel); > > + while ((cur_em = eclass_member_iterator_next(&it)) != NULL) > > { > > - EquivalenceMember

Re: Enhancing Memory Context Statistics Reporting

2025-04-07 Thread Rahila Syed
Hi Daniel, Andres, > > > >> +} MemoryContextState; > > > > > > IMO that's too generic a name for something in a header. > > > > > >> +} MemoryContextId; > > > > > > This too. Particularly because MemoryContextData->ident exist but is > > > something different. > > > > Renamed both to use MemoryC

Re: pg16 && GSSAPI && Heimdal/Macos

2025-04-07 Thread Nico Williams
On Wed, Dec 06, 2023 at 10:57:15PM -0500, Tom Lane wrote: > The immediate reason for dropping that support is that Heimdal doesn't > have gss_store_cred_into(), without which we can't support delegated > credentials. AFAICT, Apple's version doesn't have that either. > We could argue about how impo

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-07 Thread Michael Paquier
On Tue, Apr 08, 2025 at 02:00:35AM +, Hayato Kuroda (Fujitsu) wrote: > Your patch looks good to me and it could pass on my env. PSA patches for PG16. > Patch for PG17 is not changed, just renamed. @@ -1287,6 +1288,17 @@ LogStandbySnapshot(void) Assert(XLogStandbyInfoActive()); +#ifdef

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-07 Thread Michael Paquier
On Mon, Apr 07, 2025 at 05:25:32PM -0400, Tom Lane wrote: > What version are you testing? We did some work in that area in the > v17 cycle (a45c78e32). I am puzzled by the target version used here, as well. If there is more that can be improved, v19 would be the version to consider for future im

Re: AIO v2.5

2025-04-07 Thread Andres Freund
Hi, On 2025-04-06 23:00:00 +0300, Alexander Lakhin wrote: > 02.04.2025 14:58, Andres Freund wrote: > When running multiple installcheck's against a single server (please find > the ready-to-use script attached (I use more sophisticated version with > additional patches to make installcheck pass cl

Re: BAS_BULKREAD vs read stream

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 16:28:20 -0400, Andres Freund wrote: > On 2025-04-07 15:24:43 -0400, Melanie Plageman wrote: > > On Sun, Apr 6, 2025 at 4:15 PM Andres Freund wrote: > > > > > > I think we should consider increasing BAS_BULKREAD TO something like > > > Min(256, io_combine_limit * (effective_io

Re: pg_dump/restore failure (dependency?) on BF serinus

2025-04-07 Thread Tom Lane
Andres Freund writes: > On 2025-04-08 00:11:55 -0400, Tom Lane wrote: >> This feels quite adjacent to my complaint here: >> https://www.postgresql.org/message-id/2045026.1743801143%40sss.pgh.pa.us >> though perhaps it's not exactly the same. > That does sound rather plausible. What an odd coincid

Re: Proposal - Allow extensions to set a Plan Identifier

2025-04-07 Thread Michael Paquier
On Mon, Apr 07, 2025 at 06:11:06PM -0700, Lukas Fittl wrote: > However, since we're basically at feature freeze (and it seems unlikely > this gets into 18), I have a quick alternate proposal: > > What if, for 18, we just make DoJumble exported to be used by plugins? Yes, I am not planning to rush

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread Tom Lane
David Rowley writes: > I certainly could get on board with renaming if there's consensus to > do so. I don't think that's going to happen in the next 9 hours. Is it > worth pushing this out to v19 because we can't agree on the name of a > struct field? I'd be disappointed if we miss this because o

Re: Add pg_buffercache_evict_all() and pg_buffercache_mark_dirty[_all]() functions

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 19:37:50 +0300, Nazir Bilal Yavuz wrote: > > > > > + relation_close(rel, AccessExclusiveLock); > > > > > > > > Hm. Why are we dropping the lock here early? It's probably ok, but it's > > > > not > > > > clear to me why we should do so. > > > > > > We are dropping the lock af

Re: pg_dump/restore failure (dependency?) on BF serinus

2025-04-07 Thread Tom Lane
Andres Freund writes: >> # Running: pg_restore --create -j2 -d postgres >> /home/bf/bf-build/serinus/HEAD/pgsql.build/testrun/pg_upgrade/002_pg_upgrade/data/tmp_test_KTss/regression.dump >> pg_restore: error: could not execute query: ERROR: there is no unique >> constraint matching given keys f

pg_dump/restore failure (dependency?) on BF serinus

2025-04-07 Thread Andres Freund
Hi, https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=serinus&dt=2025-04-07%2017%3A45%3A28&stg=pg_upgrade-check failed in a problem-indicating way in the new dump/restore test that Ashutosh added. > # Running: pg_restore --create -j2 -d postgres > /home/bf/bf-build/serinus/HEAD/pgsql

Re: Draft for basic NUMA observability

2025-04-07 Thread Tomas Vondra
On 4/7/25 17:51, Andres Freund wrote: > Hi, > > On 2025-04-06 13:56:54 +0200, Tomas Vondra wrote: >> On 4/6/25 01:00, Andres Freund wrote: >>> On 2025-04-05 18:29:22 -0400, Andres Freund wrote: I think one thing that the docs should mention is that calling the numa functions/views will f

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread David Rowley
On Tue, 8 Apr 2025 at 04:54, Ashutosh Bapat wrote: > - foreach(lc2, cur_ec->ec_members) > + setup_eclass_member_iterator(&it, cur_ec, rel); > + while ((cur_em = eclass_member_iterator_next(&it)) != NULL) > { > - EquivalenceMember *cur_em = (EquivalenceMember *) lfirst(lc2); > - > /* > * Ignore chi

Horribly slow pg_upgrade performance with many Large Objects

2025-04-07 Thread Hannu Krosing
Hi Hackers ## The issue I have now met a not insignificant number of cases where pg_upgrade performance is really bad when the database has a large number of Large Objects. The average time to `pg_dump --binary-upgrade --format=custom ...` a database and then `pg_restore ...` it back is 1 minute

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-07 Thread Jacob Champion
On Mon, Apr 7, 2025 at 3:26 PM Jacob Champion wrote: > Sounds good. Any opinions from the gallery on what a "libpq plugin > subdirectory" in pkglibdir should be called? ("client", "modules", > "plugins"...?) Hm, one immediate consequence of hardcoding pkglibdir is that we can no longer rely on LD

RE: Fix 035_standby_logical_decoding.pl race conditions

2025-04-07 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > I have changed quite a few comments and commit message for the PG17 > patch in the attached. Can you update PG16 patch based on this and > also use the same commit message as used in attached for all the three > patches? Your patch looks good to me and it could pass on my env. PSA

Move tests of contrib/spi/ out of the core regression tests

2025-04-07 Thread Tom Lane
The attached patch removes test cases concerned with contrib/spi/ from the core regression tests and instead puts them into new test files in contrib/spi/ itself. My original motivation for looking at this was the discussion in [1] about whether to remove contrib/spi/refint.c entirely, since it's

Re: Proposal - Allow extensions to set a Plan Identifier

2025-04-07 Thread Sami Imseih
> What if, for 18, we just make DoJumble exported to be used by plugins? We will need to export InitJumble as well, but DoJumble calls _jumbleNode which only includes queryjumblefuncs.switch.c, so I don't think that will be enough for Plan. -- Sami Imseih Amazon Web Services (AWS)

Re: Parameter NOT NULL to CREATE DOMAIN not the same as CHECK (VALUE IS NOT NULL)

2025-04-07 Thread Bruce Momjian
On Thu, Jan 30, 2025 at 08:26:54AM -0500, Bruce Momjian wrote: > On Wed, Jan 8, 2025 at 04:24:34PM -0500, Tom Lane wrote: > > Bruce Momjian writes: > > > I think this needs some serious research. > > > > We've discussed this topic before. The spec's definition of IS [NOT] > > NULL for composite

Re: Proposal - Allow extensions to set a Plan Identifier

2025-04-07 Thread Lukas Fittl
Whilst rebasing the pg_stat_plans work on top of this, I realized that we should probably make this a conditional instead of an assert - if you are jumbling a tree that contains expressions you'd want to be able to jumble a node that has a location (but don't record it). Attached v4 that modifies i

Re: psql suggestion "select " offers nothing, can we get functions like "\df "

2025-04-07 Thread Greg Sabino Mullane
On Mon, Apr 7, 2025 at 12:55 PM Kirill Reshke wrote: > Maybe we can tab-complete here if prefix matches pg_% ? Does that makes > good use case? > >> I think you will have to get much more specific than pg_ before tab completion makes sense. You are still looking at hundreds of answers. The exampl

Re: speedup COPY TO for partitioned table.

2025-04-07 Thread Kirill Reshke
On Fri, 4 Apr 2025, 15:17 Kirill Reshke, wrote: > Hi! > > First of all, a commit message does not need to contain SQL examples > of what it does. We should provide human-readable explanations and > that's it. > > Next, about changes to src/test/regress/sql/copy2.sql. I find the sql > you used to

Re: Add pg_buffercache_evict_all() and pg_buffercache_mark_dirty[_all]() functions

2025-04-07 Thread Nazir Bilal Yavuz
Hi, On Tue, 8 Apr 2025 at 02:29, Andres Freund wrote: > On 2025-04-07 19:37:50 +0300, Nazir Bilal Yavuz wrote: > > > > > > + relation_close(rel, AccessExclusiveLock); > > > > > > > > > > Hm. Why are we dropping the lock here early? It's probably ok, but > > > > > it's not > > > > > clear to

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-04-07 Thread Rushabh Lathia
On Tue, Apr 8, 2025 at 1:24 AM Alvaro Herrera wrote: > On 2025-Apr-07, jian he wrote: > > > CREATE TABLE t (a int, b int); > > INSERT INTO t VALUES (NULL, 1), (300, 3); > > ALTER TABLE t ADD CONSTRAINT nn NOT NULL a NOT VALID; -- ok > > ALTER TABLE t add column c float8 default random(); > > the

Can we use Statistics Import and Export feature to perforamance testing?

2025-04-07 Thread Ryohei Takahashi (Fujitsu)
Hi, I have a question about Statistics Import and Export. * Background I'm working for PGEcons[1], which is the PostgreSQL consortium in Japan. Several companies participating in PGEcons have the following request for PostgreSQL. They have two environments, production environment and staging

Re: Parallel heap vacuum

2025-04-07 Thread Peter Smith
Hi Sawada-san, Here are some review comments for the patch v16-0002 == Commit message 1. Missing period. /cleanup, ParallelVacuumState/cleanup. ParallelVacuumState/ ~~~ 2. Typo /paralel/parallel/ ~~~ 3. Heap table AM disables the parallel heap vacuuming for now, but an upcoming patch use

Re: [PATCH] Automatic client certificate selection support for libpq v1

2025-04-07 Thread Bruce Momjian
Great, TODO item removed, thanks. --- On Mon, Mar 31, 2025 at 04:29:39PM -0400, Seth Robertson wrote: > > Yes, at first glance the service file looks like it should work and is > a much more elegant and generic method than

Re: prevent 006_transfer_modes.pl from leaving files behind

2025-04-07 Thread Michael Paquier
On Mon, Apr 07, 2025 at 04:45:52PM -0500, Nathan Bossart wrote: > The other pg_upgrade tests chdir to tmp_check prior to running pg_upgrade > to avoid leaving behind delete_old_cluster.{sh,bat}. 006_transfer_modes.pl > should, too. However, this test is a little different because it loops > over

Re: AIO v2.5

2025-04-07 Thread Andres Freund
Hi, On 2025-04-05 06:43:52 -0700, Noah Misch wrote: > Yeah. Maybe this (untested): Something like that works. I adopted your formulation of this, mine was in GetLocalVictimBuffer(), which seems slightly less future proof. > > > If that's right, it would still be nice to reach the right > > >

Re: Enhancing Memory Context Statistics Reporting

2025-04-07 Thread Daniel Gustafsson
> On 7 Apr 2025, at 17:43, Andres Freund wrote: > > Hi, > > On 2025-04-07 15:41:37 +0200, Daniel Gustafsson wrote: >> I think this function can be a valuable debugging aid going forward. > > What I am most excited about for this is to be able to measure server-wide and > fleet-wide memory usage

Re: BAS_BULKREAD vs read stream

2025-04-07 Thread Melanie Plageman
On Sun, Apr 6, 2025 at 4:15 PM Andres Freund wrote: > > I think we should consider increasing BAS_BULKREAD TO something like > Min(256, io_combine_limit * (effective_io_concurrency + 1)) Do you mean Max? If so, this basically makes sense to me. Overall, I think even though the ring is about reu

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-07 Thread Nathan Bossart
On Mon, Apr 07, 2025 at 10:33:47PM +0200, Hannu Krosing wrote: > The obvious solution would be to handle the table > `pg_largeobject_metadata` the same way as we currently handle > `pg_largeobject `by not doing anything with it in `pg_dump > --binary-upgrade` and just handle the contents it like we

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2025-04-07 Thread Masahiko Sawada
On Sat, Apr 5, 2025 at 1:31 AM jian he wrote: > > On Sat, Apr 5, 2025 at 5:33 AM Masahiko Sawada wrote: > > > > On Fri, Apr 4, 2025 at 4:55 AM jian he wrote: > > > > > > On Tue, Mar 25, 2025 at 2:31 PM vignesh C wrote: > > > > > > > > 2) Here in error we say column c1 violates not-null constrai

Re: [PATCH] Re: Proposal to Enable/Disable Index using ALTER INDEX

2025-04-07 Thread Sami Imseih
> Attached v16 with feedback and rebased. Thanks, and I realized the original tab-complete I proposed was not entirely correct. I fixed it and also shortened the commit message. -- Sami Imseih Amazon Web Services (AWS) v17-0001-Introduce-the-ability-to-set-index-visibility-us.patch Description

Re: Draft for basic NUMA observability

2025-04-07 Thread Tomas Vondra
On 4/7/25 23:50, Jakub Wartak wrote: > On Mon, Apr 7, 2025 at 11:27 PM Tomas Vondra wrote: >> >> Hi, >> >> I've pushed all three parts of v29, with some additional corrections >> (picked lower OIDs, bumped catversion, fixed commit messages). > > Hi Tomas, great, awesome! (this is an awesome fe

Re: Draft for basic NUMA observability

2025-04-07 Thread Jakub Wartak
On Mon, Apr 7, 2025 at 11:27 PM Tomas Vondra wrote: > > Hi, > > I've pushed all three parts of v29, with some additional corrections > (picked lower OIDs, bumped catversion, fixed commit messages). Hi Tomas, great, awesome! (this is an awesome feeling)! Thank You for such incredible support on th

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-07 Thread Jacob Champion
On Mon, Apr 7, 2025 at 11:47 AM Christoph Berg wrote: > Mmmmh. Since we are currently only talking about 3 symbols, it doesn't > sound very likely that we'd have to bump this in a major branch. The ABI extends to the pointers we're using, though. This module uses PGconn* internals and libpq-int.h

prevent 006_transfer_modes.pl from leaving files behind

2025-04-07 Thread Nathan Bossart
The other pg_upgrade tests chdir to tmp_check prior to running pg_upgrade to avoid leaving behind delete_old_cluster.{sh,bat}. 006_transfer_modes.pl should, too. However, this test is a little different because it loops over all of the available transfer modes and runs pg_upgrade for each one sup

Re: Draft for basic NUMA observability

2025-04-07 Thread Tomas Vondra
Hi, I've pushed all three parts of v29, with some additional corrections (picked lower OIDs, bumped catversion, fixed commit messages). On 4/7/25 23:01, Jakub Wartak wrote: > On Mon, Apr 7, 2025 at 9:51 PM Tomas Vondra wrote: > >>> So it looks like that the new way to iterate on the buffers tha

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-07 Thread Tom Lane
Hannu Krosing writes: > I have now met a not insignificant number of cases where pg_upgrade > performance is really bad when the database has a large number of > Large Objects. What version are you testing? We did some work in that area in the v17 cycle (a45c78e32). rega

Re: [fixed] Trigger test

2025-04-07 Thread Tom Lane
I wrote: > Dmitrii Bondar writes: >> I considered removing it entirely, but that seemed too drastic a >> solution (and, at the very least, I don't have enough expertise to make >> that decision). > I'm not that thrilled with giving up on refint.c either. But in its > current state, it's a pret

Re: [PATCH] Re: Proposal to Enable/Disable Index using ALTER INDEX

2025-04-07 Thread Shayon Mukherjee
On Mon, Apr 7, 2025 at 4:01 PM Sami Imseih wrote: > Thanks for the update! > > The changes in v15 look good to me. The patch does need to be rebased, > and I also think you should add a tab-complete for CREATE INDEX > > > simseih@bcd07415af92 postgresql % git diff > diff --git a/src/bin/psql/tab-

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread Ashutosh Bapat
On Mon, Apr 7, 2025 at 4:34 AM David Rowley wrote: > > On Sat, 5 Apr 2025 at 16:55, David Rowley wrote: > > I am still thinking about the duplicate members being returned from > > the iterator for child join rels due to them being duplicated into > > each component relid element in ec_childmember

Re: sync_standbys_defined read/write race on startup

2025-04-07 Thread Maksim.Melnikov
Hi everyone, thanks for your comments. I've just wanted to share little cosmetic fixes for previous patch. Best Regards, Maksim MelnikovFrom 748c5fe8efd4c6365f380f7f9bbeba5e91887ab4 Mon Sep 17 00:00:00 2001 From: Maksim Melnikov Date: Wed, 26 Mar 2025 16:44:09 +0300 Subject: [PATCH] Fix sync_sta

Re: BAS_BULKREAD vs read stream

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 15:24:43 -0400, Melanie Plageman wrote: > On Sun, Apr 6, 2025 at 4:15 PM Andres Freund wrote: > > > > I think we should consider increasing BAS_BULKREAD TO something like > > Min(256, io_combine_limit * (effective_io_concurrency + 1)) > > Do you mean Max? If so, this basically

Re: [PATCH] Re: Proposal to Enable/Disable Index using ALTER INDEX

2025-04-07 Thread Sami Imseih
Thanks for the update! The changes in v15 look good to me. The patch does need to be rebased, and I also think you should add a tab-complete for CREATE INDEX simseih@bcd07415af92 postgresql % git diff diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 8e2eb50205e.

[Question: linux pipes ( ... | gzip | cat ... ) in tests failing] Re: [PATCH] Adding pg_dump flag for parallel export to pipes

2025-04-07 Thread Hannu Krosing
Just to bring this out separately : Does anybody have any ideas why pipe commands prematurely close inside tests ? > Re: 003-pg_dump_basic_tests has a few basic validation tests for > correctmflag combinations. We need to write more automated tests in > 002_pg_dump.pl but have been running into

Re: Draft for basic NUMA observability

2025-04-07 Thread Tomas Vondra
On 4/7/25 20:11, Bertrand Drouvot wrote: > Hi, > > On Mon, Apr 07, 2025 at 12:42:21PM -0400, Andres Freund wrote: >> Hi, >> >> On 2025-04-07 18:36:24 +0200, Tomas Vondra wrote: >> >> I was thinking of checking if the BufferDesc indicates BM_VALID or >> BM_TAG_VALID. > > Yeah, that's what I did

Re: Adding pg_dump flag for parallel export to pipes

2025-04-07 Thread Hannu Krosing
Just to bring this out separately : Does anybody have any idea why pipe commands close inside tests ? Re: 003-pg_dump_basic_tests has a few basic validation tests for correctmflag combinations. We need to write more automated tests in 002_pg_dump.pl but have been running into some issues with envi

Re: AIO v2.5

2025-04-07 Thread Alexander Lakhin
Hello Andres, 07.04.2025 19:20, Andres Freund wrote: iteration 8: Sun Apr  6 19:22:39 UTC 2025 installchecks finished: Sun Apr  6 19:23:47 UTC 2025 2025-04-06 19:22:44.216 UTC [349525] LOG:  could not read blocks 0..0 in file "base/6179194/2606": Operation canceled 2025-04-06 19:22:44.216 UTC [

Re: as per commit 643a1a61985bef2590496, move create/open dir code to function using switch case of pg_backup_directory.c file also

2025-04-07 Thread Mahendra Singh Thalor
On Mon, 7 Apr 2025 at 18:52, Andrew Dunstan wrote: > > > On 2025-04-07 Mo 8:25 AM, Mahendra Singh Thalor wrote: > > Hi, > > In commit 643a1a61985bef2590496, we did some cleanup and we replaced > > if-else with switch case. > > Basically, we made a function to open a directory in pg_dumpall. > > In

Re: Periodic FSM vacuum doesn't happen in one-pass strategy vacuum.

2025-04-07 Thread Masahiko Sawada
On Mon, Apr 7, 2025 at 8:30 AM Melanie Plageman wrote: > > On Fri, Apr 4, 2025 at 6:07 PM Masahiko Sawada wrote: > > > > I'm going to push this fix up to HEAD and v17 early next week, unless > > there is no objection. > > I started studying this again looking back at the thread associated > with

Re: [PATCH] clarify palloc comment on quote_literal_cstr

2025-04-07 Thread Steve Chavez
I haven't found a similar style of comment on any other function call. I've attached a new patch using the style you suggest. That being said, I do find the first form much more readable, but I understand this is a debatable subject. Best regards, Steve Chavez On Mon, 7 Apr 2025 at 06:33, Ashut

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-07 Thread Christoph Berg
Re: Jacob Champion > > > +This module ABI is an internal implementation detail, so it's subject to > > > change > > > +without warning, even during minor releases (however unlikely). The > > > compiled > > > +version of libpq-oauth should always match the compiled version of libpq. > > > > Should

Re: Modern SHA2- based password hashes for pgcrypto

2025-04-07 Thread Tom Lane
I wrote: > Bernd Helmle writes: >> Here is a patch that tries to address all these issues (including >> Andres' report). I've adjusted the error message and use ereport(), so >> it might be more useful if we deal with not just single byte letters. > I'd like to get mamba back to green, so I'll ta

Re: Modern SHA2- based password hashes for pgcrypto

2025-04-07 Thread Álvaro Herrera
On Mon, Apr 7, 2025, at 7:50 PM, Tom Lane wrote: > I'd like to get mamba back to green, so I'll take care of this > (unless Alvaro is already on it?) Please feel free, thank you.

Re: Draft for basic NUMA observability

2025-04-07 Thread Bertrand Drouvot
Hi, On Mon, Apr 07, 2025 at 12:42:21PM -0400, Andres Freund wrote: > Hi, > > On 2025-04-07 18:36:24 +0200, Tomas Vondra wrote: > > I was thinking of checking if the BufferDesc indicates BM_VALID or > BM_TAG_VALID. Yeah, that's what I did propose in [1] (when we were speaking about get_mempolic

Re: FmgrInfo allocation patterns (and PL handling as staged programming)

2025-04-07 Thread Chapman Flack
On 04/06/25 13:59, Tom Lane wrote: > polymorphic arguments: the element type of an anyarray argument can > change on-the-fly from one call to the next in the same query. I > think this is only possible when you're fed pg_stats.most_common_vals > or one of its sibling columns, but that's enough to

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-04-07 Thread Alvaro Herrera
On 2025-Apr-07, jian he wrote: > CREATE TABLE t (a int, b int); > INSERT INTO t VALUES (NULL, 1), (300, 3); > ALTER TABLE t ADD CONSTRAINT nn NOT NULL a NOT VALID; -- ok > ALTER TABLE t add column c float8 default random(); > the last query should not fail. Agreed. > if we want more places use C

Re: Draft for basic NUMA observability

2025-04-07 Thread Andres Freund
On 2025-04-04 19:07:12 +0200, Jakub Wartak wrote: > They actually look good to me. We've discussed earlier dropping > s/numa_//g for column names (after all views contain it already) so > they are fine in this regard. > There's also the question of consistency: (bufferid, page_num, > node_id) -- ma

Adding pg_dump flag for parallel export to pipes

2025-04-07 Thread Nitin Motiani
Hi Hackers, We are proposing the ability to specify a pipe command to pg_dump by a flag. And attaching the patch set. Why : Currently it is quite simple to pipe the output of pg_dump for text format to a pipe at command line and do any manipulations necessary. Following is an example : pg

Re: Draft for basic NUMA observability

2025-04-07 Thread Tomas Vondra
On 4/7/25 18:42, Andres Freund wrote: > ... >>> >>> Of course that would mean that we'd always need to >>> pg_numa_touch_mem_if_required(), not just the first time round, because we >>> previously might not have for a page that is now valid. But compared to the >>> cost of actually allocating page

Re: Improve monitoring of shared memory allocations

2025-04-07 Thread Tomas Vondra
Hi, I'm sorry, but I'm afraid this won't make it into PG18 :-( AFAICS the updated patch is correct / not buggy for non-shared hash tables, but considering I missed a pretty serious flaw before pushing the original patch, and that the tests didn't catch that either ... The risk/benefit is not rea

Re: AIX support

2025-04-07 Thread Heikki Linnakangas
On 05/04/2025 21:29, Srirama Kucherlapati wrote:  - WRT to the MEMSET_LOOP_LIMIT flag, this is set to “0”, which would internally use Yes, I understand what it does. But why? Whatever benchmarking was done back in 2006 by is no longer relevant. We ran the program , mentioned in the below lin

Re: speedup COPY TO for partitioned table.

2025-04-07 Thread Kirill Reshke
Sorry, wrong thread Best regards, Kirill Reshke On Mon, 7 Apr 2025, 19:54 Kirill Reshke, wrote: > > On Fri, 4 Apr 2025, 15:17 Kirill Reshke, wrote: > >> Hi! >> >> First of all, a commit message does not need to contain SQL examples >> of what it does. We should provide human-readable explanati

Re: Draft for basic NUMA observability

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 18:36:24 +0200, Tomas Vondra wrote: > > Forcing all those pages to be allocated via pg_numa_touch_mem_if_required() > > itself wouldn't be too bad - in fact I'd rather like to have an explicit way > > of doing that. The problem is that that leads to all those allocations to > >

Re: Enhancing Memory Context Statistics Reporting

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 21:57:57 +0530, Rahila Syed wrote: > > > diff --git a/src/backend/postmaster/auxprocess.c > > b/src/backend/postmaster/auxprocess.c > > > index 4f6795f7265..d3b4df27935 100644 > > > --- a/src/backend/postmaster/auxprocess.c > > > +++ b/src/backend/postmaster/auxprocess.c > > > @@

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-07 Thread Peter Eisentraut
On 07.04.25 16:43, Andres Freund wrote: While I was looking into this I found that Debian's going to use the existence of an SONAME to check other things, which I assume will make Christoph's life harder. I have switched over to 'libpq-oauth-.so', without any SONAME or symlinks. Yes, this is cor

Re: Add pg_buffercache_evict_all() and pg_buffercache_mark_dirty[_all]() functions

2025-04-07 Thread Nazir Bilal Yavuz
Hi, On Mon, 7 Apr 2025 at 16:38, Andres Freund wrote: > > Hi, > > On 2025-04-04 18:36:57 +0300, Nazir Bilal Yavuz wrote: > > v6 is attached. Additional changes prior to v5: > > > > * Tests are added. > > I don't think the mark-dirty stuff is quite ready to commit. Given that, could > you change t

Re: Draft for basic NUMA observability

2025-04-07 Thread Andres Freund
Hi, On 2025-04-06 13:51:34 +0200, Tomas Vondra wrote: > On 4/6/25 00:29, Andres Freund wrote: > >> + > >> + if (firstNumaTouch) > >> + elog(DEBUG1, "NUMA: page-faulting the buffercache for > >> proper NUMA readouts"); > > > > Over the patchseries the related code is dup

Re: Enhancing Memory Context Statistics Reporting

2025-04-07 Thread Rahila Syed
Hi, Please see some responses below. On Mon, Apr 7, 2025 at 9:13 PM Andres Freund wrote: > Hi, > > On 2025-04-07 15:41:37 +0200, Daniel Gustafsson wrote: > > I think this function can be a valuable debugging aid going forward. > > What I am most excited about for this is to be able to measure s

Re: Logging which local address was connected to in log_line_prefix

2025-04-07 Thread Tom Lane
Robert Haas writes: > The only thing that makes me a little bit sad is that we don't seem to > have added this to pg_stat_activity. Hmm, that seems like it'd be a completely separate discussion. My main objection to the idea is that if we do that then everybody will pay the overhead for it, whet

Re: [PATCH] Fix build on MINGW on ARM64

2025-04-07 Thread Andrew Dunstan
On 2025-04-07 Mo 11:05 AM, Andrew Dunstan wrote: On 2025-04-07 Mo 12:58 AM, Tom Lane wrote: Andrew Dunstan writes: On 2025-04-01 Tu 11:15 AM, Andrew Dunstan wrote: On 2025-04-01 Tu 8:47 AM, vignesh C wrote: There is an existing CF entry for this at [1]. If no one picks this till the end o

Re: Draft for basic NUMA observability

2025-04-07 Thread Andres Freund
Hi, On 2025-04-06 13:56:54 +0200, Tomas Vondra wrote: > On 4/6/25 01:00, Andres Freund wrote: > > On 2025-04-05 18:29:22 -0400, Andres Freund wrote: > >> I think one thing that the docs should mention is that calling the numa > >> functions/views will force the pages to be allocated, even if they'

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-07 Thread Tom Lane
David Rowley writes: > I think this is worthy of getting into v18. Does anyone else think > differently? It'd be good to know that soon. v41 passes an eyeball check for me. regards, tom lane

Re: Enhancing Memory Context Statistics Reporting

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 15:41:37 +0200, Daniel Gustafsson wrote: > I think this function can be a valuable debugging aid going forward. What I am most excited about for this is to be able to measure server-wide and fleet-wide memory usage over time. Today I have actually very little idea about what mem

Re: Logging which local address was connected to in log_line_prefix

2025-04-07 Thread Robert Haas
On Sun, Apr 6, 2025 at 8:50 PM Greg Sabino Mullane wrote: > I lean for #1. Yes, there is some inconsistency, but it feels like the right > thing to do, and this is a feature I suspect not many people will use anyway. I just saw the commit message here and thought I would show up to say that it s

Re: Periodic FSM vacuum doesn't happen in one-pass strategy vacuum.

2025-04-07 Thread Melanie Plageman
On Fri, Apr 4, 2025 at 6:07 PM Masahiko Sawada wrote: > > I'm going to push this fix up to HEAD and v17 early next week, unless > there is no objection. I started studying this again looking back at the thread associated with commit c120550edb86. There was actually a long discussion about how thi

Re: Logging which local address was connected to in log_line_prefix

2025-04-07 Thread Tom Lane
Greg Sabino Mullane writes: > On Sun, Apr 6, 2025 at 6:01 PM Tom Lane wrote: >> 1. Ignore the inconsistency, commit 0004 as-is. >> >> 2. Change the output to be an empty string in background processes. >> This is consistent, but it goes against our upthread feeling that >> "[none]" would avoid c

Re: Commit fest 2025-03

2025-04-07 Thread vignesh C
On Mon, 31 Mar 2025 at 08:34, vignesh C wrote: > > On Mon, 24 Mar 2025 at 10:07, vignesh C wrote: Here's a quick commitfest status report as of today: status | start | 17th | 24th | 31st | 07th +-+---+---++---

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 15:59:19 +0200, Peter Eisentraut wrote: > On 05.04.25 02:27, Jacob Champion wrote: > > On Tue, Apr 1, 2025 at 3:40 PM Jacob Champion > > wrote: > > > Maybe a better idea would be to ship an SONAME of > > > `libpq-oauth.so.0.`, without any symlinks, so that there's > > > never an

Re: Draft for basic NUMA observability

2025-04-07 Thread Tomas Vondra
Hi, Here's a v26 of this patch series, merging the various fixup patches. I've also reordered the patches so that the pg_buffercache part is last. The two other patches are ready to go, and it seems better to push the built-in catalog before the pg_buffercache contrib module. For 0001 and 0002, I

Re: [PATCH] Fix build on MINGW on ARM64

2025-04-07 Thread Andrew Dunstan
On 2025-04-07 Mo 12:58 AM, Tom Lane wrote: Andrew Dunstan writes: On 2025-04-01 Tu 11:15 AM, Andrew Dunstan wrote: On 2025-04-01 Tu 8:47 AM, vignesh C wrote: There is an existing CF entry for this at [1]. If no one picks this till the end of this CF, we can move it to next CF. [1] - https:/

Re: pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-07 Thread Tom Lane
Melanie Plageman writes: > Yea, my email was held in moderation for days. I guess cross-posting is > flagged. I thought I saw people regularly cc pgsql-hackers when replying to > pgsql-committers, but I guess not. Yeah, that used to be common practice, but we made a policy change to discourage cr

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-07 Thread Andres Freund
Hi, On 2025-04-04 17:27:46 -0700, Jacob Champion wrote: > += Load-Time ABI = > + > +This module ABI is an internal implementation detail, so it's subject to > change > +without warning, even during minor releases (however unlikely). The compiled > +version of libpq-oauth should always match the c

Re: pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-07 Thread Alexander Korotkov
Hi, David! On Mon, Apr 7, 2025 at 12:15 PM David Rowley wrote: > > On Mon, 7 Apr 2025 at 19:39, Melanie Plageman > wrote: > > +++ > > C:/cirrus/build/testrun/pg_upgrade/002_pg_upgrade/data/results/subselect.out > > 2025-04-04 14:47:20.358393500 + > > @@ -2769,15 +2769,16 @@ > > EXPLAIN (C

Re: pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-07 Thread Melanie Plageman
On Mon, Apr 7, 2025 at 5:15 AM David Rowley wrote: > Are these failures from patches applied to master prior to 3ba2cdaa? Yea, my email was held in moderation for days. I guess cross-posting is flagged. I thought I saw people regularly cc pgsql-hackers when replying to pgsql-committers, but I g

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-07 Thread Peter Eisentraut
On 05.04.25 02:27, Jacob Champion wrote: On Tue, Apr 1, 2025 at 3:40 PM Jacob Champion wrote: Maybe a better idea would be to ship an SONAME of `libpq-oauth.so.0.`, without any symlinks, so that there's never any ambiguity about which module belongs with which libpq. While I was looking into

Re: ci: Allow running mingw tests by default via environment variable

2025-04-07 Thread Andres Freund
Hi, On 2025-04-07 16:03:48 +0300, Nazir Bilal Yavuz wrote: > On Wed, 5 Mar 2025 at 18:51, Andres Freund wrote: > > I'm inclined to think we should apply to this to all branches with CI > > support, > > not just master. It's kind of annoying to have CI infrastructure changes > > like > > this t

Re: Enhancing Memory Context Statistics Reporting

2025-04-07 Thread Daniel Gustafsson
Following up on some off-list comments, attached is a v26 with a few small last changes: * Improved documentation (docs and comments) * Fixed up Shmem sizing and init * Delayed registering to the shmem cleanup to get it earlier in cleanup * Renamed a few datastructures to improve readabili

  1   2   >