Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-21 Thread Japin Li
On Wed, 22 Jan 2025 at 10:25, Yura Sokolov wrote: > 22.01.2025 09:09, Japin Li пишет: >> Hi, Yura Sokolov >> Thanks for updating the patch. >> I test the v2 patch using BenchmarkSQL 1000 warehouse, and here is the tpmC >> result: >> case | min| avg| max >> -

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-21 Thread Yura Sokolov
22.01.2025 09:09, Japin Li пишет: On Sun, 19 Jan 2025 at 17:56, Yura Sokolov wrote: 17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK,

Re: Eager aggregation, take 3

2025-01-21 Thread Richard Guo
On Wed, Jan 22, 2025 at 1:36 AM Robert Haas wrote: > Thanks for the example. What seems to be happening here is that each > of the three joins increases the number of rows by a multiple of > either 166 or 333. Aggregating reduces the number of rows to 3. I am > not sure that we should be too conce

Re: Pgoutput not capturing the generated columns

2025-01-21 Thread Peter Smith
Patch v54-0002 LGTM, although it is missing an explanatory commit message. e.g. Should be something like: -- Adds the documentation for the 'pubgencols' column of catalog pg_publication. This was accidentally missing from commit 7054186. -- == Kind Regards, Peter Smith. Fujitsu Austr

Re: SQLJSON: errmsg(" .. should ...") -> must

2025-01-21 Thread Amit Langote
On Tue, Jan 21, 2025 at 11:42 PM Alvaro Herrera wrote: > On 2025-Jan-13, Alvaro Herrera wrote: > > > There's a few recent SQL/JSON error messages in which we say something > > "should" be something else. We avoid this, so I think we shouldn't use > > it here either. > > Pushed. Sorry Alvaro for

add function argument name to substring and substr

2025-01-21 Thread jian he
hi. attached patch add function argument name to function substring and substr you can see the visual changes. mater behavior: \df substring List of functions Schema | Name| Result data type | Argument data types | Type +---+

Re: Pgoutput not capturing the generated columns

2025-01-21 Thread Peter Smith
On Tue, Jan 21, 2025 at 7:28 PM vignesh C wrote: > > On Mon, 20 Jan 2025 at 06:14, Peter Smith wrote: > > > > Hi Vignesh, > > > > Review comments for patch v53-0001: > > > > > > Maybe I have some fundamental misunderstanding here, but I don't see > > why "this approach cannot be used with psql be

Re: Allow NOT VALID foreign key constraints on partitioned tables.

2025-01-21 Thread Amul Sul
On Wed, Jan 22, 2025 at 2:36 AM Álvaro Herrera wrote: > > > Suppose I have a hierarchy like this > > parent > | > child >/\ > / \ > /grandchild2 > / > grandchild1 > > and I have a validated constraint on grandchild1 and an invalid > constraint on c

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-21 Thread Japin Li
On Sun, 19 Jan 2025 at 17:56, Yura Sokolov wrote: > 17.01.2025 17:00, Zhou, Zhiguo пишет: >> On 1/16/2025 10:00 PM, Yura Sokolov wrote: >>> >>> Good day, Zhiguo. >>> >>> Excuse me, I feel sneaky a bit, but I've started another thread >>> just about increase of NUM_XLOGINSERT_LOCK, because I can me

Re: pg_createsubscriber TAP test wrapping makes command options hard to read.

2025-01-21 Thread Michael Paquier
On Tue, Jan 21, 2025 at 02:17:03PM +, Dagfinn Ilmari Mannsåker wrote: > Thanks again for reviewing this monster patch. +$node->command_checks_all( + [ 'pg_amcheck', '--exclude-user' => 'no_such_user', 'postgres' ], Extra error spotted here with s/--exclude-user/--username/. The double --ve

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-21 Thread Nisha Moond
On Tue, Jan 21, 2025 at 8:22 AM Peter Smith wrote: > > Some review comments for patch v61-0001. > > == > src/backend/replication/slot.c > > InvalidatePossiblyObsoleteSlot: > > 1. > /* > * Check if the slot needs to be invalidated. If it needs to be > - * invalidated, and is not currently a

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-21 Thread Nisha Moond
On Tue, Jan 21, 2025 at 8:26 AM Peter Smith wrote: > > Some review comments for patch v61-0002 > > == > src/backend/replication/slot.c > > 1. > * Whether a slot needs to be invalidated depends on the cause. A slot is > - * removed if it: > + * invalidated if it: > * - RS_INVAL_WAL_REMOVED:

RE: pgbench without dbname worked differently with psql and pg_dump

2025-01-21 Thread Hayato Kuroda (Fujitsu)
Dear David, > As none of those programs "create" a database the wording "The default is to > create a database" seems quite a poor choice to standardize on. Oh, I misunderstood. Sorry. I wanted to suggest like below: ``` The default is to connect to a database with the same name as the current s

Re: Bug in detaching a partition with a foreign key.

2025-01-21 Thread Amul Sul
On Tue, Jan 21, 2025 at 7:25 PM Álvaro Herrera wrote: > > On 2025-Jan-20, Sami Imseih wrote: > > > Patch looks good to me, > > Thanks, pushed. > A big thanks to Álvaro and Sami for getting it fixed! Regards, Amul

Re: SQL:2011 application time

2025-01-21 Thread Tom Lane
Peter Eisentraut writes: > I have committed the fix for foreign key NO ACTION (patch 0002, this did > not require patch 0001). That commit seems to be causing occasional buildfarm failures: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=indri&dt=2025-01-22%2001%3A29%3A35 https://buildf

Re: Skip collecting decoded changes of already-aborted transactions

2025-01-21 Thread Peter Smith
On Wed, Jan 22, 2025 at 5:36 AM Masahiko Sawada wrote: > > On Sun, Jan 19, 2025 at 7:53 PM Amit Kapila wrote: > > > > On Fri, Jan 17, 2025 at 11:19 PM Masahiko Sawada > > wrote: > > > > > > On Wed, Jan 15, 2025 at 4:43 PM Peter Smith wrote: > > > > > > > > My thoughts are that any consistency

RE: create subscription with (origin = none, copy_data = on)

2025-01-21 Thread Zhijie Hou (Fujitsu)
On Tuesday, January 21, 2025 1:31 AM vignesh C wrote: Hi, > > On Mon, 20 Jan 2025 at 17:31, Amit Kapila wrote: > > > > On Sat, Jan 18, 2025 at 10:31 AM vignesh C wrote: > > > > > > Attached patch has the fix for this issue which includes the > > > partition tables also for the publication now

Re: Statistics Import and Export

2025-01-21 Thread Corey Huinker
On Tue, Jan 21, 2025 at 8:37 PM jian he wrote: > hi. > now stats_lock_check_privileges comments need to change? > > * Lock relation in ShareUpdateExclusive mode, check privileges, and close > the > * relation (but retain the lock). > > since the above comments will not be true for RELKIND_INDEX

Re: pgbench without dbname worked differently with psql and pg_dump

2025-01-21 Thread David G. Johnston
On Tue, Jan 21, 2025 at 6:54 PM Hayato Kuroda (Fujitsu) < kuroda.hay...@fujitsu.com> wrote: > > IIUC createdb has the good description [1] and it can be re-used > everywhere. > As none of those programs "create" a database the wording "The default is to create a database" seems quite a poor choic

Re: allow trigger to get updated columns

2025-01-21 Thread Tom Lane
I wrote: > If this is not in fact completely broken, the reason must be that > ats_modifiedcols will always be the same for the same values of > ats_tgoid/ats_relid/ats_event/ats_table (within a given transaction). > That seems unlikely, Indeed, it's false. I was able to make a test case demonstr

RE: pgbench without dbname worked differently with psql and pg_dump

2025-01-21 Thread Hayato Kuroda (Fujitsu)
Dear Ashutosh, > I see the same behaviour. The code in pgbench which decided the > database to connect to, looks similar to the relevant code in > createdb. But createdb documentation is in agreement with the code. > [1] > ``` > dbname > > Specifies the name of the database to be created. The nam

Re: Statistics Import and Export

2025-01-21 Thread jian he
hi. now stats_lock_check_privileges comments need to change? * Lock relation in ShareUpdateExclusive mode, check privileges, and close the * relation (but retain the lock). since the above comments will not be true for RELKIND_INDEX. Yes, there are comments within the function explaining the ex

Re: Pre-allocating WAL files

2025-01-21 Thread Andy Fan
Andres Freund writes: Hi, > FWIW, I've seen the fsyncs around recycling being a rather substantial > bottleneck. To the point of the main benefit of larger segments being the > reduction in number of fsyncs at the end of a checkpoint. I think we should > be able to make the fsyncs a lot more ef

Re: Some ExecSeqScan optimizations

2025-01-21 Thread Amit Langote
On Fri, Jan 17, 2025 at 2:05 PM Amit Langote wrote: > Here's v5 with a few commit message tweaks. > > Barring objections, I would like to push this early next week. Pushed yesterday. Thank you all. -- Thanks, Amit Langote

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-01-21 Thread Michael Paquier
On Thu, Jan 02, 2025 at 12:46:04PM -0800, Lukas Fittl wrote: > Inspired by a prior proposal by Sami Imseih for tracking Plan IDs [0], as > well as extensions like pg_stat_plans [1] (unmaintained), pg_store_plans > [2] (not usable on production, see notes later) and aurora_stat_plans [3] > (enabled

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-21 Thread Sami Imseih
> You might be interested by this thread "Thinking about EXPLAIN ALTER TABLE": > https://www.postgresql.org/message-id/CAM-w4HNm1M5J-ow8UjTcqRe3JPxkVCrGe56tRpPUSePSdGcZ_w%40mail.gmail.com I reviewed this thread, and the primary issue with the EXPLAIN command lies in the inability to predict all t

Re: Show WAL write and fsync stats in pg_stat_io

2025-01-21 Thread Michael Paquier
On Thu, Jan 16, 2025 at 11:40:51AM +0300, Nazir Bilal Yavuz wrote: > I encountered another problem while rebasing the patch. The problem is > basically we do not expect any pending stats while restoring the stats > at the initdb. However, WAL IOs (WAL read and WAL init IOs for now) > may happen bef

Re: XMLDocument (SQL/XML X030)

2025-01-21 Thread Jim Jones
On 21.01.25 23:45, Robert Treat wrote: > Is there some concrete use case you have seen that this would help > with? Not objecting to adding it, but you've mentioned this migration > idea twice but it seems to me this doesn't conform with existing > implementations, and I don't see much benefit in

Re: allow trigger to get updated columns

2025-01-21 Thread Tom Lane
Peter Eisentraut writes: > On 2020-03-05 13:53, Daniel Gustafsson wrote: >> +1 on the patchset, marking this entry as Ready For Committer. > and done While looking at a pending patch, I happened to notice that commit 71d60e2aa added a field ats_modifiedcols to AfterTriggerSharedData, but did not

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-21 Thread Daniel Gustafsson
> On 21 Jan 2025, at 22:13, Joe Conway wrote: > I think this is a non-issue. Every implementation I have seen, the OS-level > enabling of FIPS mode is just a way to ensure openssl is automatically put > into FIPS mode when the library is loaded, just as if (and not depending on) > the applicat

Re: speedup COPY TO for partitioned table.

2025-01-21 Thread Melih Mutlu
Hi Jian, Thanks for the patch. jian he , 19 Ara 2024 Per, 15:03 tarihinde şunu yazdı: > attached copy_par_regress_test.sql is a simple benchmark sql file, > a partitioned table with 10 partitions, 2 levels of indirection. > The simple benchmark shows around 7.7% improvement in my local environme

Re: XMLDocument (SQL/XML X030)

2025-01-21 Thread Robert Treat
On Tue, Jan 21, 2025 at 5:58 AM Jim Jones wrote: > On 20.01.25 23:21, Chapman Flack wrote: > > Therefore I'm thinking that, given the specifics of our XML support, > > a fully conformant and efficient XMLDOCUMENT could be implemented > > just by returning its XML argument. > > > After your explana

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-21 Thread Sami Imseih
This will only reduce the size of the $PGDATA/pg_stat/pg_stat_statements.txt file. Even with 100k entries, the most I have seen pg_stat_statements.max set to, that will be less than 1 MB of disk saving. The default config of 5k entries will be much less. Regards, Sami

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2025-01-21 Thread Tom Lane
I wrote: > I wonder if we should try to fix the GIN AM to avoid that. > The column being indexed is of an array type in these cases, but the > index entries aren't. It seems inconsistent that it sets up the index > column's attndims and atttypid this way. Ah, I see the problem: it's not GIN's fau

Re: Proposal: "query_work_mem" GUC, to distribute working memory to the query's individual operators

2025-01-21 Thread Jeff Davis
On Fri, 2025-01-10 at 10:00 -0800, James Hunter wrote: > How should “query_work_mem” work? Let’s start with an example: > suppose > we have an OLAP query that has 2 Hash Joins, and no other operators > that use work_mem. So we plan first, and then assign available memory afterward? If we do it tha

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2025-01-21 Thread Oliver Ford
On Tue, Jan 21, 2025 at 1:21 AM Tatsuo Ishii wrote: > It needs lots of work including modifying CREATE FUNCTION > command. Instead you could add an API to WinObject access functions to > export ignore_nulls value. Then let each window function check it. If > the window function should not take IGN

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-21 Thread Joe Conway
On 1/21/25 15:59, Tom Lane wrote: Daniel Gustafsson writes: It could indeed be useful, but I doubt we can make it portable to check for anything but the state of OpenSSL. If the operating system has a FIPS mode then we won't capture that. That might not be a problem since if the OS is in FIPS

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-21 Thread Tom Lane
Daniel Gustafsson writes: > It could indeed be useful, but I doubt we can make it portable to check for > anything but the state of OpenSSL. If the operating system has a FIPS mode > then we won't capture that. That might not be a problem since if the OS is in > FIPS mode then OpenSSL most likel

Re: Allow NOT VALID foreign key constraints on partitioned tables.

2025-01-21 Thread Álvaro Herrera
Suppose I have a hierarchy like this parent | child /\ / \ /grandchild2 / grandchild1 and I have a validated constraint on grandchild1 and an invalid constraint on child. What happens if I add a constraint on parent? In my understanding, it sho

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-21 Thread Daniel Gustafsson
> On 21 Jan 2025, at 21:59, Tom Lane wrote: > (If we end up inventing a FIPS-mode flag, I would fully expect > interested vendors to patch our code to force it on when the > OS-level flag is set, which is exactly what they will have done > to OpenSSL. We should design our behavior with that in m

Re: pg_dump --no-comments confusion

2025-01-21 Thread Álvaro Herrera
On 2024-Nov-04, Bruce Momjian wrote: > Someone emailed me privately saying they were confused because [...] > I think this is someone with limited English ability, which could > explain the confusion. I just realized you added "Reported-by: Tom Lane" to the commit message of this change (f722dd3

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-21 Thread Daniel Gustafsson
> On 21 Jan 2025, at 18:51, Joe Conway wrote: > On 1/21/25 06:39, Daniel Gustafsson wrote: >> If we add such an alternative output we also need the other case where FIPS >> is >> disabled and the functions work, which means we add no test coverage at all >> as >> both options are allowed to pas

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2025-01-21 Thread Tom Lane
Bruce Momjian writes: > I did write a patch in Novemer 2023 to pass the dimension to the layers > that needed it, but it was considered too much code compared to its > value: > https://www.postgresql.org/message-id/zvwi_ozt8z9mc...@momjian.us Ah, I'd forgotten that discussion. It looks lik

Re: Statistics Import and Export

2025-01-21 Thread Corey Huinker
> > >> >> > Sorry about that, my nvim config is auto-including stuff and it's annoying. > > Now with less includes and fewer typos: From 93e67e5304132e680b628b3e92f9fd242d998abd Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Tue, 21 Jan 2025 11:52:58 -0500 Subject: [PATCH v42] Lock table firs

Re: Parallel heap vacuum

2025-01-21 Thread Masahiko Sawada
On Sun, Jan 19, 2025 at 7:50 AM Tomas Vondra wrote: > > Hi, > > Thanks for the new patches. I've repeated my benchmarking on v8, and I > agree this looks fine - the speedups are reasonable and match what I'd > expect on this hardware. I don't see any suspicious results like with > the earlier patc

Re: Non-text mode for pg_dumpall

2025-01-21 Thread Mahendra Singh Thalor
Thanks Jian for the detailed review and testing. On Mon, 20 Jan 2025 at 21:32, jian he wrote: > > hi. > some minor issues come to my mind when I look at it again. > > looking at set_null_conf, > i think "if (archDumpFormat != archNull)" can be: > > if (archDumpFormat != archNull) > { > OPF = fope

Re: SQL:2011 application time

2025-01-21 Thread Peter Eisentraut
On 12.01.25 00:19, Paul Jungwirth wrote: On 1/4/25 13:39, Paul Jungwirth wrote: These updates fix a problem in the unaccent contrib module. When I added a new parameter to get_func_namespace, I changed a call there. Then I when took out that parameter, I didn't update the extension again. Othe

Re: New feature request for adding session information to PostgreSQL transaction log

2025-01-21 Thread Andres Freund
Hi, On 2025-01-21 13:28:43 -0500, Chapman Flack wrote: > On 01/19/25 12:02, Tom Lane wrote: > > You can build that yourself, typically by adding a trigger that stores > > the value of "current_user" into inserted/updated rows. (If you want > > to also track deletions, a separate audit log table w

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-01-21 Thread Artem Gavrilov
On Thu, Jan 2, 2025 at 10:47 PM Lukas Fittl wrote: > > The first patch allows use of node jumbling by other unit files / > extensions, which would help an out-of-core extension avoid duplicating all > the node jumbling code. > > The second patch adds a function for the extensible cumulative stati

Re: New feature request for adding session information to PostgreSQL transaction log

2025-01-21 Thread Tom Lane
Chapman Flack writes: > I wonder how close one could get to the customer request (better > forensics without having to build extra columns and triggers at the > SQL level) with an extension and existing hooks. An extension would be a great solution if we can make it work. > I haven't used it, bu

Re: Skip collecting decoded changes of already-aborted transactions

2025-01-21 Thread Masahiko Sawada
On Sun, Jan 19, 2025 at 7:53 PM Amit Kapila wrote: > > On Fri, Jan 17, 2025 at 11:19 PM Masahiko Sawada > wrote: > > > > On Wed, Jan 15, 2025 at 4:43 PM Peter Smith wrote: > > > > > > My thoughts are that any consistency improvement is a step in the > > > right direction so even "don't increase

Re: Year of first commit

2025-01-21 Thread Tom Lane
Marcos Pegoraro writes: > Why is this important ? Well, if that file has a 2014-2025, I can know how > old it is, more or less which version was included, etc > Additionally, I see all of you adding or removing a single letter to > sources, why not adjust those years on header files ? Our git his

Re: New feature request for adding session information to PostgreSQL transaction log

2025-01-21 Thread Chapman Flack
On 01/19/25 12:02, Tom Lane wrote: > You can build that yourself, typically by adding a trigger that stores > the value of "current_user" into inserted/updated rows. (If you want > to also track deletions, a separate audit log table would work > better.) The event-trigger feature might also be us

Re: Compression of bigger WAL records

2025-01-21 Thread Andrey M. Borodin
> On 12 Jan 2025, at 17:43, Andrey M. Borodin wrote: > > I attach a prototype patch. Here's v2, now it passes all the tests with wal_debug. Some stats. On this test create table a as select random() from generate_series(1,1e7); select pg_stat_reset_shared('wal'); create index on a(random );

Re: Year of first commit

2025-01-21 Thread Marcos Pegoraro
Em ter., 21 de jan. de 2025 às 14:50, Álvaro Herrera < alvhe...@alvh.no-ip.org> escreveu: > I have to ask, why do you think this is important? > I'm just reading sources, navigating on files history and learning, so don't be furious with me. Why is this important ? Well, if that file has a 2014-2

Re: Using Expanded Objects other than Arrays from plpgsql

2025-01-21 Thread Tom Lane
Michel Pelletier writes: > Thanks Tom! These applied cleanly to my test env and actually increased my > livejournal graph benchmark by a million edges per second, so I guess I > didn't have all the previous changes in my last build as you noted. Nice! I hope somebody will review this, because I

Re: Using Expanded Objects other than Arrays from plpgsql

2025-01-21 Thread Michel Pelletier
On Wed, Jan 15, 2025 at 10:09 AM Tom Lane wrote: > I noticed that v2 of this patch series failed to apply after > 7b27f5fd3, so here's v3. No non-trivial changes. > Thanks Tom! These applied cleanly to my test env and actually increased my livejournal graph benchmark by a million edges per sec

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-21 Thread Joe Conway
On 1/21/25 06:39, Daniel Gustafsson wrote: On 20 Jan 2025, at 01:26, Koshi Shibagaki (Fujitsu) wrote: Thank you for moving this discussion forward. Maybe we could document that the test should fail if fips is enabled? FWIW I have not tested at all on a fips enabled machine. I will see about

Re: Year of first commit

2025-01-21 Thread Álvaro Herrera
On 2025-Jan-21, Marcos Pegoraro wrote: > But these numbers seem inaccurate, because there are 1644 copyright strings > but 1521 are 1996-2025. So, JSONB, BRIN, FDW, Logical Replication and lots > of others, all of them were started in 1996 or have any related code ? I have to ask, why do you thin

RE: Proposal for Updating CRC32C with AVX-512 Algorithm.

2025-01-21 Thread Devulapalli, Raghuveer
> Hello! I'm Matthew Sterrett and I'm a coworker of Raghuveer; he asked me to > look into the Windows build failures related to pg_comp_crc32c. > > It seems that the only thing that was required to fix that is to mark > pg_comp_crc32c as PGDLLIMPORT, so I added a patch that does just that. > I'm n

Re: Year of first commit

2025-01-21 Thread Tom Lane
Marcos Pegoraro writes: > But these numbers seem inaccurate, because there are 1644 copyright strings > but 1521 are 1996-2025. [ shrug... ] I did not claim that this idea has been adhered to 100%. In a code base as old and large as ours, very few things are adhered to 100%.

Re: How to deinitialize a connection for background worker

2025-01-21 Thread Tom Lane
"ISHAN CHHANGANI ." writes: > I am working on a project which requires me to create a background worker. > This worker checks some array(shared memory) every second and then connects > to the database mentioned within the array. That is I want to connect to > different databases using the same

Re: Year of first commit

2025-01-21 Thread Marcos Pegoraro
Em ter., 21 de jan. de 2025 às 12:36, Tom Lane escreveu: > so we prefer to use the same copyright dates as related older files even > in less > clear-cut cases. > Thank you Tom for your time. But these numbers seem inaccurate, because there are 1644 copyright strings but 1521 are 1996-2025. So,

Re: Accept recovery conflict interrupt on blocked writing

2025-01-21 Thread Anthonin Bonnefoy
Hi, Thanks for the detailed explanations, I've definitely misinterpreted how interrupts and errors were handled. On Fri, Jan 17, 2025 at 7:03 PM Andres Freund wrote: > > Might be worth using it it in src/test/postmaster/t/002_start_stop.pl? That > has e.g. code to send a startup message. I've c

Re: [External] : Re: New feature request for adding session information to PostgreSQL transaction log

2025-01-21 Thread Sumanth Vishwaraj
Hi Tom, Our customers prefer "in product" solution instead of trigger-based solution due to the overheads and risks associated with triggers on their Production systems. Oracle already provides the user session information in the redo/transaction logs. We have created a feature request, and M

How to deinitialize a connection for background worker

2025-01-21 Thread ISHAN CHHANGANI .
I am working on a project which requires me to create a background worker. This worker checks some array(shared memory) every second and then connects to the database mentioned within the array. That is I want to connect to different databases using the same background worker without killing it.

Re: Orphaned users in PG16 and above can only be managed by Superusers

2025-01-21 Thread Robert Haas
On Thu, Jan 9, 2025 at 12:31 AM Ashutosh Sharma wrote: > Starting from PG16, it seems that orphaned users can only be managed > by superusers. For example, if userA creates userB, and userB creates > userC, then both userB (the parent of userC) and userA (the > grandparent of userC) would typicall

Re: Eager aggregation, take 3

2025-01-21 Thread Robert Haas
On Tue, Jan 21, 2025 at 3:33 AM Richard Guo wrote: > I've been thinking about this proposal, and it's quite appealing. It > would significantly reduce both the planning effort and implementation > complexity, while still yielding reasonable planning results. > > One concern I have with this propo

Re: Enhancing Memory Context Statistics Reporting

2025-01-21 Thread Fujii Masao
On 2025/01/21 20:27, Rahila Syed wrote: Hi Tomas,  I've tried the pgbench test again, to see if it gets stuck somewhere, and I'm observing this on a new / idle cluster: $ pgbench -n -f test.sql -P 1 test -T 60 pgbench (18devel) progress: 1.0 s, 1647.9 tps, lat 0.604

Re: Pre-allocating WAL files

2025-01-21 Thread Andres Freund
Hi, On 2025-01-21 10:13:14 -0600, Nathan Bossart wrote: > On Tue, Jan 21, 2025 at 09:52:51AM -0600, Nathan Bossart wrote: > > On Tue, Jan 21, 2025 at 03:31:27AM +, Andy Fan wrote: > >> 3. Why is the purpose of preallocated_segments directory? what in my > >> mind is we just prellocate the norm

Re: Statistics Import and Export

2025-01-21 Thread Corey Huinker
> > > > > /* > * For indexes, we follow what do_analyze_rel() does so as to avoid any > * deadlocks with analyze/vacuum, which is to take out a > * ShareUpdateExclusive on table/matview first and only after that take > * a a AccessShareLock on the index itself. > */ > t

Re: Pre-allocating WAL files

2025-01-21 Thread Nathan Bossart
On Tue, Jan 21, 2025 at 09:52:51AM -0600, Nathan Bossart wrote: > On Tue, Jan 21, 2025 at 03:31:27AM +, Andy Fan wrote: >> 3. Why is the purpose of preallocated_segments directory? what in my >> mind is we just prellocate the normal filename so that XLogWrite could >> open it directly. This is

doc: explain pgstatindex fragmentation

2025-01-21 Thread Benoit Lobréau
Hi, I like the clarification but I think that: A higher leaf_fragmentation indicates that the physical order of the index leaf pages increasingly deviates from their logical order. Would be cleaner than: The higher leaf_fragmentation is, the less the physical order of the index leaf pages c

Re: Pre-allocating WAL files

2025-01-21 Thread Nathan Bossart
On Tue, Jan 21, 2025 at 03:31:27AM +, Andy Fan wrote: > Come from [0] and thanks for working on this. Here are some design > review/question after my first going through the patch. Thanks for taking a look. > 1. walwriter vs checkpointer? I prefer to walwriter for now because.. > > a. chec

Re: [PATCH] Add roman support for to_number function

2025-01-21 Thread Tom Lane
Hunaid Sohail writes: > The leading spaces are consumed in the RN (from the main loop > in Num_processor), and this behavior seems consistent with how > simple numbers are handled. The Roman numeral parsing > appears to start from where "RN" is in the format after > leading spaces are consumed. Y

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-21 Thread Ivan Kush
What does this patch give on aglobal scale? Does it save much memory or increase performance? How many times? On 1/21/25 13:51, m.litsa...@postgrespro.ru wrote: // Mutex should be last field, as this field isn't copied to dump file Updated. 2) You didn't take into account the upgrade. Saved i

Re: Statistics Import and Export

2025-01-21 Thread jian he
On Tue, Jan 21, 2025 at 2:43 PM Corey Huinker wrote: > > > > On Mon, Jan 20, 2025 at 10:59 PM Corey Huinker > wrote: >>> >>> Are you saying that there is a path for a partitioned index to have >>> stats today? If so, we can just follow that locking protocol. If not, >>> I'm concerned about tryin

Re: Year of first commit

2025-01-21 Thread Tom Lane
Marcos Pegoraro writes: > Today Amit Langote committed execScan.h and put on its header years from > 1996 to 2025. Those years are what exactly ? 1996 is when the open-source Postgres effort started. There's a bigger philosophical issue here, which is whether a new file shouldn't just list the c

Re: SQLJSON: errmsg(" .. should ...") -> must

2025-01-21 Thread Alvaro Herrera
On 2025-Jan-13, Alvaro Herrera wrote: > There's a few recent SQL/JSON error messages in which we say something > "should" be something else. We avoid this, so I think we shouldn't use > it here either. Pushed. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/

Re: pg_createsubscriber TAP test wrapping makes command options hard to read.

2025-01-21 Thread Dagfinn Ilmari Mannsåker
Michael Paquier writes: > On Tue, Jan 21, 2025 at 10:31:01AM +1100, Peter Smith wrote: >> I applied the v5* patches and ran make check-world. All passed OK. >> >> Your fat comma changes have greatly improved readability, particularly >> in 040_createsubscriber (the file that caused me to start t

Re: Eager aggregation, take 3

2025-01-21 Thread Richard Guo
On Tue, Jan 21, 2025 at 2:57 AM Tom Lane wrote: > However, a partial-aggregation path does not generate the same data > as an unaggregated path, no matter how fuzzy you are willing to be > about the concept. So I'm having a very hard time accepting that > it ought to be part of the same RelOptInf

Re: Bug in detaching a partition with a foreign key.

2025-01-21 Thread Álvaro Herrera
On 2025-Jan-20, Sami Imseih wrote: > Patch looks good to me, Thanks, pushed. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/

Re: Add XMLNamespaces to XMLElement

2025-01-21 Thread Pavel Stehule
Hi út 21. 1. 2025 v 11:45 odesílatel Umar Hayat napsal: > On Tue, 21 Jan 2025 at 04:04, Jim Jones wrote: > > > > Hi Umar > > > > On 20.01.25 17:19, Umar Hayat wrote: > > > Hi Jim & Pavel, > > > Sorry for getting back a bit late on this. Few more case you might > > > need consider: > > > > > > A

Year of first commit

2025-01-21 Thread Marcos Pegoraro
Today Amit Langote committed execScan.h and put on its header years from 1996 to 2025. Those years are what exactly ? This file has 1996, but sometimes is the first commit year of that file, sometimes is completely unrelated. What is right for it ? Need that to be adjusted ? /*

Re: Timeline issue if StartupXLOG() is interrupted right before end-of-recovery record is done

2025-01-21 Thread Roman Eskin
Hi Andrey, Thank you for your feedback! I think here you can just specify target timeline for the standby instance_1 and it will continue recovery from instance_2. Most likely yes, but nevertheless it looks more like a W/A. Persisting recovery signal file for some _timeout_ seems super dan

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-21 Thread Daniel Gustafsson
> On 20 Jan 2025, at 01:26, Koshi Shibagaki (Fujitsu) > wrote: > > Thank you for moving this discussion forward. > >> Maybe we could document that the test should fail if fips is enabled? >> >> FWIW I have not tested at all on a fips enabled machine. I will see about >> doing >> that... > I t

Re: Enhancing Memory Context Statistics Reporting

2025-01-21 Thread Rahila Syed
Hi Tomas, I've tried the pgbench test > again, to see if it gets stuck somewhere, and I'm observing this on a > new / idle cluster: > > $ pgbench -n -f test.sql -P 1 test -T 60 > pgbench (18devel) > progress: 1.0 s, 1647.9 tps, lat 0.604 ms stddev 0.438, 0 failed > progress: 2.0 s, 1374.3 tps, la

Re: Purpose of wal_init_zero

2025-01-21 Thread Ritu Bhandari
Hi @Andres Freund > I'm not sure I understand the specifics here - did the high WAL generation > rate result in the recycling taking too long? Or did checkpointer take too > long to write out data, and because of that recycling didn't happen frequently > enough? If the WAL generation rate highl

Re: XMLDocument (SQL/XML X030)

2025-01-21 Thread Jim Jones
On 20.01.25 23:21, Chapman Flack wrote: > Therefore I'm thinking that, given the specifics of our XML support, > a fully conformant and efficient XMLDOCUMENT could be implemented > just by returning its XML argument. After your explanation, I tend to agree. v3, attached, incorporates these chan

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-21 Thread m . litsarev
// Mutex should be last field, as this field isn't copied to dump file Updated. 2) You didn't take into account the upgrade. Saved in Postgres with this byte and try to load in version without this byte. The PGSS_DUMP_FILE format is defined by PGSS_FILE_HEADER magic number (the first four byte

Re: Orphaned users in PG16 and above can only be managed by Superusers

2025-01-21 Thread Ashutosh Sharma
Hi All, On Thu, Jan 9, 2025 at 11:01 AM Ashutosh Sharma wrote: > > Hi All, > > Starting from PG16, it seems that orphaned users can only be managed > by superusers. For example, if userA creates userB, and userB creates > userC, then both userB (the parent of userC) and userA (the > grandparent o

Re: Add XMLNamespaces to XMLElement

2025-01-21 Thread Umar Hayat
On Tue, 21 Jan 2025 at 04:04, Jim Jones wrote: > > Hi Umar > > On 20.01.25 17:19, Umar Hayat wrote: > > Hi Jim & Pavel, > > Sorry for getting back a bit late on this. Few more case you might > > need consider: > > > > As I mentioned in my first static review about XMLTable existing > > behaviour m

Re: pgbench without dbname worked differently with psql and pg_dump

2025-01-21 Thread Ashutosh Bapat
On Tue, Jan 21, 2025 at 12:33 PM Hayato Kuroda (Fujitsu) wrote: > > Dear hackers, > > Recently the 'd' option of pgbench is unified with other applications like > psql and pg_dump, > but I found further difference. pgbebch uses an OS user as the dbanme even > when the '-U' is > specified. Please

Re: Psql meta-command conninfo+

2025-01-21 Thread Hunaid Sohail
Hi, On Mon, Jan 20, 2025 at 6:34 PM Maiquel Grassi wrote: > >That leads me to also wonder why don't we change \conninfo to have this > >tabular behavior instead of creating a separate command for it. Why do > >we need to keep the existing form of \conninfo? To me it seems strictly > >less usef

Re: table_tuple_lock's snapshot argument

2025-01-21 Thread Alexander Korotkov
On Tue, Jan 21, 2025 at 12:03 PM Alexander Korotkov wrote: > On Sun, Dec 29, 2024 at 3:59 PM Heikki Linnakangas wrote: > > > > > /* > > > * Lock a tuple in the specified mode. > > > * > > > * Input parameters: > > > *relation: relation containing tuple (caller must hold suitable lock) > >

Re: table_tuple_lock's snapshot argument

2025-01-21 Thread Alexander Korotkov
Hi! On Sun, Dec 29, 2024 at 3:59 PM Heikki Linnakangas wrote: > > > /* > > * Lock a tuple in the specified mode. > > * > > * Input parameters: > > *relation: relation containing tuple (caller must hold suitable lock) > > *tid: TID of tuple to lock > > *snapshot: snapshot to use

Re: Bug in detaching a partition with a foreign key.

2025-01-21 Thread Álvaro Herrera
On 2025-Jan-20, Sami Imseih wrote: > Patch looks good to me, but I am not sure about this part of the comment: > > "Only the topmost one is to be considered here; the child constraints > must be left alone," > > In this case, none of the pg_constraint entries are actually considered. > right?

Re: Non-text mode for pg_dumpall

2025-01-21 Thread jian he
hi. + printfPQExpBuffer(query, + "SELECT substring ( " + " '%s' , " + " '%s' ) ", str, ptrn); + result = executeQuery(conn, query->data); + if (PQresultStatus(result) == PGRES_TUPLES_OK) + { + if (PQntuples(result) == 1) + { + const char *outstr; + + outstr = PQgetvalue(result, 0, 0); i think he

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-21 Thread Daniel Gustafsson
> On 21 Jan 2025, at 01:40, Jacob Champion > wrote: > On Mon, Jan 20, 2025 at 2:10 PM Daniel Gustafsson wrote: >> + /* Done. Tear down the async implementation. */ >> + conn->cleanup_async_auth(conn); >> + conn->cleanup_async_auth = NULL; >> + Assert(conn->altsock == PGINVALID_SOCKET);

Re: [PATCH] Add roman support for to_number function

2025-01-21 Thread Hunaid Sohail
On Tue, Jan 21, 2025 at 1:35 PM Hunaid Sohail wrote: > Agreed. Your changes in v7 consumes leading spaces if leading space > is present in format (' RN'). But we need it to work on cases like: > SELECT to_number(' XIV', 'RN'); > So, I can add this logic in my next patch. > So, I was playing wi

  1   2   >