Re: Psql meta-command conninfo+

2025-01-09 Thread Hunaid Sohail
Hi, On Fri, Jan 10, 2025 at 1:50 AM Sami Imseih wrote: > > If we go with the 3 column format, then we will just > > have a bunch of repeated values for Category, which > > looks cluttered, IMO. > > "cluttered" is maybe the wrong description. I meant the output > will look overwhelming due to the

Re: Make pg_stat_io view count IOs as bytes instead of blocks

2025-01-09 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 Jan 2025 at 04:47, Michael Paquier wrote: > > On Thu, Jan 09, 2025 at 03:30:37PM +, Bertrand Drouvot wrote: > > We first use an Assert in is_ioop_tracked_in_bytes() and then we return > > a value "just" to check another Assert. I wonder if it wouldn't make more > > sense > > t

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-09 Thread Andy Fan
Andy Fan writes: > Hi: > > I run into the {subject} issue with the below setup. > > cat foo.sql > > \setshell txn_mode echo ${TXN_MODE} > \setshell speed echo ${SPEED} > \setshell sleep_ms echo ${SLEEP_MS} > \setshell subtxn_mode echo ${SUBTXN_MODE} > > select 1; > > $ TXN_MODE=-1 SPEED=1 SLEEP_M

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2025-01-09 Thread jian he
On Wed, Jan 8, 2025 at 3:05 PM Kirill Reshke wrote: > > So, in this version you essentially removed support for REJECT_LIMIT + > SET_TO_NULL feature? Looks like a promising change. It is more likely > to see this committed. > So, +1 on that too. > > However, v9 lacks tests for REJECT_LIMIT vs erro

Re: Infinite loop in XLogPageRead() on standby

2025-01-09 Thread Michael Paquier
On Fri, Mar 01, 2024 at 01:16:37PM +0900, Kyotaro Horiguchi wrote: > This code intends to prevent a page header error from causing a record > reread, when a record is required to be read from multiple sources. We > could restrict this to only fire at segment boundaries. At segment > boundaries, we

Re: Small refactoring around vacuum_open_relation

2025-01-09 Thread Michael Paquier
On Thu, Jan 09, 2025 at 01:00:06PM +0100, Daniel Gustafsson wrote: > On 9 Jan 2025, at 11:45, Alvaro Herrera wrote: >> Dunno what others think, this seems useless churn to me. > > I agree, I don't see this providing enough value to warrant the changes. Same here, let's leave things as they are.

Re: An improvement of ProcessTwoPhaseBuffer logic

2025-01-09 Thread Michael Paquier
On Thu, Jan 09, 2025 at 11:21:38AM +0300, Давыдов Виталий wrote: > Could you please send the latest version of the patch, if any? I haven't > found any new patches in the latest email. I've applied fixes for this stuff as of e3584258154f (down to 13) and 7e125b20eed6 (down to 17) with tests for a

Re: WAL-logging facility for pgstats kinds

2025-01-09 Thread Michael Paquier
On Tue, Dec 31, 2024 at 11:29:26AM +, Bertrand Drouvot wrote: > === 1 > > + * Copyright (c) 2001-2024, PostgreSQL Global Development Group > > As pgstat_kind.h is a new file, s/Copyright (c) 2001-2024/Copyright (c) 2025/ > maybe? Fixed. > No more comments, as v2-0002 is "just" moving so

Re: Small refactoring around vacuum_open_relation

2025-01-09 Thread Ashutosh Bapat
> On Thu, Jan 9, 2025 at 5:30 PM Daniel Gustafsson wrote: > > > On 9 Jan 2025, at 11:45, Alvaro Herrera wrote: > > > Dunno what others think, this seems useless churn to me. > > I agree, I don't see this providing enough value to warrant the changes. > I agree about most of the changes however

Re: Some ExecSeqScan optimizations

2025-01-09 Thread David Rowley
On Fri, 10 Jan 2025 at 02:46, Amit Langote wrote: > > On Mon, Jan 6, 2025 at 10:18 PM David Rowley wrote: > > I've attached my workings of what I was messing around with. It seems > > to perform about the same as your version. I think maybe we'd need > > some sort of execScan.h instead of where I

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-01-09 Thread Abhishek Chanda
Thanks for the feedback, everyone. Attached a followup with the following changes compared to the initial version: 1. Converted sslkeylogfile to a connection parameter 2. Added a mechanism to chmod the key log file to 0600 3. Added docs and tests I tested this manually. Also ran make check and ma

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2025-01-09 Thread Peter Smith
Hi Robert. The content and rendering of patch v2 LGTM. Should the word wrapping within the file doc/src/sgml/logical-replication.sgml be tidied up though? == Kind Regards, Peter Smith. Fujitsu Australia

Re: CREATE SUBSCRIPTION - add missing test case

2025-01-09 Thread Peter Smith
On Sun, Dec 8, 2024 at 10:57 AM Tomas Vondra wrote: > > On 8/22/24 05:21, Peter Smith wrote: > > ... > >>> > >>> I also don't see a test for this error condition. However, it is not > >>> clear to me how important is it to cover this error code path. This > >>> code has existed for a long time and

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2025-01-09 Thread Robert Treat
On Thu, Jan 9, 2025 at 2:46 AM Peter Smith wrote: > > On Sat, Jan 4, 2025 at 4:23 AM Robert Treat wrote: > > > > On Wed, Dec 18, 2024 at 5:56 AM Amit Kapila wrote: > > > > > > On Thu, Feb 8, 2024 at 9:57 AM Laurenz Albe > > > wrote: > > > > > > > > On Thu, 2024-02-08 at 13:40 +1100, Peter Smit

Re: Log a warning in pg_createsubscriber for max_slot_wal_keep_size

2025-01-09 Thread Peter Smith
On Mon, Jan 6, 2025 at 1:29 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for creating a patch. I have one comment about it. > > check_publisher() assumed that the SQL function > `pg_catalog.current_setting('max_slot_wal_keep_size')` > will return the numeric, but it just return

pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-09 Thread Andy Fan
Hi: I run into the {subject} issue with the below setup. cat foo.sql \setshell txn_mode echo ${TXN_MODE} \setshell speed echo ${SPEED} \setshell sleep_ms echo ${SLEEP_MS} \setshell subtxn_mode echo ${SUBTXN_MODE} select 1; $ TXN_MODE=-1 SPEED=1 SLEEP_MS=0 SUBTXN_MODE=-1 pgbench -n -ffoo.sql

Re: Proposal: add new API to stringinfo

2025-01-09 Thread Tatsuo Ishii
> Looks generally reasonable to me. Thanks for looking into the patch. > +/* > + * initStringInfoInternal > + * > + * Initialize a StringInfoData struct (with previously undefined contents) > + * to describe an empty string. > + * The initial memory allocation size is specified by 'initsize'. > +

Re: Make pg_stat_io view count IOs as bytes instead of blocks

2025-01-09 Thread Michael Paquier
On Thu, Jan 09, 2025 at 03:30:37PM +, Bertrand Drouvot wrote: > We first use an Assert in is_ioop_tracked_in_bytes() and then we return > a value "just" to check another Assert. I wonder if it wouldn't make more > sense > to get rid of this function and use a macro instead, something like? >

Re: New GUC autovacuum_max_threshold ?

2025-01-09 Thread wenhui qiu
HI Nathan Frédéric Yhuel On 1/7/25 23:57, Nathan Bossart wrote: > Here is a rebased patch for cfbot. AFAICT we are still pretty far from > consensus on which approach to take, unfortunately. > > For what it's worth, although I would have preferred the sub-linear > growth thing, I'd much rath

Re: Log a warning in pg_createsubscriber for max_slot_wal_keep_size

2025-01-09 Thread Peter Smith
Hi Shubham, Some review comments for patch v8=0001. == Commit message. 1. By ensuring 'max_slot_wal_keep_size' is -1, this patch prevents the potential deletion of required WAL files on the publisher that could disrupt logical replication. A test case has been added to validate correct warni

Re: IANA timezone abbreviations versus timezone_abbreviations

2025-01-09 Thread Tom Lane
I wrote: > 0003 is new work that fixes the pg_timezone_abbrevs view so > that its output matches the new behavior. Hmph ... the cfbot doesn't like that one bit. It appears that the "LMT" abbrev is known in my machine's system-provided tzdata tree, but not when building our own timezone files. I

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

2025-01-09 Thread Michail Nikolaev
Hello! Sorry, I should have expressed my thoughts in more detail, as they don't matter as much as the time you took to answer. >I don't quite read it as covering IOS. To me, the comment is more > along the lines of (extensively extended): My idea was just to add a few more details about the lock

Re: Conflict detection for update_deleted in logical replication

2025-01-09 Thread Masahiko Sawada
On Wed, Jan 8, 2025 at 7:26 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, January 9, 2025 9:48 AM Masahiko Sawada > wrote: > > Hi, > > > > > On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Wednesday, January 8, 2025 6:33 PM Masahiko Sawada > > wrote: > > > > > >

Re: Collation & ctype method table, and extension hooks

2025-01-09 Thread Jeff Davis
On Mon, 2024-12-02 at 23:58 -0800, Jeff Davis wrote: > On Mon, 2024-12-02 at 16:39 +0100, Andreas Karlsson wrote: > > I feel your first patch in the series is something you can just > > commit. > > Done. > > I combined your patches and mine into the attached v10 series. Here's v12 after committ

Re: per backend WAL statistics

2025-01-09 Thread Michael Paquier
On Thu, Jan 09, 2025 at 07:05:54AM +, Bertrand Drouvot wrote: > PFA v3 which is v2 refactoring with your proposed above changes. An extra thing I have finished by doing is removing PgStat_BackendPendingIO, then applied the change. It was useful when returned as a result of pgstat_prep_backend

Re: IANA timezone abbreviations versus timezone_abbreviations

2025-01-09 Thread Tom Lane
Aleksander Alekseev writes: > I tested and reviewed the patch. It fixes the originally reported bug > and looks good to me. Thanks for looking at it! I'm not sure we have full consensus on this yet, but at least there seem to be a majority in favor. So here's a v2 with some loose ends cleaned up

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-09 Thread Tomas Vondra
On 1/9/25 21:42, Melanie Plageman wrote: > On Tue, Dec 31, 2024 at 6:07 PM Tomas Vondra wrote: >> >> This means that ultimately it's either (1) or (3), and the more I've >> been looking into this the more I prefer (1), for a couple reasons: >> >> * It's much simpler (it doesn't really change an

Re: Proposal: add new API to stringinfo

2025-01-09 Thread Nathan Bossart
On Thu, Jan 09, 2025 at 03:21:41PM +0900, Tatsuo Ishii wrote: > Ok, I have created v3 patch to do more inlining as you suggested. With > the patch I confirmed that there's no call to functions except palloc > in makeStringInfo, makeStringInfoExt, initStringInfo and > initStringInfoExt in the asm co

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-09 Thread Tomas Vondra
On 1/9/25 23:18, Melanie Plageman wrote: > On Sun, Jan 5, 2025 at 10:00 PM Tomas Vondra wrote: >> >> I think the general idea and formula explained in [1] is right, but >> while working on the PoC patch I started to think about how to formalize >> this. And I ended up creating two tables that I th

Re: Fix a wrong errmsg in AlterRole()

2025-01-09 Thread Nathan Bossart
On Thu, Jan 09, 2025 at 12:27:20PM -0600, Nathan Bossart wrote: > LGTM. I'll plan on committing this shortly. Committed. -- nathan

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-09 Thread Daniel Gustafsson
> On 9 Jan 2025, at 23:35, Jacob Champion > wrote: > > On Thu, Jan 9, 2025 at 12:40 PM Daniel Gustafsson wrote: >> + * Find the start of the .well-known prefix. IETF rules state this must be >> + * at the beginning of the path component, but OIDC defined it at the end >> + * instead, so we have

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-09 Thread Jacob Champion
On Thu, Jan 9, 2025 at 12:40 PM Daniel Gustafsson wrote: > + * Find the start of the .well-known prefix. IETF rules state this must be > + * at the beginning of the path component, but OIDC defined it at the end > + * instead, so we have to search for it anywhere. > I was looking for a reference f

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-09 Thread Melanie Plageman
On Sun, Jan 5, 2025 at 10:00 PM Tomas Vondra wrote: > > I think the general idea and formula explained in [1] is right, but > while working on the PoC patch I started to think about how to formalize > this. And I ended up creating two tables that I think visualize is > pretty nicely. > > Imagine a

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-09 Thread Daniel Gustafsson
> On 9 Jan 2025, at 02:45, Melanie Plageman wrote: > Attached is v2 (required a rebase). I think this is a really good restructuring which will make life easier for our users. Some of the comments below are on wording which wasn't introduced in this patch, but which I hadn't thought about befor

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

2025-01-09 Thread Matthias van de Meent
On Thu, 9 Jan 2025 at 22:00, Michail Nikolaev wrote: > > Hello. > > One thing I think we could add to the patches is to adapt the 10-years-old > comment below with notice about IOS: > > /* > * We save the LSN of the page as we read it, so that we know whether it > * safe to apply LP_DEAD hints to

Re: AIO v2.0

2025-01-09 Thread Ants Aasma
On Thu, 9 Jan 2025 at 18:25, Andres Freund wrote: > > I'm curious about this because the checksum code should be fast enough > > to easily handle that throughput. > > It seems to top out at about ~5-6 GB/s on my 2x Xeon Gold 6442Y > workstation. But we don't have a good ready-made way of testing t

Re: Sample rate added to pg_stat_statements

2025-01-09 Thread Ilia Evdokimov
On 22.11.2024 09:08, Alexander Korotkov wrote: On Wed, Nov 20, 2024 at 12:07 AM Michael Paquier wrote: On Tue, Nov 19, 2024 at 09:39:21AM -0500, Greg Sabino Mullane wrote: Oh, and a +1 in general to the patch, OP, although it would also be nice to start finding the bottlenecks that cause suc

Re: Sample rate added to pg_stat_statements

2025-01-09 Thread Ilia Evdokimov
On 09.01.2025 22:13, Alena Rybakina wrote: Hi! Thank you for the work with this subject. I looked at your patch and noticed that this part of the code is repeated several times: if (nesting_level == 0)     {     if (!IsParallelWorker())     current_query_sampled = pg_prng_doubl

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

2025-01-09 Thread Michail Nikolaev
Hello. One thing I think we could add to the patches is to adapt the 10-years-old comment below with notice about IOS: /* * We save the LSN of the page as we read it, so that we know whether it * safe to apply LP_DEAD hints to the page later. This allows us to drop * the pin for MVCC scans, which

Re: AIO v2.0

2025-01-09 Thread Andres Freund
Hi, On 2025-01-09 20:10:24 +0200, Ants Aasma wrote: > On Thu, 9 Jan 2025 at 18:25, Andres Freund wrote: > > > I'm curious about this because the checksum code should be fast enough > > > to easily handle that throughput. > > > > It seems to top out at about ~5-6 GB/s on my 2x Xeon Gold 6442Y > >

Re: Psql meta-command conninfo+

2025-01-09 Thread Sami Imseih
> If we go with the 3 column format, then we will just > have a bunch of repeated values for Category, which > looks cluttered, IMO. "cluttered" is maybe the wrong description. I meant the output will look overwhelming due to the repeated values :) Regards, Sami

Re: Psql meta-command conninfo+

2025-01-09 Thread Sami Imseih
> I dislike totally discarding the category information we have available to > us. How about making a long output with just three columns in non-expanded > mode. Setting, Value, Category ? We can probably make due without a category as the name of the setting is informative enough, right? If

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-09 Thread Melanie Plageman
On Tue, Dec 31, 2024 at 6:07 PM Tomas Vondra wrote: > > This means that ultimately it's either (1) or (3), and the more I've > been looking into this the more I prefer (1), for a couple reasons: > > * It's much simpler (it doesn't really change anything on the basic > behavior, doesn't introduce a

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-09 Thread Daniel Gustafsson
> On 20 Dec 2024, at 02:00, Jacob Champion > wrote: > v40 also contains: A few small comments on v40 rather than saving up for a longer email: + ereport(LOG, errmsg("Internal error in OAuth validator module")); Tiny nitpick, the errmsg() should start with lowercase 'i'. + libpq_append_conn_

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-01-09 Thread Jacob Champion
On Wed, Jan 8, 2025 at 5:17 PM Tom Lane wrote: > I think it might be safer if we only accepted it as a connection > parameter and not via an environment variable. Making it a connection parameter also keeps us from colliding with any other linked libraries' use of SSLKEYLOGFILE (I'm thinking abou

pg_dump, pg_dumpall, pg_restore: Add --no-policies option

2025-01-09 Thread Nikolay Samokhvalov
pg_dump[all] and pg_restore have a lot of "--no-XXX" options, I noticed there is no "--no-policies"; the patch implements it for pg_dump, pg_dumpall, and pg_restore. This can be useful in scenarios where policies need to be redefined in the target system or when moving data between environments w

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-01-09 Thread Alvaro Herrera
On 2025-Jan-09, Antonin Houska wrote: > It seems you accidentally fixed another problem :-) I was referring to the > 'lockmode' argument of make_new_heap(). I can try to write a patch for that > but ... > > > Meanwhile the patch 0004 has some seemingly trivial conflicts. If you > > want to rebas

Re: Psql meta-command conninfo+

2025-01-09 Thread David G. Johnston
On Thu, Jan 9, 2025 at 12:28 PM Sami Imseih wrote: > > v35 seems fine to me from a UI standpoint; I suggest we move forward > > with that. > > I am also OK with moving forward with a single \conninfo+, but > I think we should include all parameters in [1] as part of the output. > These are the pa

Re: Psql meta-command conninfo+

2025-01-09 Thread Sami Imseih
> v35 seems fine to me from a UI standpoint; I suggest we move forward > with that. I am also OK with moving forward with a single \conninfo+, but I think we should include all parameters in [1] as part of the output. These are the parameters the server reports back to the client. I think they are

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-09 Thread Jacob Champion
On Thu, Jan 9, 2025 at 8:17 AM Peter Eisentraut wrote: > > Maybe it would work to just use plain "int" as the type here. Any > socket number must fit into int anyway in order for PQsocket() to be > able to return it. The way I understand Windows socket handles, this > should work. Looks like it

Re: Sample rate added to pg_stat_statements

2025-01-09 Thread Alena Rybakina
Hi! Thank you for the work with this subject. I looked at your patch and noticed that this part of the code is repeated several times: if (nesting_level == 0)     {     if (!IsParallelWorker())     current_query_sampled = pg_prng_double(&pg_global_prng_state) < pgss_sample_rate;

Re: Psql meta-command conninfo+

2025-01-09 Thread Alvaro Herrera
On 2024-Dec-26, Maiquel Grassi wrote: > The previous version was good to go and ready for a commit as soon as > the final review is finalized. About David’s proposal, I’m still a > little unsure, but it seems like it has a lot of potential. What do > you all think? If it’s a strong direction to ex

Re: Fix a wrong errmsg in AlterRole()

2025-01-09 Thread Nathan Bossart
On Thu, Jan 09, 2025 at 10:13:11AM +0800, cca5507 wrote: > I modified the patch according to Tom's suggestion. LGTM. I'll plan on committing this shortly. -- nathan

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-01-09 Thread Nathan Bossart
On Thu, Jan 09, 2025 at 11:22:05AM +, devanga.susmi...@fujitsu.com wrote: > This email aims to discuss the contribution of optimized hex_encode and > hex_decode functions for ARM (aarch64) machines. These functions are > widely used for encoding and decoding binary data in the bytea data type.

Re: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-01-09 Thread Andres Freund
Hi, On 2025-01-07 15:46:26 -0500, Melanie Plageman wrote: > For table storage options, those related to vacuum but not autovacuum > are in the main StdRdOptions struct. Of those, some are overridden by > VACUUM command parameters which are parsed out into the VacuumParams > struct. Though the memb

Re: New GUC autovacuum_max_threshold ?

2025-01-09 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 09:32:58PM +, Vinícius Abrahão wrote: > Please also provide the tests on the new parameter you want to introduce. I skimmed around and didn't see any existing tests for these kinds of parameters, which of course isn't a great reason not to add tests, but it's also not c

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-09 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 06:07:44PM -0500, Tom Lane wrote: > Nathan Bossart writes: >> AFAICT TAS_SPIN() is only used for s_lock(), which doesn't appear to be >> used by LWLocks. But I did retry my test from upthread without >> pg_stat_statements and was surprised to find a reproducible 4-6% >> re

Re: New GUC autovacuum_max_threshold ?

2025-01-09 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 07:01:53PM -0500, Robert Treat wrote: > To be frank, this patch feels like a solution in search of a problem, > and as I read back through the thread, it isn't clear what problem > this is intended to fix. Thanks for sharing your thoughts. FWIW I've heard various strategie

Re: Non-text mode for pg_dumpall

2025-01-09 Thread Mahendra Singh Thalor
On Thu, 9 Jan 2025 at 08:11, Mahendra Singh Thalor wrote: > > On Thu, 9 Jan 2025 at 02:30, Guillaume Lelarge wrote: > > > > Hi, > > > > Le mer. 8 janv. 2025 à 17:41, Mahendra Singh Thalor a écrit : > >> > >> On Wed, 8 Jan 2025 at 20:07, Mahendra Singh Thalor wrote: > >> > > >> > Hi all, > >> >

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-01-09 Thread Antonin Houska
Alvaro Herrera wrote: > On 2024-Dec-11, Antonin Houska wrote: > > > Oh, it was too messy. I think I was thinking of too many things at once > > (such > > as locking the old heap, the new heap and the new heap's TOAST). Also, one > > thing that might have contributed to the confusion is that mak

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-01-09 Thread Pavel Stehule
Hi čt 9. 1. 2025 v 14:35 odesílatel Alvaro Herrera napsal: > On 2024-Dec-11, Antonin Houska wrote: > > > Oh, it was too messy. I think I was thinking of too many things at once > (such > > as locking the old heap, the new heap and the new heap's TOAST). Also, > one > > thing that might have cont

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-09 Thread Tomas Vondra
On 1/9/25 17:17, Melanie Plageman wrote: > On Tue, Dec 31, 2024 at 6:07 PM Tomas Vondra wrote: >> >> So I decided to revisit the three patches from 2019. Attached are >> rebased and cleaned up versions. A couple comments on each one: >> >> >> 1) v20241231-adjust-limit-0001-Account-for-batch-fil

Re: Psql meta-command conninfo+

2025-01-09 Thread Sami Imseih
> If we loop through conn->pstatus, we will be bypassing the official API. > The list is of type pgParameterStatus, which is an internal struct defined in > libpq-int.h. > As the file's header warns, including this file can lead to issues. Or am I > missing something? you're right about the warn

RE: Exporting float_to_shortest_decimal_buf(n) with Postgres 17 on Windows

2025-01-09 Thread Regina Obe
> Regina Obe писал(а) 2025-01-03 23:08: > > I tested this with a patched version of the EDB PG 17 installer that > > includes this patch and it fixed the issue I was having described in: > > > > https://www.postgresql.org/message- > id/01db1df8%2449765bb0%24dc6313 > > 10%24%40pcorp.us > > > > I

Re: AIO v2.0

2025-01-09 Thread Andres Freund
Hi, On 2025-01-09 10:59:22 +0200, Ants Aasma wrote: > On Wed, 8 Jan 2025 at 22:58, Andres Freund wrote: > > master: ~18 GB/s > > patch, buffered:~20 GB/s > > patch, direct, worker: ~28 GB/s > > patch, direct, uring: ~35 GB/s > > > > > > This was with io_workers=32, io_m

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-09 Thread Peter Eisentraut
On 09.01.25 02:45, Melanie Plageman wrote: On Wed, Jan 8, 2025 at 8:39 AM Peter Eisentraut wrote: On 07.01.25 18:31, Melanie Plageman wrote: Oh, one thing I forgot to say. Though I increased the indentation of some of the subsections that I moved, I didn't rewrap the lines because they were

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-09 Thread Melanie Plageman
On Tue, Dec 31, 2024 at 6:07 PM Tomas Vondra wrote: > > So I decided to revisit the three patches from 2019. Attached are > rebased and cleaned up versions. A couple comments on each one: > > > 1) v20241231-adjust-limit-0001-Account-for-batch-files-in-ha.patch > > I believe this is the way to go,

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-09 Thread Peter Eisentraut
On 08.01.25 21:29, Jacob Champion wrote: It'd also couple clients against libpq-int.h, so they'd have to remember to recompile every release. I'm worried that'd cause a bunch of ABI problems... Couldn't that function use PQsocket() to get at the actual socket from the PGconn handle? It's an out

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-09 Thread Zhou, Zhiguo
On 1/7/2025 10:49 AM, Юрий Соколов wrote: On 6 Jan 2025, at 09:46, Zhou, Zhiguo wrote: Hi Yura and Wenhui, Thanks for kindly reviewing this work! On 1/3/2025 9:01 PM, wenhui qiu wrote: Hi Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I think it will be challenged,do we make

Re: Make pg_stat_io view count IOs as bytes instead of blocks

2025-01-09 Thread Bertrand Drouvot
Hi, On Thu, Jan 09, 2025 at 02:20:16PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > On Thu, 9 Jan 2025 at 11:11, Michael Paquier wrote: > > > > On Thu, Jan 09, 2025 at 10:15:20AM +0300, Nazir Bilal Yavuz wrote: > > > I am a bit confused, are you suggesting these two alternatives: > > > 1- Making pg

Re: SCRAM pass-through authentication for postgres_fdw

2025-01-09 Thread Matheus Alcantara
Thanks for the review! Em qua., 8 de jan. de 2025 às 07:49, Peter Eisentraut escreveu: > > This patch is surprisingly compact and straightforward for providing > such complex functionality. > > I have one major code comment that needs addressing: > > In src/interfaces/libpq/fe-auth-scram.c, there

Re: pure parsers and reentrant scanners

2025-01-09 Thread Tom Lane
Peter Eisentraut writes: > The second patch contemplates raising the minimum required flex version, > but what to? Meh, let's just rip out the version check. It's no longer very relevant. Nobody is going to be using anything older than 2.5.35. While 2.5.35 produces compile warnings, it does st

Re: Sample rate added to pg_stat_statements

2025-01-09 Thread Ilia Evdokimov
On 09.01.2025 05:29, Sami Imseih wrote: Unfortunately, these changes do not achieve the intended sampling goal. I looked into this more deeply: while the sampled-out queries do not appear in pg_stat_statements, an entry is still allocated in the hash table after normalization, which, in my view

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-09 Thread Nazir Bilal Yavuz
Hi, Thanks for working on this! On Wed, 8 Jan 2025 at 22:26, Andres Freund wrote: > > I'm currently to plan the four patches relatively soon, unless somebody speaks > up, of course. They seem fairly uncontroversial. The renaming of the phases > doesn't need to wait much longer, I think. > > The

Re: Some ExecSeqScan optimizations

2025-01-09 Thread Amit Langote
On Mon, Jan 6, 2025 at 10:18 PM David Rowley wrote: > On Sat, 21 Dec 2024 at 00:41, Amit Langote wrote: > > To address (1), I tried assigning specialized functions to > > PlanState.ExecProcNode in ExecInitSeqScan() based on whether qual or > > projInfo are NULL. Inspired by David Rowley’s suggest

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-01-09 Thread Alvaro Herrera
On 2024-Dec-11, Antonin Houska wrote: > Oh, it was too messy. I think I was thinking of too many things at once (such > as locking the old heap, the new heap and the new heap's TOAST). Also, one > thing that might have contributed to the confusion is that make_new_heap() has > the 'lockmode' argum

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-01-09 Thread Alena Rybakina
Hi! On 04.10.2024 12:05, Andrei Lepikhov wrote: We also have an implementation of VALUES -> ARRAY transformation. Because enterprises must deal with users' problems, many of these users employ automatically generated queries. Being informed very well of the consensus about that stuff, we've de

Re: Virtual generated columns

2025-01-09 Thread Dean Rasheed
On Wed, 8 Jan 2025 at 16:14, Peter Eisentraut wrote: > > One thing I could use some review on is the access control handling and > security in general. You can create virtual generated columns that have > their own access privileges but which can read columns that the user > does not have access

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-09 Thread Daniel Gustafsson
> On 9 Jan 2025, at 02:30, Melanie Plageman wrote: > > On Wed, Jan 8, 2025 at 6:35 AM Daniel Gustafsson wrote: >> >>> On 7 Jan 2025, at 21:14, Tom Lane wrote: >> >>> I might be wrong, but I had the idea that our docs website has a >>> capability to provide such redirects. You'd probably need

Re: add support for the old naming libs convention on windows (ssleay32.lib and libeay32.lib)

2025-01-09 Thread Daniel Gustafsson
> On 7 Jan 2025, at 19:25, Darek Ślusarczyk wrote: > Thanks for reviewing the patch - I've added a few changes and it looks as > follows: Thanks for the new version, I've tested it in CI against v17 and plan to commit it soon. -- Daniel Gustafsson

Re: Virtual generated columns

2025-01-09 Thread jian he
On Thu, Jan 9, 2025 at 3:28 AM Peter Eisentraut wrote: > > On 08.01.25 17:38, Tom Lane wrote: > > Peter Eisentraut writes: > >> On 03.12.24 15:15, jian he wrote: > >>> SELECT attrelid, attname, attgenerated FROM pg_attribute WHERE > >>> attgenerated IN ('v') and (attnotnull or not atthasdef); > >

Re: ecpg command does not warn COPY ... FROM STDIN;

2025-01-09 Thread Ryo Kanbayashi
> On 2025/01/09 20:34, Ryo Kanbayashi wrote: > > Dear Tom, Fujii-san, Kuroda-san, > > > > I saw comments of yours and recognized that better fix is below. > > > > - Fix of first attached patch which does not change warning message > > > > And I created patch entry of commitfest :) > > https://commi

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-01-09 Thread Daniel Gustafsson
> On 9 Jan 2025, at 02:17, Tom Lane wrote: > I think it might be safer if we only accepted it as a connection > parameter and not via an environment variable. +1 -- Daniel Gustafsson

Re: Small refactoring around vacuum_open_relation

2025-01-09 Thread Daniel Gustafsson
> On 9 Jan 2025, at 11:45, Alvaro Herrera wrote: > Dunno what others think, this seems useless churn to me. I agree, I don't see this providing enough value to warrant the changes. -- Daniel Gustafsson

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-09 Thread Bertrand Drouvot
Hi, On Wed, Jan 08, 2025 at 02:26:15PM -0500, Andres Freund wrote: > I'm currently to plan the four patches relatively soon, unless somebody speaks > up, of course. They seem fairly uncontroversial. The renaming of the phases > doesn't need to wait much longer, I think. Thanks for the patches.

Re: ecpg command does not warn COPY ... FROM STDIN;

2025-01-09 Thread Fujii Masao
On 2025/01/09 20:34, Ryo Kanbayashi wrote: Dear Tom, Fujii-san, Kuroda-san, I saw comments of yours and recognized that better fix is below. - Fix of first attached patch which does not change warning message And I created patch entry of commitfest :) https://commitfest.postgresql.org/52/54

Re: Several buildfarm animals fail tests because of shared memory error

2025-01-09 Thread Robins Tharakan
Hi Alexander, On Sun, 22 Dec 2024 at 17:57, Robins Tharakan wrote: > > So if that's a key reason here, then probably by this time next week things should > settle down. I've begun setting it correctly (2 done with a few more to go) - although > given that some machines are at work, I'll try

Re: ecpg command does not warn COPY ... FROM STDIN;

2025-01-09 Thread Ryo Kanbayashi
Dear Tom, Fujii-san, Kuroda-san, I saw comments of yours and recognized that better fix is below. - Fix of first attached patch which does not change warning message And I created patch entry of commitfest :) https://commitfest.postgresql.org/52/5497/ What should I do additionally? Do I need to

Re: Exporting float_to_shortest_decimal_buf(n) with Postgres 17 on Windows

2025-01-09 Thread Vladlen Popolitov
Regina Obe писал(а) 2025-01-03 23:08: I tested this with a patched version of the EDB PG 17 installer that includes this patch and it fixed the issue I was having described in: https://www.postgresql.org/message-id/01db1df8%2449765bb0%24dc631310%24%40pcorp.us It would be great if this was

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-09 Thread Ashutosh Bapat
On Tue, Dec 31, 2024 at 10:15 AM Masahiko Sawada wrote: > > Hi all, > > Logical decoding (and logical replication) are available only when > wal_level = logical. As the documentation says[1], Using the 'logical' > level increases the WAL volume which could negatively affect the > performance. For

Re: Make pg_stat_io view count IOs as bytes instead of blocks

2025-01-09 Thread Nazir Bilal Yavuz
Hi, On Thu, 9 Jan 2025 at 11:11, Michael Paquier wrote: > > On Thu, Jan 09, 2025 at 10:15:20AM +0300, Nazir Bilal Yavuz wrote: > > I am a bit confused, are you suggesting these two alternatives: > > 1- Making pgstat_count_io_op_n() static and continuing to use > > pgstat_count_io_op() as it is. >

Re: Windows meson build

2025-01-09 Thread Vladlen Popolitov
Kohei Harikae (Fujitsu) писал(а) 2025-01-06 16:57: Hi, - 0001 icu - 0002 libxml - 0003 libxslt - 0004 lz4 - 0005 tcl - 0006 zlib - 0007 zstd These libraries could not be built by setting extra_include_dirs and extra_lib_dirs. These libraries may not have .pc file created. For example, if I

Re: Small refactoring around vacuum_open_relation

2025-01-09 Thread Alvaro Herrera
On 2025-Jan-09, Ashutosh Bapat wrote: > Sorry for reviewing late. The patch looks ok. Dunno what others think, this seems useless churn to me. > I found some more > static const struct > { > LOCKMODE hwlock; > int lockstatus; > int updstatus; > } > > tupleLockExtraInfo[MaxLockTupleMode + 1] = >

Re: Small refactoring around vacuum_open_relation

2025-01-09 Thread Ashutosh Bapat
On Fri, Aug 2, 2024 at 4:00 PM Kirill Reshke wrote: > > Thanks for review! > > On Fri, 2 Aug 2024 at 14:31, Ashutosh Bapat > wrote: > > > > On Fri, Aug 2, 2024 at 1:55 PM Kirill Reshke wrote: > > > > > > I hate to be "that guy", but there are many places in sources where we use > > > LOCKMODE lo

Re: Recovering from detoast-related catcache invalidations

2025-01-09 Thread Heikki Linnakangas
On 07/01/2025 23:56, Noah Misch wrote: On Tue, Dec 24, 2024 at 12:18:09AM +0200, Heikki Linnakangas wrote: I'm thinking of the attached to fix this. It changes the strategy for detecting concurrent cache invalidations. Instead of the "recheck" mechanism that was introduced in commit ad98fb1422,

Re: AIO v2.0

2025-01-09 Thread Ants Aasma
On Wed, 8 Jan 2025 at 22:58, Andres Freund wrote: > master: ~18 GB/s > patch, buffered:~20 GB/s > patch, direct, worker: ~28 GB/s > patch, direct, uring: ~35 GB/s > > > This was with io_workers=32, io_max_concurrency=128, > effective_io_concurrency=1000 (doesn't need to

Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

2025-01-09 Thread Jakub Wartak
On Thu, Jan 9, 2025 at 4:12 AM Thomas Munro wrote: > > On Tue, Jan 7, 2025 at 9:54 AM Thomas Munro wrote: > > On Tue, Jan 7, 2025 at 5:23 AM Andrew Dunstan wrote: > > > Do you have a plan for moving ahead with this? > > > > I think that all looks good, and I'll go ahead and commit it in the > >

Re: Eager aggregation, take 3

2025-01-09 Thread Richard Guo
On Thu, Jan 9, 2025 at 12:15 PM jian he wrote: > hi. > in create_grouping_expr_infos > > tce = lookup_type_cache(exprType((Node *) tle->expr), > TYPECACHE_BTREE_OPFAMILY); > if (!OidIsValid(tce->btree_opf) || > !OidIsValid(tce->btree_opin

Re: Log a warning in pg_createsubscriber for max_slot_wal_keep_size

2025-01-09 Thread Shubham Khanna
On Thu, Jan 9, 2025 at 12:46 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for updating the patch. Few comments. > > ``` > +#include "common/int.h"/* include for strtoi64 */ > ``` > > I could build the source code without the inclusion. Can you remove? > >

RE: Conflict detection for update_deleted in logical replication

2025-01-09 Thread Zhijie Hou (Fujitsu)
On Wednesday, January 8, 2025 3:49 PM Nisha Moond wrote: > > On Tue, Jan 7, 2025 at 6:04 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > Attached the V19 patch which addressed comments in [1][2][3][4][5][6][7]. > > > > Here are a couple of initial review comments on v19 patch set: > > 1) The sub

  1   2   >