Re: pg_dump, pg_dumpall, pg_restore: Add --no-policies option

2025-01-14 Thread jian he
hi. around src/bin/pg_dump/pg_dump.c line 1117 right after "ropt->no_comments = dopt.no_comments;" we also need add ropt->no_policies = dopt.no_policies; ? overall looks good to me. The tests seem wrong per https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F5499 I have no idea how

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-01-14 Thread Tom Lane
John Naylor writes: > Okay, I added a comment. I also agree with Michael that my quick > one-off was a bit hard to read so I've cleaned it up a bit. I plan to > commit the attached by Friday, along with any bikeshedding that > happens by then. Couple of thoughts: 1. I was actually hoping for a c

Re: Virtual generated columns

2025-01-14 Thread vignesh C
On Tue, 14 Jan 2025 at 19:08, Peter Eisentraut wrote: > > > I've also added a patch that addresses logical replication. It > basically adds back some of the prohibitions against including generated > columns in publications that have been lifted, but this time only for > virtual generated columns

Re: Eager aggregation, take 3

2025-01-14 Thread Richard Guo
On Wed, Jan 15, 2025 at 12:07 AM Robert Haas wrote: > On Sun, Jan 12, 2025 at 9:04 PM Richard Guo wrote: > > Attached is an updated version of this patch that addresses Jian's > > review comments, along with some more cosmetic tweaks. I'm going to > > be looking at this patch again from the poin

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-01-14 Thread John Naylor
On Tue, Jan 14, 2025 at 11:57 PM Nathan Bossart wrote: > > On Tue, Jan 14, 2025 at 12:59:04AM -0500, Tom Lane wrote: > > John Naylor writes: > >> We can do about as well simply by changing the nibble lookup to a byte > >> lookup, which works on every compiler and architecture: > > Nice. I tried

Re: Skip collecting decoded changes of already-aborted transactions

2025-01-14 Thread Amit Kapila
On Wed, Jan 15, 2025 at 3:11 AM Masahiko Sawada wrote: > > It seems we agreed on RBTXN_IS_PREPARED and rbtxn_is_prepared(). > Adding 'IS' seems to clarify the transaction having this flag *is* a > prepared transaction. Both other two constants RBTXN_SENT_PREAPRE and > RBTXN_SKIPPED_PREPARE seem no

Re: Issue with markers in isolation tester? Or not?

2025-01-14 Thread Tom Lane
Michael Paquier writes: > On Tue, Jan 14, 2025 at 11:48:28AM -0800, Noah Misch wrote: >> I misunderstood, and I was mistaken to see this as a bug fix. The >> isolationtester is acting per its definition, and this would be a definition >> change. Do others have opinions on the merits of today's d

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2025-01-14 Thread Shubham Khanna
On Tue, Jan 14, 2025 at 4:53 PM Shlok Kyal wrote: > > On Fri, 27 Dec 2024 at 12:06, Shubham Khanna > wrote: > > > > On Fri, Dec 27, 2024 at 11:30 AM vignesh C wrote: > > > > > > > > > > The documentation requires a minor update: instead of specifying > > > subscriptions, the user will specify mu

Re: Pgoutput not capturing the generated columns

2025-01-14 Thread Peter Smith
On Mon, Jan 13, 2025 at 8:27 PM Amit Kapila wrote: > > On Mon, Jan 13, 2025 at 5:25 AM Peter Smith wrote: > > > > Future -- there probably need to be further clarifications/emphasis to > > describe how the generated column replication feature only works for > > STORED generated columns (not VIRTU

Re: per backend WAL statistics

2025-01-14 Thread Michael Paquier
On Fri, Jan 10, 2025 at 09:40:38AM +, Bertrand Drouvot wrote: > Please find attached v4 taking into account 2c14037bb5. +} PgStat_WalCounters; + +typedef struct PgStat_WalStats +{ + PgStat_WalCounters wal_counters; I know that's a nit, but perhaps that could be a patch of its own, pushi

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-14 Thread Shlok Kyal
On Thu, 2 Jan 2025 at 15:57, Nisha Moond wrote: > > On Thu, Jan 2, 2025 at 8:16 AM Peter Smith wrote: > > > > Hi Nisha, > > > > Here are some minor review comments for patch v58-0002. > > > > Thank you for your feedback! Please find the v59 patch set addressing > all the comments. > Note: There a

Re: Pgoutput not capturing the generated columns

2025-01-14 Thread Peter Smith
Hi Vignesh. Some review comments for patch 0001 == GENERAL 1. AFAIK there are still many places in the docs where there is no distinction made between the stored/virtual generated cols. Maybe we need another patch in this patch set to address all of these? For example, CREATE PUBLICATION sa

Re: Issue with markers in isolation tester? Or not?

2025-01-14 Thread Michael Paquier
On Tue, Jan 14, 2025 at 11:48:28AM -0800, Noah Misch wrote: > I misunderstood, and I was mistaken to see this as a bug fix. The > isolationtester is acting per its definition, and this would be a definition > change. Do others have opinions on the merits of today's definition vs. the > proposed d

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-14 Thread Ashutosh Bapat
On Tue, Jan 14, 2025 at 3:15 AM Masahiko Sawada wrote: > > On Mon, Jan 13, 2025 at 1:31 AM Ashutosh Bapat > wrote: > > > > On Mon, Jan 13, 2025 at 2:52 PM vignesh C wrote: > > > > > > I felt that the only disadvantage with this approach is that we > > > currently wait for all in-progress transac

Re: refactor AlterDomainAddConstraint (alter domain add constraint)

2025-01-14 Thread jian he
On Wed, Jan 15, 2025 at 12:37 AM Alvaro Herrera wrote: > > Hello, > > On 2024-Dec-09, jian he wrote: > > > ALTER DOMAIN ADD CONSTRAINT syntax more simple than CREATE DOMAIN. > > Your proposed patch makes the code simpler, yes, but I think it also > makes the error messages worse. I don't think th

Re: Allow NOT VALID foreign key constraints on partitioned tables.

2025-01-14 Thread Amul Sul
On Mon, Jan 6, 2025 at 9:53 AM Amul Sul wrote: > I made the minor changes to the attached version and rebased it against the latest master(9a45a89c38f). Regards, Amul v2-0001-Refactor-Split-ATExecValidateConstraint.patch Description: Binary data v2-0002-Allow-NOT-VALID-foreign-key-constraint

Re: Accept recovery conflict interrupt on blocked writing

2025-01-14 Thread Thomas Munro
Bonjour Anthonin, On Mon, Jan 13, 2025 at 11:31 PM Anthonin Bonnefoy wrote: > To avoid blocking recovery for an extended period of time, this patch > changes client write interrupts by handling recovery conflict > interrupts instead of ignoring them. Since the interrupt happens while > we're like

Re: CREATE TABLE NOT VALID for check and foreign key

2025-01-14 Thread Amul Sul
On Thu, Dec 5, 2024 at 3:06 PM Alvaro Herrera wrote: > > Hello, > > On 2024-Dec-05, jian he wrote: > > > I found for foreign keys, check constraints, > > you specify it as NOT VALID, it will not be marked as NOT VALID in the > > CREATE TABLE statement. > > Uhmm, okay. > > > reading transformCheckC

Re: Infinite loop in XLogPageRead() on standby

2025-01-14 Thread Michael Paquier
On Wed, Dec 25, 2024 at 12:00:59PM +0900, Michael Paquier wrote: > All of them refer to an infinite loop reachable in the startup process > when we read an incorrect incomplete record just after a failover or > when a WAL receiver restarts. Not sure which way is best in order to > fix all of them

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-14 Thread vignesh C
On Tue, 14 Jan 2025 at 03:03, Masahiko Sawada wrote: > > Yeah, delegating the activation to the background process such as the > checkpointer would also be one solution. This would work with the > approach that we enable the logical decoding via > pg_activate_logical_decoding(). On the other hand,

Re: Conflict detection for update_deleted in logical replication

2025-01-14 Thread Amit Kapila
On Wed, Jan 15, 2025 at 5:57 AM Masahiko Sawada wrote: > > On Mon, Jan 13, 2025 at 8:39 PM Amit Kapila wrote: > > > > As of now, I can't think of a way to throttle the publisher when the > > apply_worker lags. Basically, we need some way to throttle (reduce the > > speed of backends) when the app

Re: convert libpgport's pqsignal() to a void function

2025-01-14 Thread Tom Lane
Nathan Bossart writes: > On Tue, Jan 14, 2025 at 10:02:46PM -0500, Tom Lane wrote: >> LGTM, although I don't know enough about Windows to know if the >> "== SIG_ERR" test in that path is correct. > It's apparently not [0]. :( Bleah. > My guess is that this has something to do with redefining S

Re: convert libpgport's pqsignal() to a void function

2025-01-14 Thread Nathan Bossart
On Tue, Jan 14, 2025 at 10:02:46PM -0500, Tom Lane wrote: > LGTM, although I don't know enough about Windows to know if the > "== SIG_ERR" test in that path is correct. It's apparently not [0]. :( My guess is that this has something to do with redefining SIG_ERR in win32_port.h. We might be abl

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-14 Thread vignesh C
On Mon, 13 Jan 2025 at 12:48, Peter Smith wrote: > > On Mon, Jan 13, 2025 at 5:52 PM vignesh C wrote: > > > > On Thu, 2 Jan 2025 at 15:57, Nisha Moond wrote: > > > > > > On Thu, Jan 2, 2025 at 8:16 AM Peter Smith wrote: > > > > > > > > Hi Nisha, > > > > > > > > Here are some minor review commen

Re: convert libpgport's pqsignal() to a void function

2025-01-14 Thread Tom Lane
Nathan Bossart writes: > Thanks to commit 9a45a89, legacy-pqsignal.c now has its own dedicated > extern for pqsignal(), which decouples it enough that we can follow through > with changing libpqport's pqsignal() to a void function. > Thoughts? LGTM, although I don't know enough about Windows to

convert libpgport's pqsignal() to a void function

2025-01-14 Thread Nathan Bossart
(moving to a new thread) On Mon, Jan 13, 2025 at 10:04:31AM -0600, Nathan Bossart wrote: > On Sat, Jan 11, 2025 at 02:04:13PM -0500, Tom Lane wrote: >> BTW, this decouples legacy-pqsignal.c from pqsignal.c enough >> that we could now do what's contemplated in the comments from >> 3b00fdba9: simpli

Re: Issue with markers in isolation tester? Or not?

2025-01-14 Thread Michail Nikolaev
Hello, Noah! > I misunderstood, and I was mistaken to see this as a bug fix. The > isolationtester is acting per its definition, and this would be a definition > change. Yes, you are right, but I think it is better to clarify it somehow because from my point of view that definition feels like

Re: Conflict detection for update_deleted in logical replication

2025-01-14 Thread Masahiko Sawada
On Mon, Jan 13, 2025 at 8:39 PM Amit Kapila wrote: > > On Tue, Jan 14, 2025 at 7:14 AM Masahiko Sawada wrote: > > > > On Sun, Jan 12, 2025 at 10:36 PM Amit Kapila > > wrote: > > > > > > I don't think we can avoid accumulating garbage especially when the > > > workload on the publisher is more.

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-14 Thread Tom Lane
I wrote: > OK, I'll make it so. Thanks for looking at it! Or not. My idea worked okay in v17, but not in older branches. Pre-v17, libpq itself can call pqsignal (though only in non- thread-safe builds). With this patch, that would have resulted in pulling src/port/pqsignal.o into libpq. libpq

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-14 Thread Andres Freund
Hi, On 2025-01-14 13:06:31 +, Bertrand Drouvot wrote: > On Tue, Jan 14, 2025 at 12:58:44AM -0500, Andres Freund wrote: > > The current code has the weird behaviour of going through PM_WAIT_BACKENDS. > > I > > left it like that for now. In fact more paths do so now, because we did so > > for

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-14 Thread Sami Imseih
thanks for the review! > The comments do not reflect the function names ("total" is missing to give > pg_stat_get_total_vacuum_time() and such). fixed > I think it's better to define the macro just before its first usage (meaning > just after pg_stat_get_vacuum_count()): that would be consistent

Re: Sample rate added to pg_stat_statements

2025-01-14 Thread Sami Imseih
> Alena, Sami – I apologize for not including you in the previous email. > If you're interested in this approach, I'm open to any suggestions. > > [0]: > https://www.postgresql.org/message-id/1b13d748-5e98-479c-9222-3253a734a038%40tantorlabs.com Here are my thoughts on this: There is good reason

Re: Log a warning in pg_createsubscriber for max_slot_wal_keep_size

2025-01-14 Thread Peter Smith
Hi Shubham. Patch v9-0001 LGTM. == Kind Regards, Peter Smith. Fujitsu Australia

Re: Proposal: Progressive explain

2025-01-14 Thread Adrien Nayrat
On 12/30/24 2:18 AM, Rafael Thofehrn Castro wrote: Hello community, CONTEXT: Back in October I presented the talk "Debugging active queries with mid-flight instrumented explain plans" at PGConf EU 2024 (recording: https://www.youtube.com/watch?v=6ahTb-7C05c) presenting an experimental feature t

Re: Skip collecting decoded changes of already-aborted transactions

2025-01-14 Thread Masahiko Sawada
On Mon, Jan 13, 2025 at 8:48 PM Amit Kapila wrote: > > On Tue, Jan 14, 2025 at 7:32 AM Peter Smith wrote: > > > > Hi Sawada-San. > > > > Some review comments for patch v13-0002. > > > > == > > > > I think the v12 ambiguity of RBTXN_PREPARE versus RBTXN_SENT_PREPARE > > was mostly addressed al

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-14 Thread Tom Lane
Nathan Bossart writes: > On Tue, Jan 14, 2025 at 02:52:29PM -0500, Tom Lane wrote: >> So now I'm inclined to include the ABI-compatible wrapper, which >> will ensure that extensions continue to link to libpgport's pqsignal. > Fine by me. OK, I'll make it so. Thanks for looking at it!

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-14 Thread Nathan Bossart
On Tue, Jan 14, 2025 at 02:52:29PM -0500, Tom Lane wrote: > After more thought I've realized that the asymmetrical detection > here isn't all that bad, because the outcomes are different. > If we fail to catch old-headers-and-new-library, the result will > either be a link failure or (if the extens

Re: pgsql: Consolidate docs for vacuum-related GUCs in new subsection

2025-01-14 Thread Melanie Plageman
On Tue, Jan 14, 2025 at 2:30 PM Alena Rybakina wrote: > > Looking at them, I am willing to agree with you Cool. Thanks to everyone for the review. I've pushed it. - Melanie

Re: New GUC autovacuum_max_threshold ?

2025-01-14 Thread Alena Rybakina
Hi! Thanks for your work! it is very useful. On 14.01.2025 05:09, Nathan Bossart wrote: On Mon, Jan 13, 2025 at 05:17:11PM -0600, Sami Imseih wrote: I propose renaming the GUC from "autovacuum_max_threshold" to "autovacuum_vacuum_max_threshold" to clarify that it applies only to the vacuum oper

Re: Skip collecting decoded changes of already-aborted transactions

2025-01-14 Thread Masahiko Sawada
On Mon, Jan 13, 2025 at 5:36 PM Peter Smith wrote: > > Hi Sawada-San. Here are some cosmetic review comments for the patch v13-0001. Thank you for reviewing the patch. > > == > Commit message > > 1. > This commit introduces an additional CLOG lookup to check the > transaction status, so the

Re: RFC: Additional Directory for Extensions

2025-01-14 Thread David E. Wheeler
Hello Peter & Co. On Dec 5, 2024, at 06:07, Peter Eisentraut wrote: > I've made a bit of progress on this patch, filled in some documentation and > resolved some TODO markers. Also: Finally getting around to reviewing this patch. Should it be considered part of the previous patch[1] for pur

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-14 Thread Tom Lane
Nathan Bossart writes: > On Mon, Jan 13, 2025 at 05:51:54PM -0500, Tom Lane wrote: >> (plus or minus an extern or so, but you get the idea). The point of >> this is that compiling against old headers and then linking against >> newer libpgport.a would still work. It does nothing however for the

Re: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-01-14 Thread Melanie Plageman
On Mon, Jan 13, 2025 at 5:37 PM Alena Rybakina wrote: > > Thank you for working on this patch, without this explanation it is difficult > to understand what is happening, to put it mildly. Thanks for the review! I've incorporated most of them into attached v7. > The first of them is related to

Re: Issue with markers in isolation tester? Or not?

2025-01-14 Thread Noah Misch
On Tue, Jan 14, 2025 at 12:16:22PM +0100, Michail Nikolaev wrote: > --- a/src/test/isolation/README > +++ b/src/test/isolation/README > @@ -183,9 +183,9 @@ A marker consisting solely of a step name indicates that > this step may > not be reported as completing until that other step has completed.

Re: New GUC autovacuum_max_threshold ?

2025-01-14 Thread Sami Imseih
Will just park the idea for the documentation here. If you feel this should be in a follow-up patch, I am ok with that and will follow-up on it afterwards. +++ b/doc/src/sgml/maintenance.sgml @@ -905,6 +905,12 @@ vacuum threshold = Minimum(vacuum max threshold, vacuum base threshold + vacuum

Re: Reduce TupleHashEntryData struct size by half

2025-01-14 Thread David Rowley
On Wed, 15 Jan 2025 at 07:47, Jeff Davis wrote: > > On Tue, 2025-01-14 at 22:01 +1300, David Rowley wrote: > > The trick would be to ensure ExecClearTuple() still works. > > I'm confused by this. The comment over copy_minimal_slot says: I must have confused TupleTableSlotOps.copy_minimal_tuple wi

Re: New GUC autovacuum_max_threshold ?

2025-01-14 Thread Robert Haas
On Wed, Nov 13, 2024 at 5:03 AM Frédéric Yhuel wrote: > Let's compare the current situation to the situation post-Nathan's-patch > with a cap of 100M. Consider a table 100 times larger than the one of > Robert's previous example, so pgbench scale factor 2_560_000, size on > disk 32TB. This is a g

Re: pgsql: Consolidate docs for vacuum-related GUCs in new subsection

2025-01-14 Thread Alena Rybakina
On 14.01.2025 22:01, Melanie Plageman wrote: On Tue, Jan 14, 2025 at 1:21 PM Alvaro Herrera wrote: On 2025-Jan-13, Melanie Plageman wrote: I've gone with VACUUM_AUTOVACUUM, VACUUM_COST_DELAY, and VACUUM_FREEZING, but I am open to feedback. Looks good to me. I checked these two queries, whos

Re: pgsql: Consolidate docs for vacuum-related GUCs in new subsection

2025-01-14 Thread Alvaro Herrera
On 2025-Jan-14, Melanie Plageman wrote: > On Tue, Jan 14, 2025 at 1:21 PM Alvaro Herrera > wrote: > > 55432 18devel 560655=# select name, category from pg_settings where > > (short_desc ilike '%vacuum%' or extra_desc ilike '%vacuum%') and category > > not ilike '%vacuum%'; > > nam

Re: pgsql: Consolidate docs for vacuum-related GUCs in new subsection

2025-01-14 Thread Melanie Plageman
On Tue, Jan 14, 2025 at 1:21 PM Alvaro Herrera wrote: > > On 2025-Jan-13, Melanie Plageman wrote: > > > I've gone with VACUUM_AUTOVACUUM, VACUUM_COST_DELAY, and > > VACUUM_FREEZING, but I am open to feedback. > > Looks good to me. I checked these two queries, whose results appear > correct: > > 5

Re: Reduce TupleHashEntryData struct size by half

2025-01-14 Thread Jeff Davis
On Tue, 2025-01-14 at 22:01 +1300, David Rowley wrote: > The trick would be to ensure ExecClearTuple() still works. I'm confused by this. The comment over copy_minimal_slot says: /* * Return a copy of minimal tuple representing the contents of the slot. * The copy needs to be palloc'd in the cu

Re: Reduce TupleHashEntryData struct size by half

2025-01-14 Thread Alvaro Herrera
On 2025-Jan-14, David Rowley wrote: > The trick would be to ensure ExecClearTuple() still works. You > obviously don't want to try and pfree() something that isn't a pointer > to a palloc'd chunk. I'm not sure what the best API is, but I do see > there are other places that might benefit from some

Re: pgsql: Consolidate docs for vacuum-related GUCs in new subsection

2025-01-14 Thread Alvaro Herrera
On 2025-Jan-13, Melanie Plageman wrote: > I've gone with VACUUM_AUTOVACUUM, VACUUM_COST_DELAY, and > VACUUM_FREEZING, but I am open to feedback. Looks good to me. I checked these two queries, whose results appear correct: 55432 18devel 560655=# select name, category from pg_settings where categ

Re: New GUC autovacuum_max_threshold ?

2025-01-14 Thread Sami Imseih
On Tue, Jan 14, 2025 at 11:45 AM Nathan Bossart wrote: > > On Tue, Jan 14, 2025 at 11:08:36AM -0600, Sami Imseih wrote: > > After staring at the documentation for a while, I am now > > wondering whether we are adequately describing the > > rationale for this GUC. The GUC documentation mentions tha

Re: CREATE TABLE NOT VALID for check and foreign key

2025-01-14 Thread Alvaro Herrera
On 2025-Jan-08, Alvaro Herrera wrote: > On 2025-Jan-07, Yasuo Honda wrote: > > > I'd like PostgreSQL to raise errors and/or warnings for the NOT VALID > > check constraint for CREATE TABLE. > > Ruby on Rails supports creating check constraints with the NOT VALID > > option and I was not aware tha

Re: New GUC autovacuum_max_threshold ?

2025-01-14 Thread Nathan Bossart
On Tue, Jan 14, 2025 at 11:08:36AM -0600, Sami Imseih wrote: > After staring at the documentation for a while, I am now > wondering whether we are adequately describing the > rationale for this GUC. The GUC documentation mentions that this is a > 'cap on the value calculated with autovacuum_vacuum_

Re: Bypassing cursors in postgres_fdw to enable parallel plans

2025-01-14 Thread Robert Haas
On Mon, Jan 6, 2025 at 3:52 AM Rafia Sabih wrote: > Now, to overcome this limitation, I have worked on this idea (suggested by my > colleague Bernd Helmle) of bypassing the cursors. The way it works is as > follows, > there is a new GUC introduced postgres_fdw.use_cursor, which when unset uses

Re: Adding extension default version to \dx

2025-01-14 Thread Yugo Nagata
On Fri, 10 Jan 2025 20:37:17 +0800 Julien Rouhaud wrote: > Hi, > > On Fri, Jan 10, 2025 at 01:04:14PM +0100, Magnus Hagander wrote: > > In an effort to make at least a couple of more people realize they have to > > run ALTER EXTENSION UPDATE after they've upgraded an extension, as well as > > m

Re: psql: Option to use expanded mode for various meta-commands

2025-01-14 Thread Dean Rasheed
On Thu, 9 Jan 2025 at 03:18, Greg Sabino Mullane wrote: > > On Wed, Jan 8, 2025 at 8:44 AM Dean Rasheed wrote: >> >> Attached is a more complete patch > > +1, looks good > Thanks for looking. I've pushed this now. (I realised that I had missed \lo_list, so I added support for that too, since it

Re: New GUC autovacuum_max_threshold ?

2025-01-14 Thread Sami Imseih
> Good call. Here is an updated patch. thanks for the update! After staring at the documentation for a while, I am now wondering whether we are adequately describing the rationale for this GUC. The GUC documentation mentions that this is a 'cap on the value calculated with autovacuum_vacuum_thre

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-01-14 Thread Nathan Bossart
On Tue, Jan 14, 2025 at 12:59:04AM -0500, Tom Lane wrote: > John Naylor writes: >> We can do about as well simply by changing the nibble lookup to a byte >> lookup, which works on every compiler and architecture: Nice. I tried enabling auto-vectorization and loop unrolling on top of this patch,

Re: Allow ILIKE forward matching to use btree index

2025-01-14 Thread Yugo NAGATA
On Tue, 24 Dec 2024 16:04:42 +0900 Yugo Nagata wrote: > On Fri, 20 Dec 2024 03:22:26 +0900 > Yugo Nagata wrote: > > > Hi, > > > > Currently, btree indexes cannot used for ILIKE (~~*) operator if the pattern > > has case-varying characters although LIKE (~~) expression can be converted > > to i

Re: refactor AlterDomainAddConstraint (alter domain add constraint)

2025-01-14 Thread Alvaro Herrera
Hello, On 2024-Dec-09, jian he wrote: > ALTER DOMAIN ADD CONSTRAINT syntax more simple than CREATE DOMAIN. Your proposed patch makes the code simpler, yes, but I think it also makes the error messages worse. I don't think that's an improvement from the user point of view. -- Álvaro Herrera

Re: ecpg command does not warn COPY ... FROM STDIN;

2025-01-14 Thread Fujii Masao
On 2025/01/12 18:27, Ryo Kanbayashi wrote: Thank you for reviewing patch :) The commit log matches with my recognition and has no problem. Pushed. Thanks! check_patches.sh -> utility script for testing above two patches on each target branches Should we add a regression test to ensure ec

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-14 Thread Nathan Bossart
On Mon, Jan 13, 2025 at 05:51:54PM -0500, Tom Lane wrote: > It's fair to worry about this, but I don't think my testing that would > prove a lot. AFAICS, whether somebody runs into trouble would depend > on many factors like their specific build process and what versions of > which packages they h

Re: Support --include-analyze in pg_dump, pg_dumpall, pg_restore

2025-01-14 Thread Nathan Bossart
On Tue, Jan 14, 2025 at 09:32:19AM -0500, Tom Lane wrote: > jian he writes: >> the implemented feature is as the $subject description, making pg_dump >> also includes the ANALYZE command in its output. > > Isn't this pretty much obsoleted by the ongoing work to dump and > restore statistics? Yea

Re: psql: Add leakproof field to \dAo+ meta-command results

2025-01-14 Thread Yugo NAGATA
On Tue, 14 Jan 2025 13:58:18 + Dean Rasheed wrote: > On Tue, 14 Jan 2025 at 08:44, Yugo NAGATA wrote: > > > > I've attached a updated patch v4 that includes fixes on translate_columns[] > > and ranslate_columns_pre_96[]. > > > > This looked good to me, so I've pushed both patches. > > I ch

Re: question about executor hooks

2025-01-14 Thread Sami Imseih
I recommend you to review the documentation in backend/executor/README. It explains in good detail how the executor works. Specifically the section "Query Processing Control Flow" explains what each of the hooks you reference are responsible for. Also, be aware that there are operations, called uti

Re: Use Python "Limited API" in PL/Python

2025-01-14 Thread Jakob Egger
On 07.01.2025, at 08:34, Peter Eisentraut wrote:On 02.12.24 09:51, Peter Eisentraut wrote:This patch changes PL/Python to use the Python "limited API". This API has stronger ABI stability guarantees.[0] This means, you can build PL/ Python against any Python 3.x version and use any other Python 3.

Re: Coccinelle for PostgreSQL development [1/N]: coccicheck.py

2025-01-14 Thread Aleksander Alekseev
Hi, > My apologies, I thought this would make it easier to discuss and review the > code. I will send a single email in the future. > > Should I resend this as a single email with all the patches? IMO the best solution would be re-submitting all the patches to this thread. Also please make sure

Re: Eager aggregation, take 3

2025-01-14 Thread Robert Haas
On Sun, Jan 12, 2025 at 9:04 PM Richard Guo wrote: > Attached is an updated version of this patch that addresses Jian's > review comments, along with some more cosmetic tweaks. I'm going to > be looking at this patch again from the point of view of committing > it, with the plan to commit it late

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-14 Thread Zhou, Zhiguo
Good day, Yura! On 1/10/2025 8:42 PM, Yura Sokolov wrote: If you consider hash-table fillrate, than 256 is quite enough for 128 concurrent inserters. The profile of your patch didn't show significant hotspots in the hash table functions, so I believe the 256 entries should be enough. I wi

Re: question about relation_open

2025-01-14 Thread Tom Lane
Giampaolo Capelli writes: > The function relation_open returns a non NULL pointer in my extension, but > then the last line in the following snippet crashes postgres. > Relation rel; > rel = relation_open(relid, AccessShareLock); > Assert(rel != NULL); > Assert(rel->rd_rel != NULL); > Assert(rel

Re: Support --include-analyze in pg_dump, pg_dumpall, pg_restore

2025-01-14 Thread Tom Lane
jian he writes: > the implemented feature is as the $subject description, making pg_dump > also includes the ANALYZE command in its output. Isn't this pretty much obsoleted by the ongoing work to dump and restore statistics? regards, tom lane

Re: SCRAM pass-through authentication for postgres_fdw

2025-01-14 Thread Matheus Alcantara
Em ter., 14 de jan. de 2025 às 06:21, Peter Eisentraut escreveu: > > On 09.01.25 16:22, Matheus Alcantara wrote: > > Yeah, I also think that makes sense. > > > > I've made all changes on the attached v2. > > (This should probably have been v3, since you had already sent a v2 > earlier.) > Oops, so

Re: psql: Add leakproof field to \dAo+ meta-command results

2025-01-14 Thread Dean Rasheed
On Tue, 14 Jan 2025 at 08:44, Yugo NAGATA wrote: > > I've attached a updated patch v4 that includes fixes on translate_columns[] > and ranslate_columns_pre_96[]. > This looked good to me, so I've pushed both patches. I changed the column name to "Leakproof?" with a question mark, because all oth

Re: Make pg_stat_io view count IOs as bytes instead of blocks

2025-01-14 Thread Nazir Bilal Yavuz
Hi, On Tue, 14 Jan 2025 at 06:18, Michael Paquier wrote: > > On Fri, Jan 10, 2025 at 08:23:46AM +, Bertrand Drouvot wrote: > > On Fri, Jan 10, 2025 at 10:15:52AM +0300, Nazir Bilal Yavuz wrote: > >> But I agree that having a macro has more benefits, > >> also there already is a check for the

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

2025-01-14 Thread Robert Treat
On Tue, Jan 14, 2025 at 1:24 AM Peter Smith wrote: > On Tue, Jan 14, 2025 at 4:46 PM Robert Treat wrote: > > On Mon, Jan 13, 2025 at 8:07 PM Peter Smith wrote: > > > On Tue, Jan 14, 2025 at 8:22 AM Robert Treat wrote: > > > > On Mon, Jan 13, 2025 at 3:55 AM Amit Kapila > > > > wrote: > > > >

Re: AIO v2.2

2025-01-14 Thread Robert Haas
On Mon, Jan 13, 2025 at 4:46 PM Andres Freund wrote: > Once the IO is really done, the handle is immediately recycled (and moved into > IDLE state, ready to be used again). OK, fair enough. -- Robert Haas EDB: http://www.enterprisedb.com

Re: postgres_fdw: Provide better emulation of READ COMMITTED behavior

2025-01-14 Thread Robert Haas
On Sat, Dec 7, 2024 at 4:03 AM Etsuro Fujita wrote: > On Fri, Dec 6, 2024 at 2:37 AM Robert Haas wrote: > > I have a hard time seeing how this would work if cursors are in use on > > the main server. Say I do this: > > > > DECLARE foo CURSOR FOR SELECT * FROM ft1 UNION ALL SELECT * FROM ft2; > >

Re: [PATCH] Add get_bytes() and set_bytes() functions

2025-01-14 Thread Aleksander Alekseev
Hi Dean, > Those are all instances of a value that's outside a specific range > that you might not otherwise know, rather than being out of range of > the type itself. For that, we generally don't say what the range of > the type is. For example, we currently do: > > select repeat('1', 50)::bit(50

Re: question about executor hooks

2025-01-14 Thread jian he
On Tue, Jan 14, 2025 at 5:08 PM Luca Ferrari wrote: > > In the file backend/executor/execMain.c there are the following hook types: > > /* Hooks for plugins to get control in ExecutorStart/Run/Finish/End */ > ExecutorStart_hook_type ExecutorStart_hook = NULL; > ExecutorRun_hook_type ExecutorRun_ho

Re: Recovering from detoast-related catcache invalidations

2025-01-14 Thread Heikki Linnakangas
On 12/01/2025 03:26, Noah Misch wrote: On Thu, Jan 09, 2025 at 11:39:53AM +0200, Heikki Linnakangas wrote: On 07/01/2025 23:56, Noah Misch wrote: @@ -697,9 +725,14 @@ CreateCacheMemoryContext(void) * * This is not very efficient if the target cache is nearly empty. * However, it shouldn

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-14 Thread Bertrand Drouvot
Hi, On Tue, Jan 14, 2025 at 12:58:44AM -0500, Andres Freund wrote: > Hi, > > On 2025-01-13 12:20:39 +, Bertrand Drouvot wrote: > > > We have > > > multiple copies of code to go to FatalError, that doesn't seem great. > > > > + * FIXME: This should probably not have a copy fo the code to > >

Support --include-analyze in pg_dump, pg_dumpall, pg_restore

2025-01-14 Thread jian he
hi. after watching https://www.youtube.com/live/k4A9-WZET_4?si=vz3lTud735s2vcCO then trying to hack it too. the implemented feature is as the $subject description, making pg_dump also includes the ANALYZE command in its output. if option --include-analyze not specified, then pg_dump will not dump

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-01-14 Thread Alvaro Herrera
Hello, I came across this email by chance while looking for something else. On 2024-Oct-07, Aya Iwata (Fujitsu) wrote: > Therefore, we propose Vertical Clustered Indexing (VCI), an in-memory > column store function that holds data in a state suitable for business > analysis and is also expected

RE: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-14 Thread Hayato Kuroda (Fujitsu)
Dear Daniel, Thanks for working on the project. I have few cosmetic comments. ``` + built in legacy crypto functions gen_salt(), ``` According to other lines, `gen_salt()` should be `gen_salt()`. ``` + pg_gen_salt_rounds(), and crypt() ``` Similar with [1], `pg_gen_salt_rounds` is n

Re: Sample rate added to pg_stat_statements

2025-01-14 Thread Ilia Evdokimov
On 09.01.2025 22:13, Alena Rybakina wrote: Hi! Thank you for the work with this subject. I looked at your patch and noticed that this part of the code is repeated several times: if (nesting_level == 0)     {     if (!IsParallelWorker())     current_query_sampled = pg_prng_double

Re: Modern SHA2- based password hashes for pgcrypto

2025-01-14 Thread Bernd Helmle
Am Dienstag, dem 14.01.2025 um 11:47 +0100 schrieb Alvaro Herrera: > > Oh, that's news to me. Is there a plan for it somewhere? I agree > > that > > pgcrypto is widley used and needs a proper replacement when we > > decide > > to deprecate it. > > I don't know about a plan, but > https://www.youtu

Re: question about relation_open

2025-01-14 Thread Rahila Syed
Hi, > > > I want to call the function ReadBufferExtended > to get the raw data of a given buffer > and I've read in some examples that I need to call relation_open > first, in order to get a Relation variable and also lock the relation. > > The function relation_open returns a non NULL pointer in

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2025-01-14 Thread Shlok Kyal
On Fri, 27 Dec 2024 at 12:06, Shubham Khanna wrote: > > On Fri, Dec 27, 2024 at 11:30 AM vignesh C wrote: > > > > > > > > The documentation requires a minor update: instead of specifying > > subscriptions, the user will specify multiple databases, and the > > subscription will be created on the s

Re: Issue with markers in isolation tester? Or not?

2025-01-14 Thread Michail Nikolaev
Hello, Noah! Thanks for your attention! It looks like the simplest solution is just to count the number of not-yet-completed steps and check that value. A patch with such changes (+ the same test + README update + commit message) is attached. Best regards, Mikhail. v2-0001-isolation-tester-Fi

Re: Modern SHA2- based password hashes for pgcrypto

2025-01-14 Thread Alvaro Herrera
Hello Bernd, On 2025-Jan-14, Bernd Helmle wrote: > > It's been said in my presence that pgcrypto is obsolete and > > shouldn't be used anymore.  I'm not sure I believe that, but even if > > that's true, it's clear that there's plenty of people who has an > > interest on it, so I don't see that as

Re: Retiring is_pushed_down

2025-01-14 Thread Richard Guo
On Fri, Sep 27, 2024 at 5:06 AM Tom Lane wrote: > So I'm worried that the point about lateral refs is still a problem > in your version. To be clear, the hazard is that if a WHERE clause > ends up getting placed at an outer join that's higher than any of > the OJs specifically listed in its requi

Re: Virtual generated columns

2025-01-14 Thread Peter Eisentraut
On 09.01.25 13:41, jian he wrote: we can not ALTER COLUMN DROP EXPRESSION for virtual for now. so the following comments in generated_virtual.sql conflict with the output. ``` -- check that dependencies between columns have also been removed ALTER TABLE gtest29 DROP COLUMN a; -- should not drop

Re: Psql meta-command conninfo+

2025-01-14 Thread Dean Rasheed
On Tue, 14 Jan 2025 at 08:51, Alvaro Herrera wrote: > > On 2025-Jan-14, Hunaid Sohail wrote: > > > I've tried the approach and attached the output. Does this look good? > > Hmm, I'm not sure I like the third column particularly; it's noisy to > have on all the time. I'd leave that for \conninfo+

Re: Virtual generated columns

2025-01-14 Thread Peter Eisentraut
On 09.01.25 09:38, jian he wrote: create user foo; create user bar; grant create on schema public to foo; \c - foo create table t1 (id int, ccnum text, ccredacted text generated always as (repeat('*', 12) || substr(ccnum, 13, 4)) virtual); grant select (id, ccredacted) on table t1 to bar; insert

Re: Modern SHA2- based password hashes for pgcrypto

2025-01-14 Thread Bernd Helmle
Hi Alvaro, Am Montag, dem 13.01.2025 um 17:06 +0100 schrieb Alvaro Herrera: > Hello > > I was passing by and I noticed that this needs badly pgindent, and > some > comments are enumerations that would lose formatting once through it. > For example, this would happen which is not good: > >     /*

Re: Sort functions with specialized comparators

2025-01-14 Thread Andrey Borodin
> On 14 Jan 2025, at 13:58, John Naylor wrote: > > That's not as clear-cut as I thought. To avoid regressions, I've gone > back to an earlier idea to pass the direction to the comparator, but > this time keep it simple by using the same comparator for sort and > unique, similar to v9. Looks g

Re: SCRAM pass-through authentication for postgres_fdw

2025-01-14 Thread Peter Eisentraut
On 09.01.25 16:22, Matheus Alcantara wrote: Yeah, I also think that makes sense. I've made all changes on the attached v2. (This should probably have been v3, since you had already sent a v2 earlier.) This all looks good to me. Attached is a fixup patch where I have tried to expand the doc

  1   2   >