Re: Parallel heap vacuum

2025-02-14 Thread Masahiko Sawada
On Thu, Feb 13, 2025 at 8:16 PM John Naylor wrote: > > On Thu, Feb 13, 2025 at 5:37 AM Masahiko Sawada wrote: > > > > During eager vacuum scan, we reset the eager_scan_remaining_fails > > counter when we start to scan the new region. So if we want to make > > parallel heap vacuum behaves exactly

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > Constants passed as parameters to a prepared statement will not be > handled as expected. I did not not test explicit PREPARE/EXECUTE statement, > but I assume it will have the same issue. This is the same question of supporting variou

Re: [RFC] Lock-free XLog Reservation from WAL

2025-02-14 Thread Zhou, Zhiguo
On 2/11/2025 9:25 AM, Japin Li wrote: On Mon, 10 Feb 2025 at 22:12, "Zhou, Zhiguo" wrote: On 2/5/2025 4:32 PM, Japin Li wrote: On Mon, 27 Jan 2025 at 17:30, "Zhou, Zhiguo" wrote: On 1/26/2025 10:59 PM, Yura Sokolov wrote: 24.01.2025 12:07, Japin Li пишет: On Thu, 23 Jan 2025 at 21:44, Ja

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-14 Thread Masahiko Sawada
On Tue, Feb 11, 2025 at 11:44 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Feb 11, 2025 at 02:11:10PM -0800, Masahiko Sawada wrote: > > I've updated the patch that includes comment updates and bug fixes. > > Thanks! > > > The main idea of changing WAL level online is to decouple two aspects: > >

Re: Allow io_combine_limit up to 1MB

2025-02-14 Thread Jakub Wartak
On Wed, Feb 12, 2025 at 1:03 AM Andres Freund wrote: > > Hi, > > On 2025-02-11 13:12:17 +1300, Thomas Munro wrote: > > Tomas queried[1] the limit of 256kB (or really 32 blocks) for > > io_combine_limit. Yeah, I think we should increase it and allow > > experimentation with larger numbers. Note t

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
Hi, On Fri, Feb 14, 2025 at 09:36:08AM +0100, Dmitry Dolgov wrote: > > On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > > This case with an array passed to aa function seems to cause a regression > > in pg_stat_statements query text. As you can see the text is incomplete. > > I've alre

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Thomas Munro
On Sat, Feb 15, 2025 at 12:50 PM Melanie Plageman wrote: > It fixed the issue (after an off-list correction to the patch by Thomas). Thanks! It's green again.

Decision by Monday: PQescapeString() vs. encoding violation

2025-02-14 Thread Noah Misch
The security team has a question below about how best to proceed with a recent behavior change. Commit 5dc1e42b4fa6a4434afa7d7cdcf0291351a7b873 for this week's CVE-2025-1094 changed how PQescapeString()[1] reacts to input that is not valid in the client encoding. Before that commit, the function

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-02-14 Thread Jacob Champion
On Thu, Feb 13, 2025 at 4:03 PM Michael Paquier wrote: > > If a CA is issuing Subject data that is somehow dangerous to the > > operation of the server, I think that's a security problem in and of > > itself: there are clientcert HBA modes that don't validate the > > Subject, but they're still goi

Re: pg17.3 PQescapeIdentifier() ignores len

2025-02-14 Thread Andres Freund
Hi, On 2025-02-13 14:00:09 -0500, Tom Lane wrote: > Justin Pryzby writes: > > The fprintf suggests that since 5dc1e42b4 PQescapeIdentifier ignores its > > len. > > Ugh, yes. Need something like the attached. I just pushed this fix, together with an expansion of test_escape.c. With the expande

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Melanie Plageman
On Fri, Feb 14, 2025 at 6:09 PM Thomas Munro wrote: > > > Agreed that right now is a bad time to push this to v17 --- we need to > > keep the risk factors as low as possible for the re-release. Master > > now and v17 after the re-wrap seems like the right compromise. > > Cool, will push to master

Re: Improve pgindent exclude handling: ignore empty lines

2025-02-14 Thread Andrew Dunstan
On 2025-02-08 Sa 4:39 AM, Zsolt Parragi wrote: Hello, We ran into an issue where pgindent stopped reformatting anything with our custom exclude file, and after some investigation we found the empty line accidentally inserted into the exclude file. Pgindent currently treats empty lines as vali

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Yura Sokolov
14.02.2025 17:09, Yura Sokolov пишет: > 14.02.2025 13:24, Alexander Korotkov пишет: >> On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov >> wrote: >>> On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov >>> wrote: On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov wrote: > On Thu, 13 Feb 2

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 10:39:45PM GMT, Julien Rouhaud wrote: > There seems to be an off-by-1 error in parameter numbering when merging them. There are indeed three constants, but the second is not visible in the query text. Maybe makes sense to adjust the number in this case, let me try. > Not

Re: Thread-safe nl_langinfo() and localeconv()

2025-02-14 Thread Peter Eisentraut
On 09.02.25 15:52, Peter Eisentraut wrote: This patch set is still desirable.  Here is a rebased version of the v5 patches.  I haven't applied any corrections or review comments. Here is the same patch set with some review comments. Patch 0001 looks okay to me. I'm just offering some cosmetic

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:26:19AM GMT, Sami Imseih wrote: > > > On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > > > Constants passed as parameters to a prepared statement will not be > > > handled as expected. I did not not test explicit PREPARE/EXECUTE > > > statement, > > > but I

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 09:06:24AM GMT, Sami Imseih wrote: > I think it will be sad to not include this very common case from > the start, because it is going to be one of the most common > cases. > > Wouldn't doing something like this inside IsMergeableConst > """ > if (!IsA(arg, Const) && !IsA(

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > I perhap meant "missing chunk" instead of "trimming". To me it just > > looked like a trimmed text, which was wrong. Looks like v25 > > deals with that better at least. I am just not sure about all that we are > > doing > > here as I believe it may open up big changes for bugs generating the

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > > I perhap meant "missing chunk" instead of "trimming". To me it just > > > looked like a trimmed text, which was wrong. Looks like v25 > > > deals with that better at least. I am just not sure about all that we are > > > doing > > > here as I believe it may open up big changes for bugs genera

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Yura Sokolov
14.02.2025 13:24, Alexander Korotkov пишет: > On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov wrote: >> On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov >> wrote: >>> On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov >>> wrote: On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov wrote: >>>

Parameter binding for COPY TO queries

2025-02-14 Thread Jens-Wolfhard Schicke-Uffmann
Hi, I'd like some input on the idea of adding parameter binding support to queries executed as part of a COPY TO command. Is there a technical or philosophical reason why these queries should not contain bindable parameters? As far as I could ascertain, the feature has been desired for a long tim

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
On Fri, Feb 14, 2025 at 03:20:24PM +0100, Dmitry Dolgov wrote: > > Btw, there was another mistake in the last version introducing > "$1 /*, ... */" format, the constant position has to be of course > calculated as usual. I'm not sure what you mean here, but just in case: > +SELECT * FROM test_me

Re: [Feature Request] INSERT FROZEN to Optimize Large Cold Data Imports and Migrations

2025-02-14 Thread Greg Sabino Mullane
On Fri, Feb 14, 2025 at 1:41 AM Sébastien wrote: > I had huge problems on server 3 weeks after a 6 TB migration from other > DB. I think it's sad to rewrite all data twice. > You could always COPY FREEZE into a new table, then move any remaining rows, and finally rename the tables. Should be a w

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-14 Thread Melanie Plageman
io_combine_limit 1, effective_io_concurrency 16, read ahead kb 16 On Fri, Feb 14, 2025 at 12:18 PM Tomas Vondra wrote: > > Based on off-list discussion with Melanie, I ran a modified version of > the benchmark, with these changes: Thanks! It looks like the worst offender is io_combine_limit 1 (1

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > Wouldn't doing something like this inside IsMergeableConst > > """ > > if (!IsA(arg, Const) && !IsA(arg, Param)) > > """ > > > > instead of > > """ > > if (!IsA(arg, Const)) > > """ > > > > be sufficient? > > That's exactly what the original rejected implementation was doing. I > guess to answe

Re: New buildfarm animals with FIPS mode enabled

2025-02-14 Thread Daniel Gustafsson
> On 14 Feb 2025, at 19:01, Tom Lane wrote: > I'm kind of disinclined to do all the work that'd be needed to turn > these animals completely green, especially when the reason to do it > seems to be that someone decided we should without any community > consultation. Perhaps others have different

Re: Parameter binding for COPY TO queries

2025-02-14 Thread Tom Lane
Andres Freund writes: > On 2025-02-14 10:06:13 -0500, Tom Lane wrote: >> None of those arguments apply to the sub-SELECT of a "COPY (query) TO" >> command, but there's a practical matter of how to get the parameters >> passed through the COPY to the sub-SELECT without opening Pandora's >> box for

Re: Feature Request: Add AES-128-CFB Mode Support to pgcrypto

2025-02-14 Thread Daniel Gustafsson
> On 5 Feb 2025, at 18:28, Daniel Gustafsson wrote: > >> On 5 Feb 2025, at 18:24, Álvaro Herrera wrote: > >> Anyway, at least the bytes appear to be interpreted the same by both >> openssl and this new function, so that's good news. > > Thanks for confirming. Short of a very small doc change

Re: Track the amount of time waiting due to cost_delay

2025-02-14 Thread Nathan Bossart
Committed. I noticed two things that I felt I should note here: * For the vacuum path, we call pgstat_progress_end_command() prior to accessing the value. This works because pgstat_progress_end_command() doesn't clear the st_progress_param array (that is done in pgstat_progress_start_comma

Re: generic plans and "initial" pruning

2025-02-14 Thread Alexander Lakhin
Hello Amit, 06.02.2025 04:35, Amit Langote wrote: I plan to push 0001 tomorrow, barring any objections. Please try the following script: CREATE TABLE pt (a int, b int) PARTITION BY range (a); CREATE TABLE tp1 PARTITION OF pt FOR VALUES FROM (1) TO (2); CREATE TABLE tp2 PARTITION OF pt FOR VAL

Re: Document How Commit Handles Aborted Transactions

2025-02-14 Thread Ahmed Ashour
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed Summary: - The patch adds documentation to clarify ho

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Thomas Munro
On Sat, Feb 15, 2025 at 10:50 AM Thomas Munro wrote: > On Sat, Feb 15, 2025 at 7:30 AM Melanie Plageman > wrote: > > Seems valgrind doesn't like this [1]. I'm looking into it. > > Melanie was able to reproduce this on her local valgrind and > eventually we figured out that it's my fault. I put c

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Thomas Munro
On Sat, Feb 15, 2025 at 12:03 PM Tom Lane wrote: > Thomas Munro writes: > > Here's a patch. Is there a tidier way to write this? > > Hmm, I think not with the current set of primitives. We could think > about refactoring them, but that's not a job for a band-aid patch. Thanks for looking. > >

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Tom Lane
Thomas Munro writes: > Here's a patch. Is there a tidier way to write this? Hmm, I think not with the current set of primitives. We could think about refactoring them, but that's not a job for a band-aid patch. > It should probably be back-patched to 17, because external code might > use per-b

Re: [PATCH] SVE popcount support

2025-02-14 Thread Nathan Bossart
On Thu, Feb 06, 2025 at 10:33:35AM -0600, Nathan Bossart wrote: > On Thu, Feb 06, 2025 at 08:44:35AM +, chiranmoy.bhattacha...@fujitsu.com > wrote: >>> Does this hand-rolled loop unrolling offer any particular advantage? What >>> do the numbers look like if we don't do this or if we process,

Re: Parallel heap vacuum

2025-02-14 Thread Melanie Plageman
On Wed, Feb 12, 2025 at 5:37 PM Masahiko Sawada wrote: > > Since we introduced the eagar vacuum scan (052026c9b9), I need to > update the parallel heap vacuum patch. After thinking of how to > integrate these two features, I find some complexities. The region > size used by eager vacuum scan and t

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Thomas Munro
On Sat, Feb 15, 2025 at 7:30 AM Melanie Plageman wrote: > Seems valgrind doesn't like this [1]. I'm looking into it. Melanie was able to reproduce this on her local valgrind and eventually we figured out that it's my fault. I put code into read_stream.c that calls wipe_mem(), thinking that that

Re: New buildfarm animals with FIPS mode enabled

2025-02-14 Thread Jacob Champion
On Fri, Feb 14, 2025 at 12:51 PM Daniel Gustafsson wrote: > > > On 14 Feb 2025, at 19:01, Tom Lane wrote: > > > I'm kind of disinclined to do all the work that'd be needed to turn > > these animals completely green, especially when the reason to do it > > seems to be that someone decided we shoul

Re: New buildfarm animals with FIPS mode enabled

2025-02-14 Thread Tom Lane
Jacob Champion writes: > I'm not buildfarm@, but these animals have now been stopped until we > get them figured out. Sorry -- and thanks for the ping Tom! Thanks for that. Just to be clear, I think it'd be great to run those RHEL9 animals on v17 and later. I'm only questioning whether it's wor

Re: Change GUC hashtable to use simplehash?

2025-02-14 Thread John Naylor
On Fri, Feb 14, 2025 at 6:40 PM Anton A. Melnikov wrote: > Yes, of course. I tested this patch on the current master at 9e17ac997 > in the same way and found no valgrind errors. Thanks, I'll push next week after the next minor release. PS: I now realize the source of the confusion: In the time a

Re: Move wal_buffers_full to WalUsage (and report it in pgss/explain)

2025-02-14 Thread Bertrand Drouvot
Hi, On Fri, Feb 14, 2025 at 03:24:22PM +0900, Michael Paquier wrote: > On Tue, Feb 11, 2025 at 09:37:37AM +, Bertrand Drouvot wrote: > > While at it, adding 0004 to report wal_buffers_full in VACUUM/ANALYZE > > (VERBOSE). > > Thanks for summarizing the history behind WalUsage and > wal_buffe

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-14 Thread Jakub Wartak
On Wed, Feb 12, 2025 at 9:57 PM Robert Haas wrote: > > On Wed, Feb 12, 2025 at 3:07 PM Tomas Vondra wrote: > > AFAICS the "1" value is simply one of the many "defensive" defaults in > > our sample config. It's much more likely to help than cause harm, even > > on smaller/older systems, but for ma

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Álvaro Herrera
On 2025-Feb-14, Julien Rouhaud wrote: > Since the merging is a yes/no option (I think there used to be some > discussions > about having a threshold or some other fancy modes), maybe you could instead > differentiate the merged version by have 2 constants rather than this "..." or > something lik

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > > Since the merging is a yes/no option (I think there used to be some > > > discussions > > > about having a threshold or some other fancy modes), maybe you could > > > instead > > > differentiate the merged version by have 2 constants rather than this > > > "..." or > > > something like tha

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-02-14 Thread Shlok Kyal
On Fri, 14 Feb 2025 at 10:50, Shubham Khanna wrote: > > On Thu, Feb 13, 2025 at 5:36 PM Shlok Kyal wrote: > > > > On Thu, 13 Feb 2025 at 15:20, Shubham Khanna > > wrote: > > > > > > On Tue, Feb 11, 2025 at 9:56 PM Shlok Kyal > > > wrote: > > > > > > > > On Tue, 11 Feb 2025 at 09:51, Shubham Kh

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Pavel Borisov
Hi, Alexander! On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov wrote: > > Hi, Pavel! > > On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov wrote: > > On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov > > wrote: > > > > > > On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov > > > wrote: > > > > 13.02.

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
On Fri, Feb 14, 2025 at 10:36:48AM +0100, Álvaro Herrera wrote: > On 2025-Feb-14, Julien Rouhaud wrote: > > > Since the merging is a yes/no option (I think there used to be some > > discussions > > about having a threshold or some other fancy modes), maybe you could instead > > differentiate the

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:57:01PM GMT, Julien Rouhaud wrote: > On Fri, Feb 14, 2025 at 10:36:48AM +0100, Álvaro Herrera wrote: > > On 2025-Feb-14, Julien Rouhaud wrote: > > > > > Since the merging is a yes/no option (I think there used to be some > > > discussions > > > about having a threshold

Re: Change GUC hashtable to use simplehash?

2025-02-14 Thread Anton A. Melnikov
Hi, Jhon! On 14.02.2025 08:17, John Naylor wrote: Hi Anton, could you please test if the attached passes for you? This seems the simplest way. Yes, of course. I tested this patch on the current master at 9e17ac997 in the same way and found no valgrind errors. Moreover, with -O2 build on my PC

Re: PATCH: Disallow a netmask of zero unless the IP is also all zeroes

2025-02-14 Thread Andreas Karlsson
On 2/11/25 9:25 PM, Tom Lane wrote: Greg Sabino Mullane writes: I say "of course" but few people (even tech ones) know the distinction. (Nor should they have to! But that's for a nearby thread). This patch aims to prevent this very bad footgun by only allowing a /0 if the IP consists of only ze

Re: Assignment before assert

2025-02-14 Thread Daniel Gustafsson
> On 14 Feb 2025, at 06:44, Michael Paquier wrote: > the proposed patch makes sense to me. Committed, with a backpatch down to 13. -- Daniel Gustafsson

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Daniel Gustafsson
> On 13 Feb 2025, at 18:39, Andres Freund wrote: > The banner being the same between queries made it hard to understand if a > banner that appeared in the output was from the current query or a past > query. Therefore I added a counter to it. + my $banner = "background_psql: QUERY_SEPARATO

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-02-14 Thread Amit Kapila
On Mon, Feb 19, 2024 at 12:49 PM vignesh C wrote: > > Currently ALTER SUBSCRIPTION ... SET PUBLICATION will break the > logical replication in certain cases. This can happen as the apply > worker will get restarted after SET PUBLICATION, the apply worker will > use the existing slot and replicatio

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-14 Thread Bertrand Drouvot
Hi, On Fri, Feb 14, 2025 at 12:17:48AM -0800, Masahiko Sawada wrote: > On Tue, Feb 11, 2025 at 11:44 PM Bertrand Drouvot > wrote: > Looking at the latest custodian worker patch, the basic architecture > is to have a single custodian worker and processes can ask it for some > work such as removin

Re: Virtual generated columns

2025-02-14 Thread Peter Eisentraut
On 13.02.25 14:06, jian he wrote: I didn't solve the out join semantic issue. i am wondering, can we do the virtual generated column expansion in the rewrite stage as is, and wrap the expressions in PHVs if the virtual generated columns come from the nullable side of an outer join. PlaceHolderV

Re: Simplify the logic a bit (src/bin/scripts/reindexdb.c)

2025-02-14 Thread Ranier Vilela
Hi Álvaro. Em qui., 13 de fev. de 2025 às 18:38, Álvaro Herrera < alvhe...@alvh.no-ip.org> escreveu: > On 2025-Feb-13, Ranier Vilela wrote: > > > Hi. > > > > Coverity complained about possible dereference null pointer > > in *reindex_one_database* function. > > That's not really true. > > But the

Re: pg17.3 PQescapeIdentifier() ignores len

2025-02-14 Thread Ranier Vilela
Em qui., 13 de fev. de 2025 às 16:00, Tom Lane escreveu: > Justin Pryzby writes: > > The fprintf suggests that since 5dc1e42b4 PQescapeIdentifier ignores its > len. > > Ugh, yes. Need something like the attached. > > FTR, 5dc1e42b4 et al were quite subtle patches done under extreme time > press

Re: Update Unicode data to Unicode 16.0.0

2025-02-14 Thread Peter Eisentraut
On 05.02.25 22:47, Jeff Davis wrote: (b) we should make reasonable attempts to mitigate potential problems. One idea for (b) resurfaced, which was to make a best-effort check at pg_upgrade time for affected indexes. The check would not be bulletproof, because we can't catch dependencies that

Re: pgbench without dbname worked differently with psql and pg_dump

2025-02-14 Thread Ashutosh Bapat
On Wed, Jan 22, 2025 at 7:24 AM Hayato Kuroda (Fujitsu) wrote: > > ``` > > Thanks for the confirmation. I found below executables seem to have the same > logic to decide the dbname: > > - pg_amcheck The documentation just says "If no such options are present, the default database will be checked.

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Alexander Korotkov
On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov wrote: > On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov wrote: > > On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov > > wrote: > > > On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov > > > wrote: > > > > > > > > On Thu, Feb 13, 2025 at 11:45 AM Yur

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-02-14 Thread Shubham Khanna
On Fri, Feb 14, 2025 at 4:57 PM Shlok Kyal wrote: > > On Fri, 14 Feb 2025 at 10:50, Shubham Khanna > wrote: > > > > On Thu, Feb 13, 2025 at 5:36 PM Shlok Kyal wrote: > > > > > > On Thu, 13 Feb 2025 at 15:20, Shubham Khanna > > > wrote: > > > > > > > > On Tue, Feb 11, 2025 at 9:56 PM Shlok Kyal

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-14 Thread Ashutosh Bapat
Hi Shubham, Here are some review comments from my side On Thu, Feb 13, 2025 at 8:58 AM Shubham Khanna wrote: > The attached patch contains the required changes. > clusterdb, vacuumdb use -a and -all for all databases. Do we want to use the same long option name here? Probably they don't have -d

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-14 Thread Nisha Moond
Please find the updated v78 patches after a few off-list review rounds. Here is a summary of changes in v78: patch-001: - Fixed bugs reported by Hou-san and Peter in [1] and [2]. - Fixed a race condition reported by Hou-san off-list, which could lead to an assert failure. This failure happens when

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andrew Dunstan
On 2025-02-13 Th 12:39 PM, Andres Freund wrote: Hi, One of the remaining tasks for AIO was to convert the tests to be proper tap tests. I did that and was thanked by the tests fairly randomly failing on windows. Which test fails changes from run to run. The symptom is that BackgroundPsql->qu

Re: Sample rate added to pg_stat_statements

2025-02-14 Thread Ilia Evdokimov
Hi hackers, I've decided to explore a slightly different approach to reducing spinlock contention—by introducing a simple execution time threshold. If a query’s execution time exceeds this threshold, it is recorded in pg_stat_statements; otherwise, it is ignored. As Alexander [0] pointed out,

Re: Simplify the logic a bit (src/bin/scripts/reindexdb.c)

2025-02-14 Thread Ranier Vilela
Em sex., 14 de fev. de 2025 às 09:13, Ranier Vilela escreveu: > Hi Álvaro. > > Em qui., 13 de fev. de 2025 às 18:38, Álvaro Herrera < > alvhe...@alvh.no-ip.org> escreveu: > >> On 2025-Feb-13, Ranier Vilela wrote: >> >> > Hi. >> > >> > Coverity complained about possible dereference null pointer >>

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-14 Thread Tomas Vondra
On 2/14/25 18:31, Melanie Plageman wrote: > io_combine_limit 1, effective_io_concurrency 16, read ahead kb 16 > > On Fri, Feb 14, 2025 at 12:18 PM Tomas Vondra wrote: >> >> Based on off-list discussion with Melanie, I ran a modified version of >> the benchmark, with these changes: > > Thanks! It

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Jacob Champion
On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: > commit 70291a3c66e > Author: Michael Paquier > Date: 2024-11-07 12:11:27 +0900 > > Improve handling of empty query results in BackgroundPsql::query() > > commit ba08edb0654 > Author: Michael Paquier > Date: 2024-11-06 15:31:14 +0900

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Melanie Plageman
On Thu, Feb 13, 2025 at 9:06 PM Melanie Plageman wrote: > > On Thu, Feb 13, 2025 at 8:30 PM Masahiko Sawada wrote: > > The rest looks good to me. > > Cool! I'll plan to push this tomorrow barring any objections. I've committed this and marked it as such in the CF app. - Melanie

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-14 Thread Tomas Vondra
On 2/14/25 18:14, Andres Freund wrote: > Hi, > > On 2025-02-14 10:04:41 +0100, Jakub Wartak wrote: >> Is there any reason we couldn't have new pg_test_iorates (similiar to >> other pg_test_* proggies), that would literally do this and calibrate >> best e_io_c during initdb and put the result into

Re: [Feature Request] INSERT FROZEN to Optimize Large Cold Data Imports and Migrations

2025-02-14 Thread Andres Freund
Hi, On 2025-02-13 10:52:31 +0100, Sébastien wrote: > Introduce an INSERT FROZEN feature to bypass vacuum processing for > large-scale cold data imports, reducing the impact on system performance > post-import. For large imports, migrations and major version upgrades. > Business Use-case: > > When

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > I do see the discussion here [1], sorry for not noticing it. > > > > I am not sure about this though. At minimum this needs to be documented, > > However, I think the prepared statement case is too common of a case to > > skip for the first release of tis feature, and users that will likely >

Re: Parameter binding for COPY TO queries

2025-02-14 Thread Tom Lane
Jens-Wolfhard Schicke-Uffmann writes: > I'd like some input on the idea of adding parameter binding support to > queries executed as part of a COPY TO command. Is there a technical > or philosophical reason why these queries should not contain bindable > parameters? It would require some rethinki

Re: describe special values in GUC descriptions more consistently

2025-02-14 Thread Nathan Bossart
On Thu, Feb 13, 2025 at 05:01:59PM -0600, Nathan Bossart wrote: > Okay, I took your suggestions in v7. Committed. Thanks, David, Peter, and Daniel! -- nathan

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 08:14:45 -0500, Andrew Dunstan wrote: > It's been discussed before, but I'd really really like to get rid of > BackgroundPsql. It's ugly, non-intuitive and fragile. I agree, unfortunately we're stuck with this until we have a better alternative in tree :( > Last year I did som

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 13:35:40 +0100, Daniel Gustafsson wrote: > > On 13 Feb 2025, at 18:39, Andres Freund wrote: > > > The banner being the same between queries made it hard to understand if a > > banner that appeared in the output was from the current query or a past > > query. Therefore I added a

Re: NOT ENFORCED constraint feature

2025-02-14 Thread Ashutosh Bapat
On Thu, Feb 13, 2025 at 5:27 PM Álvaro Herrera wrote: > > On 2025-Feb-13, Ashutosh Bapat wrote: > > > > So considering that, I think a three-state system makes more sense. > > > Something like: > > > > > > 1) NOT ENFORCED -- no data is checked > > > 2) NOT VALID -- existing data is unchecked, new

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
On Fri, Feb 14, 2025 at 03:56:32PM +0100, Dmitry Dolgov wrote: > > On Fri, Feb 14, 2025 at 10:39:45PM GMT, Julien Rouhaud wrote: > > There seems to be an off-by-1 error in parameter numbering when merging > > them. > > There are indeed three constants, but the second is not visible in the > query

Re: SQLFunctionCache and generic plans

2025-02-14 Thread Alexander Pyhalov
Hi, folks. I've looked through performance and found that most performance issues was caused by CachedPlanSource machinery itself. At least, a lot of it. And so decided to go along and implement plan cache for sql functions. I'm not sure that it's clean enough, but at least it seems to be wor

Re: NOT ENFORCED constraint feature

2025-02-14 Thread Isaac Morland
On Fri, 14 Feb 2025 at 10:11, Ashutosh Bapat wrote: > > > I think, what you intend to say is clearer with 4 state system {NE, E} > > > * {NV, V} = {(NE, NV), (NE, V), (E, NV), (E, V)} where (NE, V) is > > > unreachable. Let's name them S1, S2, S3, S4 respectively. > > [...] > > > Notice that the

Re: Allow default \watch interval in psql to be configured

2025-02-14 Thread Daniel Gustafsson
> On 2 Feb 2025, at 19:25, Daniel Gustafsson wrote: Attached is rebase with some small levels of polish, unless objected to I would like to go ahead with this version. -- Daniel Gustafsson v5-0001-psql-Make-default-watch-interval-configurable.patch Description: Binary data

Re: Allow io_combine_limit up to 1MB

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 09:32:32 +0100, Jakub Wartak wrote: > On Wed, Feb 12, 2025 at 1:03 AM Andres Freund wrote: > > FWIW, I see substantial performance *regressions* with *big* IO sizes using > > fio. Just looking at cached buffered IO. > > > > for s in 4 8 16 32 64 128 256 512 1024 2048 4096 8192;d

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 10:04:41 +0100, Jakub Wartak wrote: > Is there any reason we couldn't have new pg_test_iorates (similiar to > other pg_test_* proggies), that would literally do this and calibrate > best e_io_c during initdb and put the result into postgresql.auto.conf > (pg_test_iorates --adjust

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Tom Lane
Andres Freund writes: > On 2025-02-14 08:14:45 -0500, Andrew Dunstan wrote: >> ... If there is interest I will >> bring the work up to date, and maybe we can introduce this early in the v19 >> cycle. It would significantly reduce our dependency on IPC::Run, especially >> the pump() stuff. > I def

New buildfarm animals with FIPS mode enabled

2025-02-14 Thread Tom Lane
I see that somebody decided to crank up some animals running RHEL8 and RHEL9 with FIPS mode turned on. The RHEL9 animals pass on v17 and master, but not older branches; the RHEL8 animals pass nowhere. This is unsurprising given that the v17-era commits that allowed our regression tests to pass un

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-14 Thread Melanie Plageman
On Fri, Feb 14, 2025 at 1:15 PM Melanie Plageman wrote: > > On Thu, Feb 13, 2025 at 9:06 PM Melanie Plageman > wrote: > > > > On Thu, Feb 13, 2025 at 8:30 PM Masahiko Sawada > > wrote: > > > The rest looks good to me. > > > > Cool! I'll plan to push this tomorrow barring any objections. > > I'v

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 18:18:47 +0100, Tomas Vondra wrote: > FWIW this does not change anything in the detection of sequential access > patterns, discussed nearby, because the benchmarks started before Andres > looked into that. If needed, I can easily rerun these tests, I just need > a patch to apply.

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 12:14:55 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2025-02-14 08:14:45 -0500, Andrew Dunstan wrote: > >> ... If there is interest I will > >> bring the work up to date, and maybe we can introduce this early in the v19 > >> cycle. It would significantly reduce our dep

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 18:36:37 +0100, Tomas Vondra wrote: > All of this is true, ofc, but maybe it's better to have a tool providing > at least some advice I agree, a tool like that would be useful! One difficulty is that the relevant parameter space is really large, making it hard to keep the runti

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 11:12:25PM GMT, Julien Rouhaud wrote: > > > There seems to be an off-by-1 error in parameter numbering when merging > > > them. > > > > There are indeed three constants, but the second is not visible in the > > query text. Maybe makes sense to adjust the number in this ca

Re: Parameter binding for COPY TO queries

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 10:06:13 -0500, Tom Lane wrote: > None of those arguments apply to the sub-SELECT of a "COPY (query) TO" > command, but there's a practical matter of how to get the parameters > passed through the COPY to the sub-SELECT without opening Pandora's > box for every other kind of util

Re: generic plans and "initial" pruning

2025-02-14 Thread Amit Langote
Hi Alexander, On Sat, Feb 15, 2025 at 6:00 AM Alexander Lakhin wrote: > > Hello Amit, > > 06.02.2025 04:35, Amit Langote wrote: > > I plan to push 0001 tomorrow, barring any objections. > > > Please try the following script: > CREATE TABLE pt (a int, b int) PARTITION BY range (a); > CREATE TABLE