Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-02-20 Thread Peter Smith
Hi Ajin, Some review comments for patch v14-0001. == src/backend/replication/logical/reorderbuffer.c 1. + * We also try and filter changes that are not relevant for logical decoding + * as well as give the option for plugins to filter changes in advance. + * Determining whether to filt

RE: Statistics Import and Export

2025-02-20 Thread Hayato Kuroda (Fujitsu)
Dear members, I hope I'm in the correct thread. I found the commit 1fd1bd8 - it is so cool. I have a question for it. ISTM commit message said that no need to do ANALYZE again. ``` Add support to pg_dump for dumping stats, and use that during pg_upgrade so that statistics are transferred

Re: create subscription with (origin = none, copy_data = on)

2025-02-20 Thread Shlok Kyal
On Thu, 20 Feb 2025 at 16:32, Zhijie Hou (Fujitsu) wrote: > > On Wednesday, January 29, 2025 8:19 PM Shlok Kyal > wrote: > > > > I have addressed the comments. Here is an updated patch. > > Thanks for updating the patch. The patches look mostly OK to me, I only have > one minor comments in 0002.

Re: generic plans and "initial" pruning

2025-02-20 Thread Tom Lane
Amit Langote writes: > I pushed the final piece yesterday. trilobite reports that this fails under -DCLOBBER_CACHE_ALWAYS: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=trilobite&dt=2025-02-20%2019%3A37%3A12 regards, tom lane

Re: generic plans and "initial" pruning

2025-02-20 Thread Amit Langote
On Fri, Feb 21, 2025 at 3:04 PM Tom Lane wrote: > > Amit Langote writes: > > I pushed the final piece yesterday. > > trilobite reports that this fails under -DCLOBBER_CACHE_ALWAYS: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=trilobite&dt=2025-02-20%2019%3A37%3A12 Looking, thanks

Re: Virtual generated columns

2025-02-20 Thread Richard Guo
On Thu, Feb 20, 2025 at 12:25 AM Dean Rasheed wrote: > On Wed, 19 Feb 2025 at 01:42, Dean Rasheed wrote: > > One thing I don't like about this is that it's introducing more code > > duplication between pullup_replace_vars() and > > ReplaceVarsFromTargetList(). Those already had a lot of code in c

Re: Missing [NO] INDENT flag in XMLSerialize backward parsing

2025-02-20 Thread Michael Paquier
On Thu, Feb 20, 2025 at 02:27:42PM +0100, Jim Jones wrote: > This patch adds the missing [NO] INDENT flag to XMLSerialize backward > parsing. if (xexpr->op == IS_XMLSERIALIZE) + { appendStringInfo(buf, " AS %s", format_type_with_typemod(xexpr->typ

Re: Reset the output buffer after sending from WalSndWriteData

2025-02-20 Thread Masahiko Sawada
On Thu, Feb 20, 2025 at 12:50 PM Markus Wanner wrote: > > Hi, > > I recently stumbled over an issue with an unintentional re-transmission. > While this clearly was our fault in the output plugin code, I think the > walsender's exposed API could easily be hardened to prevent the bad > consequence f

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-02-20 Thread Alvaro Herrera
Hello, Thanks! I noticed a typo 'constrint' in several places; fixed in the attached. I discovered that this sequence, taken from added regression tests, CREATE TABLE notnull_tbl1 (a int); ALTER TABLE notnull_tbl1 ADD CONSTRAINT nn_parent not null a not valid; CREATE TABLE notnull_chld (a int);

Re: pg_recvlogical requires -d but not described on the documentation

2025-02-20 Thread Ashutosh Bapat
On Fri, Feb 21, 2025 at 9:56 AM Hayato Kuroda (Fujitsu) wrote: > > Dear hackers, > > Hi, I found the issue $SUBJECT. According to the doc [1]: > > ``` > -d dbname > --dbname=dbname > The database to connect to. See the description of the actions for what this > means in detail. > The dbname can b

Re: BgBufferSync(): clarification about reusable_buffers variable

2025-02-20 Thread Ashutosh Bapat
Hi Marcel, On Sat, Feb 8, 2025 at 6:24 AM M vd H wrote: > > I have been reading the source code of the BgWriter, and there is some > code in BgBufferSync() that I don't fully understand. > > In BgBufferSync(), we have the following code: > > while (num_to_scan > 0 && reusable_buffers < upcom

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Thomas Munro
On Fri, Feb 21, 2025 at 6:20 PM Thomas Munro wrote: > #define PROCNUMBER_BITS 18 > #define MAX_BACKENDS ((1 << PROCNUMBER_BITS) - 1) > #define PROCNUMBER_CHARS DECIMAL_DIGITS_FOR_BITS(PROCNUMBER_BITS) > > ... with a little helper ported to preprocessor hell from Hacker's > Delight magic[1] for tha

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Tom Lane
Thomas Munro writes: > On Fri, Feb 21, 2025 at 9:28 AM Andres Freund wrote: >> The patch curently uses a hardcoded 6 for the length of MAX_BACKENDS. Does >> anybody have a good idea for how to either >> >> a) derive 6 from MAX_BACKENDS in a way that can be used in a C array size This all seems

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Thomas Munro
On Fri, Feb 21, 2025 at 9:28 AM Andres Freund wrote: > The patch curently uses a hardcoded 6 for the length of MAX_BACKENDS. Does > anybody have a good idea for how to either > > a) derive 6 from MAX_BACKENDS in a way that can be used in a C array size Do we even check the binary digits? BTW I s

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-20 Thread Ashutosh Bapat
On Wed, Feb 19, 2025 at 4:23 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for updating the patch! > > > I have added a test case for non-dry-run mode to ensure that > > replication slots, subscriptions, and publications work as expected > > when '--all' is specified. Additionally

Re: Virtual generated columns

2025-02-20 Thread jian he
On Wed, Feb 19, 2025 at 11:25 PM Dean Rasheed wrote: > > One of the new regression tests fails, which actually appears to be a > pre-existing grouping sets bug, due to the fact that only non-Vars are > wrapped in PHVs. This bug can be triggered without virtual generated > columns: > > CREATE TABLE

pg_recvlogical requires -d but not described on the documentation

2025-02-20 Thread Hayato Kuroda (Fujitsu)
Dear hackers, Hi, I found the issue $SUBJECT. According to the doc [1]: ``` -d dbname --dbname=dbname The database to connect to. See the description of the actions for what this means in detail. The dbname can be a connection string. If so, connection string parameters will override any confli

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-20 Thread Ashutosh Bapat
On Fri, Feb 21, 2025 at 5:18 AM Peter Smith wrote: > > On Thu, Feb 20, 2025 at 10:26 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Shubham, > > > > Thanks for updating the patch quickly! > > > > > > 04. > > > > ``` > > > > +# Verify that only user databases got subscriptions (not template > >

Re: generic plans and "initial" pruning

2025-02-20 Thread Amit Langote
On Wed, Feb 12, 2025 at 8:53 PM Amit Langote wrote: > On Thu, Feb 6, 2025 at 11:35 AM Amit Langote wrote: > > Per cfbot-ci, the new test case output in 0002 needed to be updated. > > > > I plan to push 0001 tomorrow, barring any objections. > > I pushed that last Friday. With bb3ec16e, d47cbf47,

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-02-20 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 20 Feb 2025 15:28:26 -0800, Masahiko Sawada wrote: > Looking at the 0001 patch again, I have a question: we have > CopyToTextLikeOneRow() for both CSV and text format: > > +/* Implementation of the pe

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-02-20 Thread Amit Kapila
On Fri, Feb 21, 2025 at 7:57 AM Ajin Cherian wrote: > > On Fri, Feb 21, 2025 at 12:57 PM Ajin Cherian wrote: > > > > Conclusion: > > The patched code with 100 transaction throttling significantly > > improves performance, reducing execution time by ~69% when no > > published transactions are invo

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-02-20 Thread Peter Smith
On Thu, Feb 20, 2025 at 5:44 PM Shubham Khanna wrote: > > On Mon, Feb 17, 2025 at 9:49 AM Peter Smith wrote: > > ... > > == > > 1 GENERAL. Option Name? > > > > Wondering why the patch is introducing more terminology like > > "cleanup"; if we are dropping publications then why not say "drop"?

Re: Extend postgres_fdw_get_connections to return remote backend pid

2025-02-20 Thread Fujii Masao
On 2025/02/21 0:54, Sagar Shedge wrote: Hi Fujii, Naming is always tricky, but remote_backend_pid feels a bit too long. Would remote_pid be sufficient? Point looks valid. I had another perspective is to align the naming convention to pg_backend_pid(). remote_pid is not helping to identify w

Re: Add Pipelining support in psql

2025-02-20 Thread Michael Paquier
On Thu, Feb 20, 2025 at 10:29:33AM +0100, Anthonin Bonnefoy wrote: > Ha yeah, I forgot about the inactive branches. I've added the new > commands and fixed the behaviour. And I did not notice that it was as simple as forcing the status in the routines for the new meta-commands, as we do for the ex

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-02-20 Thread Ajin Cherian
On Fri, Feb 21, 2025 at 12:57 PM Ajin Cherian wrote: > > On Thu, Feb 20, 2025 at 3:08 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Ajin, > > > > > I compared the patch 1 which does not employ a hash cache and has the > > > overhead of starting a transaction every time the filter is checked. >

Re: Add “FOR UPDATE NOWAIT” lock details to the log.

2025-02-20 Thread Fujii Masao
On 2025/02/20 15:27, Yuki Seino wrote: On 2025/02/19 1:08, Fujii Masao wrote: Just an idea, but how about updating ConditionalXactLockTableWait() to do the followings? - Use LockAcquireExtended() instead of LockAcquire() to retrieve the LOCALLOCK value. - Generate a log message about the l

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-02-20 Thread Ajin Cherian
On Thu, Feb 20, 2025 at 3:08 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Ajin, > > > I compared the patch 1 which does not employ a hash cache and has the > > overhead of starting a transaction every time the filter is checked. > > > > I created a test setup of 10 million inserts in 3 different sce

Re: Improve CRC32C performance on SSE4.2

2025-02-20 Thread John Naylor
On Fri, Feb 21, 2025 at 1:24 AM Devulapalli, Raghuveer wrote: > > > > Now, there is no equivalent on MSVC and workarounds are fragile [1]. > > Maybe we could only assert initialization happened for the backend and for > > frontend either > > - add a couple manual initializations to to the frontend

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-20 Thread Melanie Plageman
On Wed, Feb 19, 2025 at 4:59 PM Nathan Bossart wrote: > > On Wed, Feb 19, 2025 at 04:36:05PM -0500, Melanie Plageman wrote: > > This makes me think I should also not cap relallfrozen when using it > > in relation_needs_vacanalyze(). There I cap it to relallvisible and > > relallvisible is capped t

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-20 Thread Thomas Munro
On Fri, Feb 21, 2025 at 11:17 AM Melanie Plageman wrote: > [v31-0001-Delay-extraction-of-TIDBitmap-per-page-offsets.patch] Nice patch, seems like a straightforward win with the benefits you explained: less work done under a lock, less work done in the second iterator if the rest of this stuff doe

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-20 Thread Peter Smith
On Thu, Feb 20, 2025 at 10:26 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for updating the patch quickly! > > > > 04. > > > ``` > > > +# Verify that only user databases got subscriptions (not template > > > databases) > > > +my @user_dbs = ($db1, $db2); > > > +foreach my $dbnam

Re: pg_upgrade: Support for upgrading to checksums enabled

2025-02-20 Thread Robert Treat
On Tue, Aug 27, 2024 at 5:57 PM Nathan Bossart wrote: > > On Mon, Aug 26, 2024 at 08:23:44AM +0200, Peter Eisentraut wrote: > > The purpose of this patch is to allow using pg_upgrade between clusters that > > have different checksum settings. When upgrading between instances with > > different ch

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2025-02-20 Thread Masahiko Sawada
On Thu, Feb 20, 2025 at 2:07 PM Noah Misch wrote: > > On Wed, Feb 19, 2025 at 10:48:29AM -0800, Masahiko Sawada wrote: > > Thank you for reviewing the patches. I've fixed these issues and > > attached the updated patches. > > Looks good. > > > I have one question about the 0001 patch; since we add

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-02-20 Thread Masahiko Sawada
On Fri, Feb 7, 2025 at 5:01 AM Sutou Kouhei wrote: > > Hi, > > In > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Tue, 4 Feb 2025 22:20:51 -0800, > Masahiko Sawada wrote: > > >> I was just looking at bit at this series of patch labelled with v31, > >> to see

Re: Proposal - Allow extensions to set a Plan Identifier

2025-02-20 Thread Sami Imseih
I put together patches to do as is being proposed. v1-0001: 1. Adds a planId field in PlannedStmt 2. Added an st_plan_id fields in PgBackendStatus 3. APIs to report and to retrieve a planId to PgBackendStatus An extension is able to set a planId in PlannedStmt directly, and while they can do the

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-02-20 Thread Daniel Gustafsson
> On 20 Feb 2025, at 04:36, Abhishek Chanda wrote: > Please find attached a new version of this patch. I rebased on master, > added better error reporting and skipped the permissions check on > windows. Please let me know if this needs any changes. I tested this > locally using meson running all T

Re: Add -k/--link option to pg_combinebackup

2025-02-20 Thread Israel Barth Rubio
Thanks for the review, Robert! I applied all of your suggestions. > I'm still not a fan of the changes to 010_links.pl; let's drop those. As discussed through a side chat, 010_links.pl is the new test file which ensures the hard links are created as expected in the output directory. I've removed

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-20 Thread Melanie Plageman
On Wed, Feb 19, 2025 at 4:14 PM Melanie Plageman wrote: > > Attached v30 makes the tuple offset extraction happen later as you > suggested. It turns out that none of the users need to worry much > about allocating and freeing -- I was able to have all users make an > offsets array on the stack. Pe

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2025-02-20 Thread Noah Misch
On Wed, Feb 19, 2025 at 10:48:29AM -0800, Masahiko Sawada wrote: > Thank you for reviewing the patches. I've fixed these issues and > attached the updated patches. Looks good. > I have one question about the 0001 patch; since we add > 'default_char_signedness' field to ControlFileData do we need

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-02-20 Thread Alena Rybakina
Hi! On 09.02.2025 18:38, Alexander Korotkov wrote: Also, aren't we too restrictive while requiring is_simple_values_sequence()? For instance, I believe cases like this (containing Var) could be transformed too. select * from t t1, lateral (select * from t t2 where t2.i in (values (t1.i), (1))

Reset the output buffer after sending from WalSndWriteData

2025-02-20 Thread Markus Wanner
Hi, I recently stumbled over an issue with an unintentional re-transmission. While this clearly was our fault in the output plugin code, I think the walsender's exposed API could easily be hardened to prevent the bad consequence from this mistake. Does anything speak against the attached one

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-20 Thread Daniel Gustafsson
> On 20 Feb 2025, at 21:21, Jacob Champion > wrote: > > On Thu, Feb 20, 2025 at 12:07 PM Tom Lane wrote: >> Looks like we're short a .gitignore entry. (It does appear that >> "make clean" cleans it up, at least.) > > So we are! Sorry about that. The attached patch gets in-tree builds > clean

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Andres Freund
Hi, On 2025-02-20 14:11:16 -0500, Tom Lane wrote: > As a matter of style, I wonder if it'd be better to have these > functions write into a caller-supplied variable. I wondered about that too, it does however make some code more awkward, e.g. because there's not a good surrounding block to put th

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-20 Thread Jacob Champion
On Thu, Feb 20, 2025 at 12:07 PM Tom Lane wrote: > Looks like we're short a .gitignore entry. (It does appear that > "make clean" cleans it up, at least.) So we are! Sorry about that. The attached patch gets in-tree builds clean for me again. Thanks, --Jacob fix-gitignore.patch Description: B

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-20 Thread Tom Lane
Daniel Gustafsson writes: > I spent a few more hours staring at this, and ran it through a number of CI > and > local builds, without anything showing up. Pushed to master with the first > set > of buildfarm animals showing green builds. After doing an in-tree "make check", I see $ git status

Re: Restrict copying of invalidated replication slots

2025-02-20 Thread Masahiko Sawada
On Wed, Feb 19, 2025 at 3:46 AM Shlok Kyal wrote: > > On Tue, 18 Feb 2025 at 15:26, Zhijie Hou (Fujitsu) > wrote: > > > > On Monday, February 17, 2025 7:31 PM Shlok Kyal > > wrote: > > > > > > On Thu, 13 Feb 2025 at 15:54, vignesh C wrote: > > > > > > > > On Tue, 4 Feb 2025 at 15:27, Shlok Kya

Re: Proposal: pg_is_volatile function

2025-02-20 Thread Pavel Stehule
Hi > What we have speculated about in the past is extending EXPLAIN > so that it can be applied to ALTER TABLE and other complicated > utility commands, and then for ALTER TABLE one bit of info it would > give you is whether a table rewrite (or even a table scan) is > required. Obviously, that's

Re: BUG #18815: Logical replication worker Segmentation fault

2025-02-20 Thread Sergey Belyashov
I have applied the patch. Replication now works. Thank you. Regards чт, 20 февр. 2025 г. в 00:55, Tom Lane : > > Sergey Belyashov writes: > > The 4th patch is not applicable for the current REL_17_STABLE branch. > > There are a lot of differences from master in the worker.c. > > If you want to t

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Tom Lane
Andres Freund writes: > Does anybody have opinions about whether we should keep a backward compatible > interface in place or not? I vote for "not" --- doesn't seem like there'll be much external code affected, and we make comparably-sized API breaks all the time. As a matter of style, I wonder

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Noah Misch
On Thu, Feb 20, 2025 at 12:40:57PM -0500, Andres Freund wrote: > On 2025-02-20 14:00:10 +1300, Thomas Munro wrote: > > On Wed, Feb 19, 2025 at 3:35 PM Andres Freund wrote: > > > After thinking about this for an embarassingly long time, I think there's > > > actually a considerably better answer fo

broken munhttps://github.com/munin-monitoring/contrib/issues/1483in plugins for PostgreSQL 17

2025-02-20 Thread Pavel Stehule
Hi Can somebody contact munin maintainers? The some plugins for PostgreSQL are particulari broken for PostgreSQL 15, but almost worked. When we migrated to PostgreSQL 17, then two plugins were fully broken. https://github.com/munin-monitoring/contrib/issues/1483 Unfortunately, the munin doesn't

Re: New buildfarm animals with FIPS mode enabled

2025-02-20 Thread Mark Wong
On Tue, Feb 18, 2025 at 02:41:18PM +0100, Álvaro Herrera wrote: > On 2025-Feb-17, Daniel Gustafsson wrote: > > > On 17 Feb 2025, at 20:23, Tom Lane wrote: > > > > Obviously, we could talk about extending the regression tests' > > > support for these cases, but I'm really dubious that it's worth

RE: Improve CRC32C performance on SSE4.2

2025-02-20 Thread Devulapalli, Raghuveer
> Now, there is no equivalent on MSVC and workarounds are fragile [1]. > Maybe we could only assert initialization happened for the backend and for > frontend either > - add a couple manual initializations to to the frontend programs where we > don't > want to lose performance for non-gcc/clang.

Re: Improvement of var_eq_non_const()

2025-02-20 Thread Tom Lane
Teodor Sigaev writes: > I'd like to suggest to improve var_eq_non_const() by using knowledge of MCV > and > estimate the selectivity as quadratic mean of non-null fraction divided by > number of distinct values (as it was before) and set of MCV selectivities. What's the statistical interpretat

Patch Review Workshop

2025-02-20 Thread Paul Jungwirth
Hello Hackers, I'm organizing a Postgres Patch Review Workshop as a way to help new Postgres contributors get experience and tips reviewing patches. Personally, I've been trying to review patches for years, but I've missed most commitfests, and I've never been very confident in my feedback. Rev

Re: Proposal: pg_is_volatile function

2025-02-20 Thread Tom Lane
Pavel Stehule writes: > čt 20. 2. 2025 v 13:48 odesílatel Andrew Farries > napsal: >> I believe the implementation of this function would be straightforward >> with a >> new function in `src/backend/utils/adt/misc.c` delegating to the existing >> `contain_volatile_functions_after_planning` functi

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-20 Thread Masahiko Sawada
On Wed, Feb 19, 2025 at 1:56 AM Bertrand Drouvot wrote: > > Hi, Thank you for looking at the patches. > > On Mon, Feb 17, 2025 at 12:07:56PM -0800, Masahiko Sawada wrote: > > On Fri, Feb 14, 2025 at 2:35 AM Bertrand Drouvot > > wrote: > > > Yeap, that was exactly my point when I mentioned the c

RE: SIMD optimization for list_sort

2025-02-20 Thread Devulapalli, Raghuveer
> Note that our current implemention is highly optimized for low-cardinality > inputs. > This is needed for aggregate queries. I found this write-up of a couple > scalar and > vectorized sorts, and they show this library doing very poorly on very-low > cardinality inputs. I would look into that b

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Andres Freund
Hi, On 2025-02-20 14:00:10 +1300, Thomas Munro wrote: > On Wed, Feb 19, 2025 at 3:35 PM Andres Freund wrote: > > After thinking about this for an embarassingly long time, I think there's > > actually a considerably better answer for a case like this: A function that > > returns a fixed-length str

Re: Proposal: pg_is_volatile function

2025-02-20 Thread Pavel Stehule
Hi čt 20. 2. 2025 v 13:48 odesílatel Andrew Farries napsal: > I'd like to propose a new function `pg_is_volatile` that would test and > return > the volatility of its argument expression. Example uses of the function > would > be: > > pg_is_volatile(1) -> false > pg_is_volatile(random()) -> true

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-20 Thread Jacob Champion
On Thu, Feb 20, 2025 at 8:30 AM Daniel Gustafsson wrote: > I spent a few more hours staring at this, and ran it through a number of CI > and > local builds, without anything showing up. Pushed to master with the first > set > of buildfarm animals showing green builds. Thank you!! And _huge tha

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-20 Thread Melanie Plageman
On Wed, Feb 19, 2025 at 8:29 AM Jakub Wartak wrote: > > On Fri, Feb 14, 2025 at 7:16 PM Andres Freund wrote: > > > Melanie has worked on this a fair bit, fwiw. > > > > My current thinking is that we'd want something very roughly like TCP > > BBR. Basically, it predicts the currently available ban

Re: Improve pgindent exclude handling: ignore empty lines

2025-02-20 Thread Andrew Dunstan
On 2025-02-18 Tu 3:02 PM, Zsolt Parragi wrote: Hello You are right, when I was writing the commit message I was thinking about trim, but the script only uses chomp. I edited the commit message to correctly describe the new behavior: ignoring only the empty line, as that was my main intention

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-20 Thread Melanie Plageman
On Sun, Feb 16, 2025 at 7:29 AM Tomas Vondra wrote: > > On 2/16/25 02:15, Tomas Vondra wrote: > > > > ... > > > > OK, I've uploaded the results to the github repository as usual > > > > https://github.com/tvondra/bitmapscan-tests/tree/main/20250214-184807 > > > > and I've generated the same PDF

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-20 Thread Daniel Gustafsson
> On 19 Feb 2025, at 15:13, Daniel Gustafsson wrote: > Unless something shows up I plan to commit this sometime tomorrow to allow it > ample time in the tree before the freeze. I spent a few more hours staring at this, and ran it through a number of CI and local builds, without anything showing

Re: Strange assertion in procarray.c

2025-02-20 Thread Mihail Nikalayeu
Hello! Noah, I have noticed you already disabled runningcheck for isolation tests already in injection_points[0]. The whole patch here was about to make it default for all types of tests for injection_points. Seems like we may close this entry. Attached patch is just to put a rebased version her

Re: Extend postgres_fdw_get_connections to return remote backend pid

2025-02-20 Thread Sagar Shedge
Hi Fujii, > Naming is always tricky, but remote_backend_pid feels a bit too long. Would remote_pid be sufficient? Point looks valid. I had another perspective is to align the naming convention to pg_backend_pid(). remote_pid is not helping to identify whether pid belongs to postgres backend or not

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-20 Thread Melanie Plageman
On Wed, Feb 19, 2025 at 6:03 PM Thomas Munro wrote: > > On Fri, Feb 14, 2025 at 9:32 AM Andres Freund wrote: > > I think we'll need to add some logic in read stream that only disables > > advice > > after a longer sequential sequence. Writing logic for that shouldn't be too > > hard, I think? De

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-20 Thread vignesh C
On Tue, 18 Feb 2025 at 15:59, Shlok Kyal wrote: > > On Mon, 17 Feb 2025 at 20:13, vignesh C wrote: > > > > On Fri, 14 Feb 2025 at 12:59, Shlok Kyal wrote: > > > > > > I have used the changes suggested by you. Also I have updated the > > > comments and the function name. > > > > There is another

Re: Statistics Import and Export

2025-02-20 Thread Andrew Dunstan
On 2025-02-20 Th 4:39 AM, Jeff Davis wrote: On Wed, 2025-02-12 at 19:00 -0800, Jeff Davis wrote: I'm still reviewing v48, but I intend to commit something soon. Committed with some revisions on top of v48: * removed the short option -X, leaving the long option "--statistics- only" with the s

Re: Fix logging for invalid recovery timeline

2025-02-20 Thread David Steele
On 2/19/25 19:45, Michael Paquier wrote: On Wed, Feb 19, 2025 at 05:35:18PM +, David Steele wrote: I like this idea but I would prefer to get the patch committed as-is first. The reason is that I'm hoping to see this batch-patched (since it is a bug) and that is less likely if the message wo

Re: IPC::Run::time[r|out] vs our TAP tests

2025-02-20 Thread Daniel Gustafsson
> On 20 Feb 2025, at 14:06, Andrew Dunstan wrote: > Actually, since ok() and friends return true iff the test succeeds, instead of > +ok(! $self->{timeout}->is_expired, 'psql query_until did not time out'); > +return undef if $self->{timeout}->is_expired; > you can avoid doing the same te

Re: Non-text mode for pg_dumpall

2025-02-20 Thread Mahendra Singh Thalor
On Thu, 20 Feb 2025 at 14:48, jian he wrote: > > hi. > about 0001 > > /* > * connectDatabase > * > * Make a database connection with the given parameters. An > * interactive password prompt is automatically issued if required. > * > * If fail_on_error is false, we return NULL without printi

Missing [NO] INDENT flag in XMLSerialize backward parsing

2025-02-20 Thread Jim Jones
Hi, This patch adds the missing [NO] INDENT flag to XMLSerialize backward parsing. For example: CREATE VIEW v1 AS SELECT   xmlserialize(     DOCUMENT '42'::xml AS text     INDENT); \sv v1 CREATE OR REPLACE VIEW public.v1 AS  SELECT XMLSERIALIZE(DOCUMENT '42'::xml AS  text INDENT) AS "xmlserializ

Re: Enhancing Memory Context Statistics Reporting

2025-02-20 Thread Rahila Syed
Hi, Please find attached the updated patches after some cleanup and test fixes. Thank you, Rahila Syed On Tue, Feb 18, 2025 at 6:35 PM Rahila Syed wrote: > Hi, > >> >> Thanks for updating the patch! >> >> The below comments would be a bit too detailed at this stage, but I’d >> like to share th

Re: IPC::Run::time[r|out] vs our TAP tests

2025-02-20 Thread Andrew Dunstan
On 2025-02-19 We 6:56 PM, Daniel Gustafsson wrote: On 19 Feb 2025, at 23:08, Andrew Dunstan wrote: On 2024-10-31 Th 6:18 PM, Heikki Linnakangas wrote: Thanks for review! Perhaps sommething like this: "Close the psql session and clean up resources. Each psql session must be closed with C bef

RE: ReplicationSlotRelease() crashes when the instance is in the single user mode

2025-02-20 Thread Hayato Kuroda (Fujitsu)
Dear hackers, Thanks everyone for giving comments! PSA new version. What's new: - Message format was modified to {"cannot use function %s in single-user mode", "function_name"} - Reporting funcname was adjusted based on the parameters. ternary operator was used. - Guard was added for functions

Proposal: pg_is_volatile function

2025-02-20 Thread Andrew Farries
I'd like to propose a new function `pg_is_volatile` that would test and return the volatility of its argument expression. Example uses of the function would be: pg_is_volatile(1) -> false pg_is_volatile(random()) -> true The motivation for the proposal is to allow testing of column default expres

Re: Issues with ON CONFLICT UPDATE and REINDEX CONCURRENTLY

2025-02-20 Thread Міхаіл Нікалаеў
Hello! Just rebased. From 2db0af6a7b5ba485464ad5a59ce106a6e438d41a Mon Sep 17 00:00:00 2001 From: nkey Date: Thu, 20 Feb 2025 14:50:58 +0300 Subject: [PATCH v8 2/4] Modify the infer_arbiter_indexes function to consider both indisvalid and indisready indexes. Ensure that at least one indisvalid

RE: ReplicationSlotRelease() crashes when the instance is in the single user mode

2025-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > Also, what about pg_replication_origin_* APIs? Do we want to restrict > those as well if we are restricting slot operations? I don't see any > solid theory presented in this thread on why we should add new checks > in multiple APIs restricting those in single-user mode. As David [1]

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-02-20 Thread Shlok Kyal
On Mon, 17 Feb 2025 at 16:20, Ajin Cherian wrote: > > On Mon, Feb 17, 2025 at 10:08 AM Peter Smith wrote: > > > > Hi Ajin. > > > > FYI - Patch set v13* no longer applies cleanly. Needs rebasing. > > > I've rebased the patch. I've also merged patch 1 into patch 2 as the > functionality of the chan

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-02-20 Thread Ashutosh Bapat
On Tue, Feb 4, 2025 at 4:07 PM Ashutosh Bapat wrote: > > If we are not interested in saving memory, there is a simpler way to > improve planning time by adding a hash table per equivalence class to > store the derived clauses, instead of a linked list, when the number > of derived clauses is highe

Re: ReplicationSlotRelease() crashes when the instance is in the single user mode

2025-02-20 Thread Amit Kapila
On Thu, Feb 20, 2025 at 4:26 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, February 20, 2025 10:23 AM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Michael, > > > > > I did not check how these call behave individually, just a few > > > comments while putting my eyes on the patch. > > > > > > +

Conflict detection for multiple_unique_conflicts in logical replication

2025-02-20 Thread Nisha Moond
Hi Hackers, (CCing people involved in related discussions) I am starting this thread to propose a new conflict detection type "multiple_unique_conflicts" that identifies when an incoming row during logical replication violates more than one UNIQUE constraint. If multiple constraints (such as the p

RE: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Shubham, Thanks for updating the patch quickly! > > 04. > > ``` > > +# Verify that only user databases got subscriptions (not template > > databases) > > +my @user_dbs = ($db1, $db2); > > +foreach my $dbname (@user_dbs) > > +{ > > + my $sub_exists = > > + $node_s1->safe_psql($

Re: Add XMLNamespaces to XMLElement

2025-02-20 Thread Jim Jones
I've attached v6, which addresses two issues: * Fixes a bug where XMLNAMESPACES declarations within a view were being serialized as XMLATTRIBUTES. * Prevents the makeString function from being called with a NULL parameter - discussed in this thread [1]. Best regards, Jim [1] https://www.postgres

Re: Commitfest app release on Feb 17 with many improvements

2025-02-20 Thread Thomas Munro
On Thu, Feb 20, 2025 at 11:53 PM Jelte Fennema-Nio wrote: > On Thu, 20 Feb 2025 at 11:07, Thomas Munro wrote: > > When the layout of https://commitfest.postgresql.org/52/ changed, > > cfbot's web scraping logic could no longer find the authors :-(. > > That's a stupid problem to have, that we are

Assert when executing query on partitioned table

2025-02-20 Thread Dmitry Koval
Hi! I got an Assert when executing an "INSERT ... ON CONFLICT ... UPDATE ..." query on partitioned table. Managed to reproduce this situation. Reproduction order. --- 1) Apply the patch [v1-0001-Triggering-Assert-on-query-with-ON-CONFLICT.patch] to "master" branch. 2) Buil

RE: create subscription with (origin = none, copy_data = on)

2025-02-20 Thread Zhijie Hou (Fujitsu)
On Wednesday, January 29, 2025 8:19 PM Shlok Kyal wrote: > > I have addressed the comments. Here is an updated patch. Thanks for updating the patch. The patches look mostly OK to me, I only have one minor comments in 0002. 1. +CREATE PUBLICATION pub_b_c_2 FOR TABLE tab_part2_1; +)); + +($resu

Re: Commitfest app release on Feb 17 with many improvements

2025-02-20 Thread Tatsuo Ishii
>> On Thu, Feb 20, 2025 at 10:53 PM Tatsuo Ishii wrote: >> > I noticed some of entries are shown with the author field being empty. >> > e.g. https://commitfest.postgresql.org/patch/5525/ >> >> When the layout of https://commitfest.postgresql.org/52/ changed, >> cfbot's web scraping logic could no

RE: ReplicationSlotRelease() crashes when the instance is in the single user mode

2025-02-20 Thread Zhijie Hou (Fujitsu)
On Thursday, February 20, 2025 10:23 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Michael, > > > I did not check how these call behave individually, just a few > > comments while putting my eyes on the patch. > > > > + if (!IsUnderPostmaster) > > + elog(ERROR, > > +

Re: Commitfest app release on Feb 17 with many improvements

2025-02-20 Thread Jelte Fennema-Nio
On Thu, 20 Feb 2025 at 11:07, Thomas Munro wrote: > > On Thu, Feb 20, 2025 at 10:53 PM Tatsuo Ishii wrote: > > I noticed some of entries are shown with the author field being empty. > > e.g. https://commitfest.postgresql.org/patch/5525/ > > When the layout of https://commitfest.postgresql.org/52/

Re: ReplicationSlotRelease() crashes when the instance is in the single user mode

2025-02-20 Thread Bertrand Drouvot
Hi, On Thu, Feb 20, 2025 at 02:22:41AM +, Hayato Kuroda (Fujitsu) wrote: > Dear Michael, > Thanks for the report and the patch! > > I did not check how these call behave individually, just a few > > comments while putting my eyes on the patch. > > > > + if (!IsUnderPostmaster) > > +

Re: Statistics Import and Export

2025-02-20 Thread vignesh C
On Thu, 20 Feb 2025 at 15:09, Jeff Davis wrote: > > On Wed, 2025-02-12 at 19:00 -0800, Jeff Davis wrote: > > I'm still reviewing v48, but I intend to commit something soon. > > Committed with some revisions on top of v48: I was checking buildfarm for another commit of mine, while checking I notic

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-20 Thread Shubham Khanna
On Thu, Feb 20, 2025 at 1:23 PM Peter Smith wrote: > > Hi Shubham, Here are some review comments for v9-0001. > > == > Commit message > > 1. > You've changed the option to '--all' but the comment message still > refers to '-all-databases' > Fixed. > == > doc/src/sgml/ref/pg_createsubscri

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-20 Thread Shubham Khanna
On Wed, Feb 19, 2025 at 4:23 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for updating the patch! > > > I have added a test case for non-dry-run mode to ensure that > > replication slots, subscriptions, and publications work as expected > > when '--all' is specified. Additionally

Re: Commitfest app release on Feb 17 with many improvements

2025-02-20 Thread Thomas Munro
On Thu, Feb 20, 2025 at 10:53 PM Tatsuo Ishii wrote: > I noticed some of entries are shown with the author field being empty. > e.g. https://commitfest.postgresql.org/patch/5525/ When the layout of https://commitfest.postgresql.org/52/ changed, cfbot's web scraping logic could no longer find the

Re: Commitfest app release on Feb 17 with many improvements

2025-02-20 Thread Tatsuo Ishii
> At the FOSDEM dev meeting we discussed potential improvements to the > commitfest app and how to handle deploying future changes with minimal > disruption to existing workflows. We're going to try a new approach: > announcing a new commitfest release ~2 weeks in advance, including > release notes

RE: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Ajin, Here are my comments. I must play with patches to understand more. 01. ``` extern bool ReorderBufferFilterByRelFileLocator(ReorderBuffer *rb, TransactionId xid, XLogRecPtr lsn, RelFileLoca

Re: Statistics Import and Export

2025-02-20 Thread Jeff Davis
On Wed, 2025-02-12 at 19:00 -0800, Jeff Davis wrote: > I'm still reviewing v48, but I intend to commit something soon. Committed with some revisions on top of v48: * removed the short option -X, leaving the long option "--statistics- only" with the same meaning. * removed the redundant --with-st

  1   2   >