Re: table inheritance versus column compression and storage settings

2024-02-06 Thread Ashutosh Bapat
On Wed, Jan 31, 2024 at 1:29 PM Ashutosh Bapat wrote: > > On Tue, Dec 5, 2023 at 6:28 PM Peter Eisentraut wrote: > > > > On 05.12.23 05:26, Ashutosh Bapat wrote: > > >> - When inheriting from multiple parents with different settings, an > > >> explicit setting in the child is required. > > > When

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-06 Thread Andrey M. Borodin
> On 7 Feb 2024, at 10:58, Dilip Kumar wrote: > >> commit_timestamp_slru_buffers >> transaction_slru_buffers >> etc > > I am not sure we are exposing anything related to SLRU to the user, I think we already tell something about SLRU to the user. I’d rather consider if “transaction_slru_buf

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-06 Thread Dilip Kumar
On Tue, Feb 6, 2024 at 8:55 PM Alvaro Herrera wrote: > > Here's the rest of it rebased on top of current master. I think it > makes sense to have this as one individual commit. > > I made CLOGShmemBuffers, CommitTsShmemBuffers and SUBTRANSShmemBuffers > compute a number that's multiple of SLRU_BA

RE: speed up a logical replica setup

2024-02-06 Thread Hayato Kuroda (Fujitsu)
Dear Euler, Sorry for posting e-mail each other. I will read your patch but I want to reply one of yours. > I started reviewing v16-0007 but didn't finish yet. The general idea is ok. However, I'm still worried about preventing some use cases if it provides only the local connection option. What

Re: Synchronizing slots from primary to standby

2024-02-06 Thread shveta malik
On Tue, Feb 6, 2024 at 7:21 PM Zhijie Hou (Fujitsu) wrote: > > > --- > > +/* Slot sync worker objects */ > > +extern PGDLLIMPORT char *PrimaryConnInfo; extern PGDLLIMPORT char > > +*PrimarySlotName; > > > > These two variables are declared also in xlogrecovery.h. Is it intentional? > > If so, I >

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-06 Thread vignesh C
On Tue, 6 Feb 2024 at 08:30, Alexander Lakhin wrote: > > Hello Amit, > > 05.02.2024 15:20, Amit Kapila wrote: > > If this can be reproduced frequently then we can even try to test the > > patch in that thread by Osumi-San [1] (I haven't tested that it > > applies cleanly but shouldn't be difficult

RE: speed up a logical replica setup

2024-02-06 Thread Hayato Kuroda (Fujitsu)
Dear Shlok, Thanks for updating the patch! > I have created a topup patch 0007 on top of v15-0006. > > I revived the patch which removes -S option and adds some options > instead. The patch add option for --port, --username and --socketdir. > This patch also ensures that anyone cannot connect to

Re: RFC: Logging plan of the running query

2024-02-06 Thread Ashutosh Bapat
On Wed, Feb 7, 2024 at 9:38 AM torikoshia wrote: > > Hi Ashutosh, > > On 2024-02-06 19:51, Ashutosh Bapat wrote: > > > Thanks for the summary. It is helpful. I think patch is also getting > > better. > > > > I have a few questions and suggestions > > Thanks for your comments. > > > 1. Prologue of

Re: speed up a logical replica setup

2024-02-06 Thread Euler Taveira
On Fri, Feb 2, 2024, at 6:41 AM, Hayato Kuroda (Fujitsu) wrote: > Thanks for updating the patch! Thanks for taking a look. > > > I'm still working on the data structures to group options. I don't like the > way > it was grouped in v13-0005. There is too many levels to reach database name. > The

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-06 Thread Michael Paquier
On Tue, Feb 06, 2024 at 03:33:36PM -0800, Andres Freund wrote: > Well, you can't just do that, because there's only one caller, namely > CopyToTextOneRow(). What I am trying to suggest is something like the > attached, just a quick hacky POC. Namely to split out CSV support from > CopyToTextOneRow(

RE: Psql meta-command conninfo+

2024-02-06 Thread Maiquel Grassi
On Tue, Feb 06, 2024 at 09:45:54PM +, Maiquel Grassi wrote: > My initial idea has always been that they should continue to appear > because \conninfo+ should show all the things that \conninfo shows and > add more information. I think that's the purpose of the 'plus.' Now we're > on a

Re: RFC: Logging plan of the running query

2024-02-06 Thread torikoshia
Hi Ashutosh, On 2024-02-06 19:51, Ashutosh Bapat wrote: Thanks for the summary. It is helpful. I think patch is also getting better. I have a few questions and suggestions Thanks for your comments. 1. Prologue of GetLockMethodLocalHash() mentions * NOTE: When there are many entries in Lo

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Peter Smith
Here are some review comments for v79-0001 == Commit message 1. The logical replication slots on the primary can be synchronized to the hot standby by enabling the "failover" parameter during pg_create_logical_replication_slot() or by enabling "failover" option of the CREATE SUBSCRIPTION comm

Re: Allow passing extra options to initdb for tests

2024-02-06 Thread Ian Lawrence Barwick
2024年2月6日(火) 19:54 Peter Eisentraut : > > I'm proposing here a way to pass extra options to initdb when run > internally during test setup in pg_regress or > PostgreSQL::Test::Cluster's init (which covers just about all test > suites other than initdb's own tests). > > For example: > > make ch

Re: clarify equalTupleDescs()

2024-02-06 Thread jian he
On Tue, Feb 6, 2024 at 8:59 PM Peter Eisentraut wrote: > > In a recent patch thread it was discussed[0] which fields should be > compared by equalTupleDescs() and whether it is ok to remove a field > from tuple descriptors and how that should affect their equality > (attstattarget in that case). >

Re: common signal handler protection

2024-02-06 Thread Andres Freund
Hi, On 2024-02-06 20:39:41 -0600, Nathan Bossart wrote: > On Tue, Nov 21, 2023 at 03:20:08PM -0600, Nathan Bossart wrote: > > * Overhead: The wrapper handler calls a function pointer and getpid(), > > which AFAICT is a real system call on most platforms. That might not be > > a tremendous amo

Re: common signal handler protection

2024-02-06 Thread Nathan Bossart
On Tue, Nov 21, 2023 at 03:20:08PM -0600, Nathan Bossart wrote: > * Overhead: The wrapper handler calls a function pointer and getpid(), > which AFAICT is a real system call on most platforms. That might not be > a tremendous amount of overhead, but it's not zero, either. I'm > particularly

Re: pgbnech: allow to cancel queries during benchmark

2024-02-06 Thread Yugo NAGATA
On Wed, 24 Jan 2024 22:17:44 +0900 Yugo NAGATA wrote: > Attached is the updated patch, v6. Currently, on non-Windows, SIGINT is received only by thread #0. CancelRequested is checked during the loop in the thread, and queries are cancelled if it is set. However, once thread #0 exits the loop d

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Masahiko Sawada
On Tue, Feb 6, 2024 at 8:21 PM Amit Kapila wrote: > > On Tue, Feb 6, 2024 at 3:33 PM Amit Kapila wrote: > > > > On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila > > > wrote: > > > > > > > > I think users can refer to LOGs to see

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-06 Thread Andres Freund
Hi, On 2024-02-06 15:11:05 +0900, Michael Paquier wrote: > On Mon, Feb 05, 2024 at 09:46:42PM -0800, Andres Freund wrote: > > No - what I mean is that it doesn't make sense to have copy_attribute_out(), > > as e.g. CopyToTextOneRow() already knows that it's dealing with text, so it > > can directl

Re: Properly pathify the union planner

2024-02-06 Thread Andy Fan
Hi, > * I think we should update truncate_useless_pathkeys() to account for > the ordering requested by the query's set operation; Nice catch. > I'm thinking that maybe it'd be better to move the work of sorting the > subquery's paths to the outer query level, specifically within the > build_se

Re: confusing / inefficient "need_transcoding" handling in copy

2024-02-06 Thread Andres Freund
Hi, On 2024-02-06 12:51:48 -0500, Tom Lane wrote: > Michael Paquier writes: > > On Mon, Feb 05, 2024 at 06:05:04PM -0800, Andres Freund wrote: > >> I haven't yet dug into the code history. One guess is that this should only > >> have been set this way for COPY FROM. > > > Looking the git history,

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 09:45:54PM +, Maiquel Grassi wrote: > My initial idea has always been that they should continue to appear > because \conninfo+ should show all the things that \conninfo shows and > add more information. I think that's the purpose of the 'plus.' Now we're > on a better pa

RE: Psql meta-command conninfo+

2024-02-06 Thread Maiquel Grassi
On Tue, Feb 06, 2024 at 03:06:05PM -0600, Nathan Bossart wrote: > On Tue, Feb 06, 2024 at 08:52:09PM +, Maiquel Grassi wrote: >> I made the adjustment in the code and updated the patch. I believe this >> is the format suggested by you all. Would this be it? > > I was thinking someth

RE: Psql meta-command conninfo+

2024-02-06 Thread Maiquel Grassi
On Tue, Feb 06, 2024 at 08:52:09PM +, Maiquel Grassi wrote: > I made the adjustment in the code and updated the patch. I believe this > is the format suggested by you all. Would this be it? I was thinking something more like SELECT pg_catalog.current_database() AS "Database",

Re: Where can I find the doxyfile?

2024-02-06 Thread Peter Eisentraut
On 02.02.24 01:19, John Morris wrote: Here is the updated patch. It should fix the meson issue when no doxygen is present. I think all the explanatory messages in doc/doxygen/meson.build are a bit much. I think it's enough to just not define the target when the required conditions (dependenc

Re: Where can I find the doxyfile?

2024-02-06 Thread Peter Eisentraut
On 05.02.24 18:29, John Morris wrote: The purpose of the filter is to bring existing Postgres comments into the doxygen output. While I haven’t done a full survey, the majority of Postgres code has comments describing functions, globals, macros and structure fields. Currently, those comments

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 03:06:05PM -0600, Nathan Bossart wrote: > On Tue, Feb 06, 2024 at 08:52:09PM +, Maiquel Grassi wrote: >> I made the adjustment in the code and updated the patch. I believe this >> is the format suggested by you all. Would this be it? > > I was thinking something more li

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 08:52:09PM +, Maiquel Grassi wrote: > I made the adjustment in the code and updated the patch. I believe this > is the format suggested by you all. Would this be it? I was thinking something more like SELECT pg_catalog.current_database() AS "Database", curre

Re: glibc qsort() vulnerability

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 03:55:58PM -0500, Tom Lane wrote: > A comparison routine that is not is probably broken, agreed. > I didn't look through the details of the patch --- I was more > curious whether we had a version of the qsort bug, because > if we do, we should fix that too. +1 -- Nathan B

Re: glibc qsort() vulnerability

2024-02-06 Thread Tom Lane
Nathan Bossart writes: > Even if the glibc issue doesn't apply to Postgres, I'm tempted to suggest > that we make it project policy that comparison functions must be > transitive. There might be no real issues today, but if we write all > comparison functions the way Mats is suggesting, it should

Re: glibc qsort() vulnerability

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 10:11:16AM -0500, Tom Lane wrote: > Mats Kindahl writes: >> There is a bug in glibc's qsort() algorithm that runs the risk of creating >> an out-of-bounds error if the comparison function is not transitive, for >> example, if subtraction is used so that it can create an ove

RE: Psql meta-command conninfo+

2024-02-06 Thread Maiquel Grassi
On 2024-02-06 19:19 +0100, Nathan Bossart wrote: > On Tue, Feb 06, 2024 at 05:27:01PM +, Maiquel Grassi wrote: > > postgres=# \conninfo+ > > Current Connection Information > >Attribute| Value > > + > > Database | postgres > >

Re: Remove Start* macros from postmaster.c to ease understanding of code

2024-02-06 Thread Nathan Bossart
On Wed, Feb 07, 2024 at 12:48:00AM +0530, Bharath Rupireddy wrote: > +1. Patch LGTM. Unless there are objections, I'll plan on committing this in the next day or two. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-06 Thread Tom Lane
I wrote: > More to the point, aren't these proposals just band-aids that > would stabilize the test without fixing the actual problem? > The same thing is likely to happen to people in the field, > unless we do something drastic like removing ALTER SUBSCRIPTION. I've been able to make the 031_colu

Re: Remove Start* macros from postmaster.c to ease understanding of code

2024-02-06 Thread Bharath Rupireddy
On Tue, Feb 6, 2024 at 10:34 PM wrote: > > Hi, > > Attached is a small patch implemented as I agree with Andres' comment > below noted while reviewing the thread > https://www.postgresql.org/message-id/flat/20240122210740.7vq5fd4woixpwx3f%40awork3.anarazel.de#6eb7595873392621d60e6b5a723941bc > > I

Re: Psql meta-command conninfo+

2024-02-06 Thread Erik Wienhold
On 2024-02-06 19:19 +0100, Nathan Bossart wrote: > On Tue, Feb 06, 2024 at 05:27:01PM +, Maiquel Grassi wrote: > > postgres=# \conninfo+ > > Current Connection Information > >Attribute| Value > > + > > Database | postgres > > User |

Re: Set log_lock_waits=on by default

2024-02-06 Thread Laurenz Albe
On Tue, 2024-02-06 at 12:29 -0500, Tom Lane wrote: > Nathan Bossart writes: > > It looks like there are two ideas: > > [...] > > * Set log_lock_waits on by default. The folks on this thread seem to > > support this idea, but given the lively discussion for enabling > > log_checkpoints by defa

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-02-06 Thread Bharath Rupireddy
On Mon, Feb 5, 2024 at 3:15 PM Bertrand Drouvot wrote: > > Thanks for the patch and +1 for the idea, I think adding those new > "invalidation reasons" make sense. Thanks for looking at it. > I think it's better to have the XID one being discussed/implemented before the > inactive_timeout one: wh

Re: Psql meta-command conninfo+

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 05:27:01PM +, Maiquel Grassi wrote: > I'm seeking to improve the \conninfo meta-command in psql. Currently, it > provides limited information about the current connection. I believe that > expanding it using the concept of "plus" [+] could ease the work of DBAs, > SysAdm

RE: Popcount optimization using AVX512

2024-02-06 Thread Amonson, Paul D
Hi All, > However, it would be much better if the improved code were available for > all relevant builds and activated if a CPUID test determines that the > relevant instructions are available, instead of requiring a compile-time > flag -- which most builds are not going to use, thus wasting the >

Re: confusing / inefficient "need_transcoding" handling in copy

2024-02-06 Thread Tom Lane
Michael Paquier writes: > On Mon, Feb 05, 2024 at 06:05:04PM -0800, Andres Freund wrote: >> I haven't yet dug into the code history. One guess is that this should only >> have been set this way for COPY FROM. > Looking the git history, this looks like an oversight of c61a2f58418e > that has added

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-02-06 Thread Bharath Rupireddy
On Tue, Feb 6, 2024 at 2:16 PM Dilip Kumar wrote: > > > Thoughts? > > > +1 for the idea, here are some comments on 0002, I will review other > patches soon and respond. Thanks for looking at it. > + inactive_at timestamptz > > Maybe we can change the field name to 'last_inactive_at'? or m

Re: Set log_lock_waits=on by default

2024-02-06 Thread Tom Lane
Nathan Bossart writes: > It looks like there are two ideas: > * Separate log_lock_waits from deadlock_timeout. It looks like this idea > has a decent amount of support, but I didn't see any patch for it so far. I think the support comes from people who have not actually looked at the code. T

Psql meta-command conninfo+

2024-02-06 Thread Maiquel Grassi
Hi, I'm seeking to improve the \conninfo meta-command in psql. Currently, it provides limited information about the current connection. I believe that expanding it using the concept of "plus" [+] could ease the work of DBAs, SysAdmins, DevOps, etc., who manage a large volume of databases and/o

Re: Remove Start* macros from postmaster.c to ease understanding of code

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 11:58:50AM -0500, reid.thomp...@crunchydata.com wrote: > Attached is a small patch implemented as I agree with Andres' comment > below noted while reviewing the thread > https://www.postgresql.org/message-id/flat/20240122210740.7vq5fd4woixpwx3f%40awork3.anarazel.de#6eb759587

Remove Start* macros from postmaster.c to ease understanding of code

2024-02-06 Thread reid . thompson
Hi, Attached is a small patch implemented as I agree with Andres' comment below noted while reviewing the thread https://www.postgresql.org/message-id/flat/20240122210740.7vq5fd4woixpwx3f%40awork3.anarazel.de#6eb7595873392621d60e6b5a723941bc I agree that its easier to not have to refer back to th

Re: Reducing connection overhead in pg_upgrade compat check phase

2024-02-06 Thread Nathan Bossart
On Tue, Feb 06, 2024 at 05:47:56PM +0100, Daniel Gustafsson wrote: >> On 6 Feb 2024, at 17:32, Nathan Bossart wrote: >> IMHO this patch is worth trying to get into v17. I'd be happy to take it >> forward if Daniel does not intend to work on it. > > I actually had the same thought yesterday and s

Re: Set log_lock_waits=on by default

2024-02-06 Thread Nathan Bossart
I saw this was marked ready-for-committer, so I took a look at the thread. It looks like there are two ideas: * Separate log_lock_waits from deadlock_timeout. It looks like this idea has a decent amount of support, but I didn't see any patch for it so far. IMHO this is arguably a prerequisite

Re: Reducing connection overhead in pg_upgrade compat check phase

2024-02-06 Thread Daniel Gustafsson
> On 6 Feb 2024, at 17:32, Nathan Bossart wrote: > > On Fri, Feb 02, 2024 at 12:18:25AM +0530, vignesh C wrote: >> With no update to the thread and the patch still not applying I'm >> marking this as returned with feedback. Please feel free to resubmit >> to the next CF when there is a new versi

Re: Reducing connection overhead in pg_upgrade compat check phase

2024-02-06 Thread Nathan Bossart
On Fri, Feb 02, 2024 at 12:18:25AM +0530, vignesh C wrote: > With no update to the thread and the patch still not applying I'm > marking this as returned with feedback. Please feel free to resubmit > to the next CF when there is a new version of the patch. IMHO this patch is worth trying to get i

Re: pg_stat_advisor extension

2024-02-06 Thread Ilia Evdokimov
Dear Team, Firstly, I would like to extend my sincere apologies for the confusion and technical oversights in our previous discussions regarding the 'pg_stat_advisor extension'. To address this and facilitate a clearer, more focused dialogue, I have initiated a new thread to consolidate our d

Re: pg_stat_advisor extension

2024-02-06 Thread Ilia Evdokimov
Hi hackers, I've encountered and addressed errors in the "0001-pg_stat_advisor-extension.patch" when applying it to the main branch, specifically trailing whitespace issues at lines 117 and 118: ``` 0001-pg_stat_advisor-extension.patch:117: trailing whitespace.   

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-06 Thread Alvaro Herrera
Here's the rest of it rebased on top of current master. I think it makes sense to have this as one individual commit. I made CLOGShmemBuffers, CommitTsShmemBuffers and SUBTRANSShmemBuffers compute a number that's multiple of SLRU_BANK_SIZE. But it's a crock, because we don't have that macro at t

Re: Possibility to disable `ALTER SYSTEM`

2024-02-06 Thread Jelte Fennema-Nio
On Tue, 30 Jan 2024 at 18:49, Robert Haas wrote: > I also think that using the GUC system to manage itself is a little > bit suspect. I wonder if it would be better to do this some other way, > e.g. a sentinel file in the data directory. For example, suppose we > refuse ALTER SYSTEM if $PGDATA/dis

Re: clarify equalTupleDescs()

2024-02-06 Thread Tom Lane
Peter Eisentraut writes: > The first want to compare what I call row-type equality, which means > they want to check specifically for equal number of attributes, and the > same attribute names, types, and typmods for each attribute. Most > callers actually want that behavior. Should compare a

Re: glibc qsort() vulnerability

2024-02-06 Thread Tom Lane
Mats Kindahl writes: > There is a bug in glibc's qsort() algorithm that runs the risk of creating > an out-of-bounds error if the comparison function is not transitive, for > example, if subtraction is used so that it can create an overflow. We don't use glibc's qsort. Have you checked whether t

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-06 Thread Tom Lane
Amit Kapila writes: > Yeah, I was worried about that. The other idea I have previously > thought was to change Alter Subscription to Drop+Create Subscription. > That should also help in bringing stability without losing any > functionality. Hm, why would that fix it? More to the point, aren't th

Re: Possibility to disable `ALTER SYSTEM`

2024-02-06 Thread David G. Johnston
On Tue, Feb 6, 2024 at 7:10 AM Peter Eisentraut wrote: > > How about ALTER SYSTEM is disabled if the file > postgresql.auto.conf.disabled exists? This is somewhat similar to making > the file read-only, but doesn't risk other tools breaking when they > encounter such a file. And it's more obviou

Re: Possibility to disable `ALTER SYSTEM`

2024-02-06 Thread Peter Eisentraut
On 31.01.24 11:16, Gabriele Bartolini wrote: I very much like the idea of a file in the data directory that also controls the copy operations. Just wanted to highlight though that in our operator we have already applied the read-only postgresql.auto.conf trick to disable the system (see http

glibc qsort() vulnerability

2024-02-06 Thread Mats Kindahl
Hi hackers, There is a bug in glibc's qsort() algorithm that runs the risk of creating an out-of-bounds error if the comparison function is not transitive, for example, if subtraction is used so that it can create an overflow. See https://packetstormsecurity.com/files/176931/glibc-qsort-Out-Of-Bo

RE: Synchronizing slots from primary to standby

2024-02-06 Thread Zhijie Hou (Fujitsu)
On Monday, February 5, 2024 10:25 PM Masahiko Sawada wrote: lvh.no-ip.org> > Subject: Re: Synchronizing slots from primary to standby > > On Mon, Feb 5, 2024 at 8:26 PM shveta malik > wrote: > > > > On Mon, Feb 5, 2024 at 10:57 AM Ajin Cherian wrote: > > > > > > Just noticed that doc/src/sgml/

RE: Synchronizing slots from primary to standby

2024-02-06 Thread Zhijie Hou (Fujitsu)
On Tuesday, February 6, 2024 3:39 PM Masahiko Sawada wrote: > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila wrote: > > > > On Mon, Feb 5, 2024 at 7:56 PM Masahiko Sawada > wrote: > > > > > > --- > > > Since Two processes (e.g. the slotsync worker and > > > pg_sync_replication_slots()) concurren

Re: Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2024-02-06 Thread Ashutosh Bapat
On Mon, Nov 6, 2023 at 2:48 PM Ashutosh Bapat wrote: > > explain.out in 0001 needed some adjustments. Without those CIbot shows > failures. Fixed in the attached patchset. 0001 is just for diagnosis, > not for actual commit. 0002 which is the actual patch has no changes > wrt to the previous versi

Re: Race condition in FetchTableStates() breaks synchronization of subscription tables

2024-02-06 Thread Alexander Lakhin
05.02.2024 13:13, vignesh C wrote: Thanks for the steps for the issue, I was able to reproduce this issue in my environment with the steps provided. The attached patch has a proposed fix where the latch will not be set in case of the apply worker exiting immediately after starting. It looks lik

clarify equalTupleDescs()

2024-02-06 Thread Peter Eisentraut
In a recent patch thread it was discussed[0] which fields should be compared by equalTupleDescs() and whether it is ok to remove a field from tuple descriptors and how that should affect their equality (attstattarget in that case). After analyzing all the callers, I have noticed that there are

Re: Memory consumed by paths during partitionwise join planning

2024-02-06 Thread Ashutosh Bapat
On Fri, Dec 15, 2023 at 5:22 AM Ashutosh Bapat wrote: > > > > That looks pretty small considering the benefits. What do you think? > > > > [1] > > https://www.postgresql.org/message-id/caexhw5stmouobe55pmt83r8uxvfcph+pvo5dnpdrvcsbgxe...@mail.gmail.com > > If you want to experiment, please use at

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-06 Thread Amit Kapila
On Tue, Feb 6, 2024 at 2:30 PM Alexander Lakhin wrote: > > 06.02.2024 09:48, Amit Kapila wrote: > > cool, is it possible to see whether this patch changes the runtime of > > this test in any noticeable way? > > > > Yes, unfortunately it does. > I've measured duration of 100 tests runs without the

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Amit Kapila
On Tue, Feb 6, 2024 at 12:08 PM Peter Smith wrote: > > Hi, I took another high-level look at all the funtion names of the > slotsync.c file. > > > Below are some suggestions (some are unchanged); probably there are > better ideas for names but my point is that the current names could be > improved

Re: [PATCH] Add native windows on arm64 support

2024-02-06 Thread Dave Cramer
On Wed, 31 Jan 2024 at 10:21, Andrew Dunstan wrote: > > On 2024-01-30 Tu 17:54, Dave Cramer wrote: > > > > > On Tue, Jan 30, 2024 at 4:56 PM Andrew Dunstan > wrote: > >> >> On 2024-01-30 Tu 09:50, Dave Cramer wrote: >> >> >> >> On Tue, 30 Jan 2024 at 08:38, Andrew Dunstan wrote: >> >>> >>> On 2

Re: Memory leak fix in rmtree.c

2024-02-06 Thread Daniel Gustafsson
> On 6 Feb 2024, at 12:45, Ильясов Ян wrote: > > I agree with your argument. > Thank you for your time. Thank you for your report! -- Daniel Gustafsson

Re: recently added jsonpath method change jsonb_path_query, jsonb_path_query_first immutability

2024-02-06 Thread Andrew Dunstan
On 2024-02-05 Mo 22:06, jian he wrote: Hi. this commit [0] changes immutability of jsonb_path_query, jsonb_path_query_first? If so, it may change other functions also. demo: begin; SET LOCAL TIME ZONE 10.5; with cte(s) as (select jsonb '"2023-08-15 12:34:56 +05:30"') select jsonb_path_quer

RE: Memory leak fix in rmtree.c

2024-02-06 Thread Ильясов Ян
I agree with your argument. Thank you for your time. Kind regards, Ian Ilyasov Juniour Software Developer at Postgres Professional

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Amit Kapila
On Tue, Feb 6, 2024 at 3:57 PM Dilip Kumar wrote: > > On Tue, Feb 6, 2024 at 3:41 PM Amit Kapila wrote: > > > > On Tue, Feb 6, 2024 at 3:23 PM Dilip Kumar wrote: > > > > > > On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada > > > wrote: > > > > > > > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-06 Thread Dilip Kumar
On Tue, Feb 6, 2024 at 4:23 PM Alvaro Herrera wrote: > > > > (We also have SimpleLruTruncate, but I think it's not as critical to > > > have a barrier there anyhow: accessing a slightly outdated page number > > > could only be a problem if a bug elsewhere causes us to try to truncate > > > in the

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Amit Kapila
On Tue, Feb 6, 2024 at 3:33 PM Amit Kapila wrote: > > On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada wrote: > > > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila wrote: > > > > > > I think users can refer to LOGs to see if it has changed since the > > > first time it was configured. I tried by exis

Allow passing extra options to initdb for tests

2024-02-06 Thread Peter Eisentraut
I'm proposing here a way to pass extra options to initdb when run internally during test setup in pg_regress or PostgreSQL::Test::Cluster's init (which covers just about all test suites other than initdb's own tests). For example: make check PG_TEST_INITDB_EXTRA_OPTS='-k -c work_mem=50MB'

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-06 Thread Alvaro Herrera
On 2024-Feb-04, Andrey M. Borodin wrote: > This patch uses wording "banks" in comments before banks start to > exist. But as far as I understand, it is expected to be committed > before "banks" patch. True -- changed that to use ControlLock. > Besides this patch looks good to me. Many thanks fo

Re: RFC: Logging plan of the running query

2024-02-06 Thread Ashutosh Bapat
Hi Atsushi, On Mon, Jan 29, 2024 at 6:32 PM torikoshia wrote: > > Hi, > > Updated the patch to fix typos and move > ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. > > > BTW since the thread is getting long, I list the some points of the > discussion so far: > > # Safety

Re: Printing backtrace of postgres processes

2024-02-06 Thread Michael Paquier
On Fri, Jan 26, 2024 at 11:58:00PM +0530, Bharath Rupireddy wrote: > You probably were looking at v21, the above change isn't there in > versions after that. Can you please review the latest version v26 > attached here? > > We might want this patch extended to the newly added walsummarizer > proce

Re: Memory leak fix in rmtree.c

2024-02-06 Thread Daniel Gustafsson
> On 6 Feb 2024, at 11:21, Ильясов Ян wrote: > > > dirnames isn't allocated at this point, it's palloc'd after this return > > statement on line 67. > > I am sorry, I pointed on the wrong branch. I see that in master > it is really in line 67th , and the allocation goes well. But in > REL_16_STA

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Amit Kapila
On Tue, Feb 6, 2024 at 9:35 AM Peter Smith wrote: > > == > GENERAL > > 1. > Should the "Chapter 30 Logical Replication" at least have another > section that mentions the feature of slot synchronization so the > information about it is easier to find? It doesn't need to say much -- > just give

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Dilip Kumar
On Tue, Feb 6, 2024 at 3:41 PM Amit Kapila wrote: > > On Tue, Feb 6, 2024 at 3:23 PM Dilip Kumar wrote: > > > > On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila > > > wrote: > > > > > > > > On Mon, Feb 5, 2024 at 7:56 PM Masahiko

RE: Memory leak fix in rmtree.c

2024-02-06 Thread Ильясов Ян
> dirnames isn't allocated at this point, it's palloc'd after this return > statement on line 67. > > -- > Daniel Gustafsson I am sorry, I pointed on the wrong branch. I see that in master it is really in line 67th , and the allocation goes well. But in REL_16_STABLE the allocation is in line 58th

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2024-02-06 Thread Yugo NAGATA
On Mon, 5 Feb 2024 14:26:46 +0800 jian he wrote: > On Mon, Feb 5, 2024 at 10:29 AM torikoshia wrote: > > > > Hi, > > > > On 2024-02-03 15:22, jian he wrote: > > > The idea of on_error is to tolerate errors, I think. > > > if a column has a not null constraint, let it cannot be used with > > > (o

Re: doc patch: Spell I/O consistently

2024-02-06 Thread Dagfinn Ilmari Mannsåker
Michael Paquier writes: > On Mon, Feb 05, 2024 at 03:59:27PM +, Dagfinn Ilmari Mannsåker wrote: >> I just noticed that a couple of places in the docs spell I/O as IO or >> even io when not referring to literal table/view/column names or values >> therein. Here's a patch to fix them. > > Make

Re: Question on LWLockMode in dsa.c

2024-02-06 Thread Dilip Kumar
On Tue, Jan 30, 2024 at 6:24 AM Masahiko Sawada wrote: > > Hi, > > While working on radix tree patch[1], John Naylor found that dsa.c > doesn't already use shared locks even in dsa_dump(). dsa_dump() seems > a pure read-only function so I thought we could use a shared lock mode > there. Is there a

Re: POC: Extension for adding distributed tracing - pg_tracing

2024-02-06 Thread Anthonin Bonnefoy
Hi! On Fri, Jan 26, 2024 at 1:43 PM Nikita Malakhov wrote: > It's a good idea to split a big patch into several smaller ones. > But you've already implemented these features - you could provide them > as sequential small patches (i.e. v13-0002-guc-context-propagation.patch and > so on) Keeping

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Amit Kapila
On Tue, Feb 6, 2024 at 3:23 PM Dilip Kumar wrote: > > On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada wrote: > > > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila wrote: > > > > > > On Mon, Feb 5, 2024 at 7:56 PM Masahiko Sawada > > > wrote: > > > > > > > > --- > > > > Since Two processes (e.g. th

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Amit Kapila
On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada wrote: > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila wrote: > > > > I think users can refer to LOGs to see if it has changed since the > > first time it was configured. I tried by existing parameter and see > > the following in LOG: > > LOG: receiv

Re: Synchronizing slots from primary to standby

2024-02-06 Thread Dilip Kumar
On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada wrote: > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila wrote: > > > > On Mon, Feb 5, 2024 at 7:56 PM Masahiko Sawada > > wrote: > > > > > > --- > > > Since Two processes (e.g. the slotsync worker and > > > pg_sync_replication_slots()) concurrently f

Re: Memory leak fix in rmtree.c

2024-02-06 Thread Daniel Gustafsson
> On 6 Feb 2024, at 10:34, Ильясов Ян wrote: > Just like some of my colleagues I've been using Svace* > and I think I've found a bug in src/common/rmtree.c . > > In 64th line function returns false in case it couldn't open a directory, > but the memory, that have been allocated for char** dirnam

Memory leak fix in rmtree.c

2024-02-06 Thread Ильясов Ян
Hello hackers, Just like some of my colleagues I've been using Svace* and I think I've found a bug in src/common/rmtree.c . In 64th line function returns false in case it couldn't open a directory, but the memory, that have been allocated for char** dirnames is not freed. The patch that has a fi

Re: pgsql: Add EXPLAIN (MEMORY) to report planner memory consumption

2024-02-06 Thread Ashutosh Bapat
On Tue, Feb 6, 2024 at 3:51 AM Justin Pryzby wrote: > > Up to now, the explain " " (two space) format is not mixed with "=". > > And, other places which show "Memory" do not use "=". David will > remember prior discussions. > https://www.postgresql.org/message-id/20200402054120.gc14...@telsasoft

Re: Properly pathify the union planner

2024-02-06 Thread Richard Guo
On Fri, Nov 24, 2023 at 6:29 AM David Rowley wrote: > I've attached the updated patch. This one is probably ready for > someone to test out. There will be more work to do, however. I just started reviewing this patch and haven't looked through all the details yet. Here are some feedbacks that

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-06 Thread Alexander Lakhin
06.02.2024 09:48, Amit Kapila wrote: cool, is it possible to see whether this patch changes the runtime of this test in any noticeable way? Yes, unfortunately it does. I've measured duration of 100 tests runs without the patch (with pristine bgwriter and with NO_TEMP_INSTALL): real    6m46,031

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-02-06 Thread Dilip Kumar
On Thu, Jan 11, 2024 at 10:48 AM Bharath Rupireddy wrote: > > Hi, > > Replication slots in postgres will prevent removal of required > resources when there is no connection using them (inactive). This > consumes storage because neither required WAL nor required rows from > the user tables/system c

RE: speed up a logical replica setup

2024-02-06 Thread Hayato Kuroda (Fujitsu)
Dear Shubham, Thanks for testing our codes! > While reviewing the v15 patches I discovered that subscription > connection string has added a lot of options which are not required > now: > v15-0001 > postgres=# select subname, subconninfo from pg_subscription; > subname|

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2024-02-06 Thread jian he
On Tue, Feb 6, 2024 at 3:46 PM Yugo NAGATA wrote: > > On Tue, 06 Feb 2024 09:39:09 +0900 (JST) > Kyotaro Horiguchi wrote: > > > At Mon, 5 Feb 2024 17:22:56 +0900, Yugo NAGATA wrote > > in > > > On Mon, 05 Feb 2024 11:28:59 +0900 > > > torikoshia wrote: > > > > > > > > Based on this, I've made

RE: speed up a logical replica setup

2024-02-06 Thread Hayato Kuroda (Fujitsu)
Dear Euler, > 03. > ``` > /* > * Is the standby server ready for logical replication? > */ > static bool > check_subscriber(LogicalRepInfo *dbinfo) > ``` > > You said "target server must be a standby" in [1], but I cannot find checks > for it. > IIUC, there are two approaches: > a) check the

  1   2   >