Re: make dist using git archive

2024-01-31 Thread Peter Eisentraut
On 26.01.24 22:18, Eli Schwartz wrote: Hello, meson developer here. Hello, and thanks for popping in! 3. Meson does not support tar.bz2 archives. Simple enough to add, but I'm a bit surprised as usually people seem to want either gzip for portability or xz for efficient compression. We ma

Re: Reducing output size of nodeToString

2024-01-31 Thread Peter Eisentraut
On 30.01.24 12:26, Matthias van de Meent wrote: Most of the other defaults I'm doubtful about. First, we are colliding here between the goals of minimizing the storage size and making the debug output more readable. I've never really wanted to make the output "more readable". The current one is

Re: Improve eviction algorithm in ReorderBuffer

2024-01-31 Thread vignesh C
On Tue, 30 Jan 2024 at 13:37, Masahiko Sawada wrote: > > On Fri, Jan 26, 2024 at 5:36 PM Masahiko Sawada wrote: > > > > On Wed, Dec 20, 2023 at 12:11 PM Amit Kapila > > wrote: > > > > > > On Wed, Dec 20, 2023 at 6:49 AM Masahiko Sawada > > > wrote: > > > > > > > > On Tue, Dec 19, 2023 at 8:02

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Masahiko Sawada
On Tue, Jan 30, 2024 at 9:53 PM shveta malik wrote: > > On Tue, Jan 30, 2024 at 4:06 PM shveta malik wrote: > > > > PFA v73-0001 which addresses the above comments. Other patches will be > > rebased and posted after pushing this one. > > Since v73-0001 is pushed, PFA rest of the patches. Changes

Re: 003_extrafiles.pl test fails on Windows with the newer Perl versions

2024-01-31 Thread Nazir Bilal Yavuz
Hi, On Wed, 31 Jan 2024 at 01:18, Andrew Dunstan wrote: > > Pushed to all live branches. Thanks for the patch. Thanks for the push! -- Regards, Nazir Bilal Yavuz Microsoft

Re: src/bin/pg_upgrade/t/004_subscription.pl test comment fix

2024-01-31 Thread Alvaro Herrera
How about rewording it more extensively? It doesn't read great IMO. I would use something like # In the upgraded instance, the running status and failover option of the # subscription with the failover option should have been preserved; the other # should not. # So regress_sub1 should still have

Re: Extending SMgrRelation lifetimes

2024-01-31 Thread Thomas Munro
On Wed, Nov 29, 2023 at 1:42 PM Heikki Linnakangas wrote: > I spent some more time digging into this, experimenting with different > approaches. Came up with pretty significant changes; see below: Hi Heikki, I think this approach is good. As I wrote in the first email, I had briefly considered

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2024-01-31 Thread Bharath Rupireddy
On Tue, Jan 30, 2024 at 11:01 PM Alvaro Herrera wrote: > > Hmm, this looks quite nice and simple. Thanks for looking at it. > My only comment is that a > sequence like this > >/* Read from WAL buffers, if available. */ >rbytes = XLogReadFromBuffers(&output_message.data[output_message.len

Re: Transaction timeout

2024-01-31 Thread Japin Li
On Tue, 30 Jan 2024 at 14:22, Andrey M. Borodin wrote: >> On 26 Jan 2024, at 19:58, Japin Li wrote: >> >> Thanks for updating the patch. Here are some comments for v24. >> >> + >> +Terminate any session that spans longer than the specified amount of >> +time in transacti

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2024-01-31 Thread Alvaro Herrera
Looking at 0003, where an XXX comment is added about taking a spinlock to read LogwrtResult, I suspect the answer is probably not, because it is likely to slow down the other uses of LogwrtResult. But I wonder if a better path forward would be to base further work on my older uncommitted patch to

Re: CI and test improvements

2024-01-31 Thread vignesh C
On Thu, 18 Jan 2024 at 06:46, Michael Paquier wrote: > > On Wed, Jan 17, 2024 at 05:34:00PM +0530, vignesh C wrote: > > Are we planning to do anything more in this thread, the thread has > > been idle for more than 7 months. If nothing more is planned for this, > > I'm planning to close this commi

Re: Emitting JSON to file using COPY TO

2024-01-31 Thread vignesh C
On Sat, 27 Jan 2024 at 11:25, Junwang Zhao wrote: > > Hi hackers, > > Kou-san(CCed) has been working on *Make COPY format extendable[1]*, so > I think making *copy to json* based on that work might be the right direction. > > I write an extension for that purpose, and here is the patch set togethe

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-01-31 Thread Antonin Houska
Alvaro Herrera wrote: > On 2024-Jan-30, Pavel Stehule wrote: > > > One of my customer today is reducing one table from 140GB to 20GB. Now he > > is able to run archiving. He should play with pg_repack, and it is working > > well today, but I ask myself, what pg_repack does not be hard to do > >

Re: Transaction timeout

2024-01-31 Thread Andrey Borodin
> On 31 Jan 2024, at 14:27, Japin Li wrote: > > LGTM. > > If there is no other objections, I'll change it to ready for committer > next Monday. I think we have a quorum, so I decided to go ahead and flipped status to RfC. Thanks! Best regards, Andrey Borodin.

Re: Emitting JSON to file using COPY TO

2024-01-31 Thread Junwang Zhao
Hi Vignesh, On Wed, Jan 31, 2024 at 5:50 PM vignesh C wrote: > > On Sat, 27 Jan 2024 at 11:25, Junwang Zhao wrote: > > > > Hi hackers, > > > > Kou-san(CCed) has been working on *Make COPY format extendable[1]*, so > > I think making *copy to json* based on that work might be the right > > direc

libpq fails to build with TSAN

2024-01-31 Thread Roman Lozko
Hi, so libpq has this line in its Makefile https://github.com/postgres/postgres/blob/6ee26c6a4bafabbd22a85f575d2446fd5ec6ad0d/src/interfaces/libpq/Makefile#L116 which checks that libpq does not use any "exit" functions. With ThreadSanitizer it triggers on function `__tsan_func_exit` which is used t

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-01-31 Thread Alvaro Herrera
This is great to hear. On 2024-Jan-31, Antonin Houska wrote: > Is your plan to work on it soon or should I try to write a draft patch? (I > assume this is for PG >= 18.) I don't have plans for it, so if you have resources, please go for it. -- Álvaro Herrera PostgreSQL Developer — ht

Re: POC, WIP: OR-clause support for indexes

2024-01-31 Thread jian he
On Wed, Jan 31, 2024 at 10:55 AM jian he wrote: > > based on my understanding of > https://www.postgresql.org/docs/current/xoper-optimization.html#XOPER-COMMUTATOR > I think you need move commutator check right after the `if > (get_op_rettype(opno) != BOOLOID)` branch > I was wrong about this part

Re: Possibility to disable `ALTER SYSTEM`

2024-01-31 Thread Gabriele Bartolini
Hi there, I very much like the idea of a file in the data directory that also controls the copy operations. Just wanted to highlight though that in our operator we have already applied the read-only postgresql.auto.conf trick to disable the system (see https://cloudnative-pg.io/documentation/curr

Re: Extending SMgrRelation lifetimes

2024-01-31 Thread Heikki Linnakangas
On 31/01/2024 10:54, Thomas Munro wrote: On Wed, Nov 29, 2023 at 1:42 PM Heikki Linnakangas wrote: I spent some more time digging into this, experimenting with different approaches. Came up with pretty significant changes; see below: Hi Heikki, I think this approach is good. As I wrote in t

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Amit Kapila
On Wed, Jan 31, 2024 at 2:02 PM Masahiko Sawada wrote: > > Thank you for updating the patches. As for the slotsync worker patch, > is there any reason why 0001, 0002, and 0004 patches are still > separated? > No specific reason, it could be easier to review those parts. > > Beside, here are some

Re: Improve the connection failure error messages

2024-01-31 Thread Nisha Moond
> AFAIK some recent commits patches (e,g [1] for the "slot sync" > development) have created some more cases of "could not connect..." > messages. So, you might need to enhance your patch to deal with any > new ones in the latest HEAD. > > == > [1] > https://github.com/postgres/postgres/commit

Re: CI and test improvements

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-31, vignesh C wrote: > Are we planning to do anything more on this? I was not sure if we > should move this to next commitfest or return it. Well, the patches don't apply anymore since .cirrus.tasks.yml has been created. However, I'm sure we still want [some of] the improvements to t

Re: POC, WIP: OR-clause support for indexes

2024-01-31 Thread Alena Rybakina
Hi, thank you for your review and interest in this subject. On 31.01.2024 13:15, jian he wrote: On Wed, Jan 31, 2024 at 10:55 AM jian he wrote: based on my understanding of https://www.postgresql.org/docs/current/xoper-optimization.html#XOPER-COMMUTATOR I think you need move commutator check r

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2024-01-31 Thread Bharath Rupireddy
On Wed, Jan 31, 2024 at 3:01 PM Alvaro Herrera wrote: > > Looking at 0003, where an XXX comment is added about taking a spinlock > to read LogwrtResult, I suspect the answer is probably not, because it > is likely to slow down the other uses of LogwrtResult. We avoided keeping LogwrtResult latest

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alexander Kuzmenkov
I'd be happy to see this backpatched. What kind of regressions are we worried about? I'd say partition-wise sort + merge should be faster than append + sort for reasonably sized tables. That's basically what tuplesort does inside. Moreso, this can enable index scans on partitions, which is an even

Re: libpq fails to build with TSAN

2024-01-31 Thread Daniel Gustafsson
> On 31 Jan 2024, at 04:21, Roman Lozko wrote: > > Hi, so libpq has this line in its Makefile > https://github.com/postgres/postgres/blob/6ee26c6a4bafabbd22a85f575d2446fd5ec6ad0d/src/interfaces/libpq/Makefile#L116 > which checks that libpq does not use any "exit" functions. With > ThreadSanitizer

Re: Switching XLog source from archive to streaming when primary available

2024-01-31 Thread Bharath Rupireddy
On Wed, Jan 3, 2024 at 4:58 PM Bharath Rupireddy wrote: > > On Thu, Dec 28, 2023 at 5:26 PM Bharath Rupireddy > wrote: > > > > I took a closer look at v14 and came up with the following changes: > > > > 1. Used advance_wal introduced by commit c161ab74f7. > > 2. Simplified the core logic and new

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Thu, Jan 18, 2024 at 6:19 AM Peter Eisentraut wrote: > > Very early in this thread, someone mentioned the name > pg_create_subscriber, and of course there is pglogical_create_subscriber > as the historical predecessor. Something along those lines seems better > to me. Maybe there are other id

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-01-31 Thread Bharath Rupireddy
On Sat, Jan 27, 2024 at 1:18 AM Bharath Rupireddy wrote: > > On Thu, Jan 11, 2024 at 10:48 AM Bharath Rupireddy > wrote: > > > > Hi, > > > > Replication slots in postgres will prevent removal of required > > resources when there is no connection using them (inactive). This > > consumes storage be

Re: Emitting JSON to file using COPY TO

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-23, jian he wrote: > > + | FORMAT_LA copy_generic_opt_arg > > + { > > + $$ = makeDefElem("format", $2, @1); > > + } > > ; > > > > I think it's not necessary. "format" option is already handled in > > copy_generic_opt_elem.

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alexander Kuzmenkov
On Wed, Jan 31, 2024 at 1:33 PM Alexander Kuzmenkov wrote: > I'd be happy to see this backpatched. What kind of regressions are we > worried about? I'd say partition-wise sort + merge should be faster > than append + sort for reasonably sized tables. That's basically what > tuplesort does inside.

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-31, Alexander Kuzmenkov wrote: > To put it another way, this change enables our usual cost model for > MergeAppend to work correctly in the presence of filters. I think we > can consider this model to be reasonably correct, and we don't > currently have major problems with MergeAppend

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread James Coleman
On Tue, Jan 30, 2024 at 10:34 PM Tom Lane wrote: > > James Coleman writes: > > I've finally had a chance to look at this, and I don't believe there's > > any real failure here, merely drift of how the planner works on master > > resulting in this query now being eligible for a different plan shap

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Daniel Westermann (DWE)
Hi, >Since we have a minor coming up very soon, I think it's not a good idea >to backpatch right now. Maybe you can push to master now, and consider >whether to backpatch later. >The problem is -- if somebody has an application that gets good plans >with the current cost model, and you change th

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Wed, Jan 31, 2024 at 9:52 AM Hayato Kuroda (Fujitsu) < kuroda.hay...@fujitsu.com> wrote: > > Dear Euler, > > I extracted some review comments which may require many efforts. I hope this makes them > easy to review. > > 0001: not changed from yours. > 0002: avoid to use replication connections. S

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Wed, Jan 31, 2024, at 11:25 AM, Fabrízio de Royes Mello wrote: > Jumping into this a bit late here... I'm trying a simple pg_createsubscriber > but getting an error: Try v11. It seems v12-0002 is not correct. > Seems we need to escape connection params similar we do in dblink [1] I think it

Re: Possibility to disable `ALTER SYSTEM`

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 12:28 AM Tom Lane wrote: > You cannot enforce such restrictions within Postgres. > It has to be done by an outside mechanism. If you think > different, you are mistaken. It seems like the biggest reason why we can't enforce such restrictions with Postgres is that you won'

Re: remaining sql/json patches

2024-01-31 Thread jian he
Hi. minor issues. I am wondering do we need add `pg_node_attr(query_jumble_ignore)` to some of our created structs in src/include/nodes/parsenodes.h in v39-0001-Add-SQL-JSON-query-functions.patch diff --git a/src/backend/parser/parse_jsontable.c b/src/backend/parser/parse_jsontable.c new file mode

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Wed, Jan 31, 2024 at 11:35 AM Euler Taveira wrote: > > On Wed, Jan 31, 2024, at 11:25 AM, Fabrízio de Royes Mello wrote: > > Jumping into this a bit late here... I'm trying a simple pg_createsubscriber but getting an error: > > > Try v11. It seems v12-0002 is not correct. Using v11 I'm getting

Re: [PATCH] Add native windows on arm64 support

2024-01-31 Thread Andrew Dunstan
On 2024-01-30 Tu 17:54, Dave Cramer wrote: On Tue, Jan 30, 2024 at 4:56 PM Andrew Dunstan wrote: On 2024-01-30 Tu 09:50, Dave Cramer wrote: On Tue, 30 Jan 2024 at 08:38, Andrew Dunstan wrote: On 2024-01-29 Mo 11:20, Dave Cramer wrote: Dave Cramer

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-01-31 Thread Tom Lane
Richard Guo writes: > On Wed, Jan 31, 2024 at 5:12 AM Tom Lane wrote: >> * Why is it okay to just use pull_varnos on a tablesample expression, >> when contain_references_to() does something different? > Hmm, the main reason is that the expression_tree_walker() function does > not handle T_Restri

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Tom Lane
Alvaro Herrera writes: > Since we have a minor coming up very soon, I think it's not a good idea > to backpatch right now. Maybe you can push to master now, and consider > whether to backpatch later. As a rule, we don't back-patch changes like this ever. People don't appreciate plans changing i

Re: cleanup patches for incremental backup

2024-01-31 Thread Robert Haas
On Tue, Jan 30, 2024 at 11:52 AM Robert Haas wrote: > Here's a patch for that. I now think > a7097ca630a25dd2248229f21ebce4968d85d10a was actually misguided, and > served only to mask some of the failures caused by waiting for the WAL > summary file. Committed. -- Robert Haas EDB: http://www.en

Re: [PATCH] Add native windows on arm64 support

2024-01-31 Thread Peter Eisentraut
On 31.01.24 16:20, Andrew Dunstan wrote: - PostgreSQL will only build for the x64 architecture on 64-bit Windows. + PostgreSQL will only build for the x64 and ARM64 architecture on 64-bit Windows. Are there any other 64-bit architectures for Windows? Possibly, the original sentence was meant

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Wed, Jan 31, 2024, at 11:55 AM, Fabrízio de Royes Mello wrote: > > On Wed, Jan 31, 2024 at 11:35 AM Euler Taveira wrote: > > > > On Wed, Jan 31, 2024, at 11:25 AM, Fabrízio de Royes Mello wrote: > > > > Jumping into this a bit late here... I'm trying a simple > > pg_createsubscriber but getti

Re: libpq fails to build with TSAN

2024-01-31 Thread Tom Lane
Daniel Gustafsson writes: > I think it should, the idea of that check is to catch calls to actual exits, > while this is instrumentation which has nothing to do with exit(2). The > attached diff should be enough to handle this. +1 regards, tom lane

Re: [PATCH] Add native windows on arm64 support

2024-01-31 Thread Andrew Dunstan
On 2024-01-31 We 10:34, Peter Eisentraut wrote: On 31.01.24 16:20, Andrew Dunstan wrote: - PostgreSQL will only build for the x64 architecture on 64-bit Windows. + PostgreSQL will only build for the x64 and ARM64 architecture on 64-bit Windows. Are there any other 64-bit architectures for W

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Masahiko Sawada
On Wed, Jan 31, 2024 at 7:42 PM Amit Kapila wrote: > > On Wed, Jan 31, 2024 at 2:02 PM Masahiko Sawada wrote: > > > > Thank you for updating the patches. As for the slotsync worker patch, > > is there any reason why 0001, 0002, and 0004 patches are still > > separated? > > > > No specific reason,

Re: make dist using git archive

2024-01-31 Thread Eli Schwartz
On 1/31/24 3:03 AM, Peter Eisentraut wrote: >> What do you use this for? IMO a more robust way to track the commit used >> is to use gitattributes export-subst to write a `.git_archival.txt` file >> containing the commit sha1 and other info -- this can be read even after >> the file is extracted, w

Re: Possibility to disable `ALTER SYSTEM`

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 5:16 AM Gabriele Bartolini wrote: > I very much like the idea of a file in the data directory that also controls > the copy operations. > > Just wanted to highlight though that in our operator we have already applied > the read-only postgresql.auto.conf trick to disable t

Re: Reducing output size of nodeToString

2024-01-31 Thread Matthias van de Meent
On Wed, 31 Jan 2024, 09:16 Peter Eisentraut, wrote: > On 30.01.24 12:26, Matthias van de Meent wrote: > >> Most of the other defaults I'm doubtful about. First, we are colliding > >> here between the goals of minimizing the storage size and making the > >> debug output more readable. > > I've ne

Re: pgsql: Clean pg_walsummary's tmp_check directory.

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 11:51 AM Tom Lane wrote: > Clean pg_walsummary's tmp_check directory. Ugh. The reason I keep doing this is because I've switched to using meson builds, where of course you don't get complaints about this. And it seems like CI doesn't tell you either. Nor does the buildfar

Abort if dup fail (src/bin/pg_dump/compress_none.c)

2024-01-31 Thread Ranier Vilela
Hi. Per Coverity. CID 1506240: (#1 of 1): Argument cannot be negative (NEGATIVE_RETURNS) 3. negative_returns: dup(fd) is passed to a parameter that cannot be negative. pg_dump function open_none, tries to associate a stream to a file descriptor, using function dup, which may fail and return negat

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Robert Haas
On Tue, Jan 30, 2024 at 6:39 PM Jelte Fennema-Nio wrote: > I agree that it's hard to prove that such heuristics will always be > better in practice than the status quo. But I feel like we shouldn't > let perfect be the enemy of good here. Sure, I agree. > I one approach that is a clear > improve

Re: speed up a logical replica setup

2024-01-31 Thread Fabrízio de Royes Mello
On Wed, Jan 31, 2024 at 12:38 PM Euler Taveira wrote: > > > Hmm. I didn't try it with the failover patch that was recently applied. Did you > have any special configuration on primary? > Nothing special, here the configurations I've changed after bootstrap: port = '5432' wal_level = 'logical' ma

Re: Reducing output size of nodeToString

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 11:17 AM Matthias van de Meent wrote: > I was also thinking about smaller per-attribute expression storage, for index > attribute expressions, table default expressions, and functions. Other than > that, less memory overhead for the serialized form of these constructs als

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Jelte Fennema-Nio
On Wed, 31 Jan 2024 at 18:23, Robert Haas wrote: > That's kind of an odd artifact, but maybe it's fine in > practice. I agree it's an odd artifact, but it's not a regression over the status quo. Achieving that was the intent of my suggestion: A change that improves some cases, but regresses nowhe

Re: psql not responding to SIGINT upon db reconnection

2024-01-31 Thread Tristan Partin
On Tue Jan 30, 2024 at 4:42 PM CST, Jelte Fennema-Nio wrote: On Tue, 30 Jan 2024 at 23:20, Tristan Partin wrote: > Not next week, but here is a respin. I've exposed pqSocketPoll as > PQsocketPoll and am just using that. You can see the diff is so much > smaller, which is great! The exports.txt

Re: make dist using git archive

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 10:50 AM Eli Schwartz wrote: > Ideally git commits should be signed, but that requires large numbers of > people to have security-minded git commit habits. From a quick check of > the postgres commit logs, only one person seems to be regularly signing > commits, which does

Re: Fix some ubsan/asan related issues

2024-01-31 Thread Tristan Partin
On Tue Jan 30, 2024 at 10:00 PM CST, Alexander Lakhin wrote: Hello, 30.01.2024 18:57, Tristan Partin wrote: > Patch 1: > > Passing NULL as a second argument to memcpy breaks ubsan, ... Maybe you would like to fix also one more similar place, reached with: create extension xml2; select xslt_proc

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-31 Thread Tomas Vondra
On 1/29/24 16:42, Ronan Dunklau wrote: > Le lundi 29 janvier 2024, 15:59:04 CET Tomas Vondra a écrit : >> I'm not sure work_mem is a good parameter to drive this. It doesn't say >> how much memory we expect the backend to use - it's a per-operation >> limit, so it doesn't work particularly well w

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 12:49 PM Jelte Fennema-Nio wrote: > Testing a bunch of scenarios to find a good one sounds like a good > idea, which can probably give us a more optimal heuristic. But it also > sounds like a lot of work, and probably results in a lot of > discussion. That extra effort migh

Re: psql not responding to SIGINT upon db reconnection

2024-01-31 Thread Jelte Fennema-Nio
On Wed, 31 Jan 2024 at 19:07, Tristan Partin wrote: > I was looking for documentation of PQsocket(), but didn't find any > standalone (unless I completely missed it). So I just copied how > PQsocket() is documented in PQconnectPoll(). I am happy to document it > separately if you think it would be

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Melih Mutlu
Robert Haas , 31 Oca 2024 Çar, 20:23 tarihinde şunu yazdı: > On Tue, Jan 30, 2024 at 6:39 PM Jelte Fennema-Nio > wrote: > > I agree that it's hard to prove that such heuristics will always be > > better in practice than the status quo. But I feel like we shouldn't > > let perfect be the enemy of

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 2:23 PM Melih Mutlu wrote: >> That seems like it might be a useful refinement of Melih Mutlu's >> original proposal, but consider a message stream that consists of >> messages exactly 8kB in size. If that message stream begins when the >> buffer is empty, all messages are s

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread Robert Haas
On Tue, Jan 30, 2024 at 9:56 PM James Coleman wrote: > I don't follow the "Idle since July" since it just hasn't received > review since then, so there's been nothing to reply to. It wasn't clear to me if you thought that the patch was ready for review since July, or if it was waiting on you sinc

Re: Incorrect cost for MergeAppend

2024-01-31 Thread David Rowley
On Thu, 1 Feb 2024 at 04:32, Tom Lane wrote: > > Alvaro Herrera writes: > > Since we have a minor coming up very soon, I think it's not a good idea > > to backpatch right now. Maybe you can push to master now, and consider > > whether to backpatch later. > > As a rule, we don't back-patch change

Re: Incorrect cost for MergeAppend

2024-01-31 Thread Alexander Kuzmenkov
On Wed, Jan 31, 2024 at 9:49 PM David Rowley wrote: > Pushed to master. > > Thanks for the report and the fix, Alexander. Thank you!

Re: Add system identifier to backup manifest

2024-01-31 Thread Robert Haas
On Thu, Jan 25, 2024 at 2:52 AM Amul Sul wrote: > Thank you for the review-comments, updated version attached. I generally agree with 0001. I spent a long time thinking about your decision to make verifier_context contain a pointer to manifest_data instead of, as it does currently, a pointer to m

Re: Improve the connection failure error messages

2024-01-31 Thread Peter Smith
On Wed, Jan 31, 2024 at 9:58 PM Nisha Moond wrote: > > >> AFAIK some recent commits patches (e,g [1] for the "slot sync" >> development) have created some more cases of "could not connect..." >> messages. So, you might need to enhance your patch to deal with any >> new ones in the latest HEAD. >>

Re: libpq fails to build with TSAN

2024-01-31 Thread Daniel Gustafsson
> On 31 Jan 2024, at 16:39, Tom Lane wrote: > > Daniel Gustafsson writes: >> I think it should, the idea of that check is to catch calls to actual exits, >> while this is instrumentation which has nothing to do with exit(2). The >> attached diff should be enough to handle this. > > +1 Pushed.

Re: Documentation: warn about two_phase when altering a subscription

2024-01-31 Thread Peter Smith
On Wed, Jan 31, 2024 at 4:55 PM Bertrand Drouvot wrote: ... > As a non native English speaker somehow I have to rely on you for those > suggestions ;-) > > They make sense to me so applied both in v2 attached. > Patch v2 looks OK to me, but probably there is still room for improvement. Let's see

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-29, Dilip Kumar wrote: > Thank you for working on this. There is one thing that I feel is > problematic. We have kept the allowed values for these GUCs to be in > multiple of SLRU_BANK_SIZE i.e. 16 and that's the reason the min > values were changed to 16 but in this refactoring patc

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Tue, Jan 30, 2024, at 6:26 AM, Hayato Kuroda (Fujitsu) wrote: > > One open item that is worrying me is how to handle the pg_ctl timeout. This > > patch does nothing and the user should use PGCTLTIMEOUT environment > > variable to > > avoid that the execution is canceled after 60 seconds (defaul

Re: src/bin/pg_upgrade/t/004_subscription.pl test comment fix

2024-01-31 Thread Peter Smith
On Wed, Jan 31, 2024 at 7:48 PM Alvaro Herrera wrote: > > How about rewording it more extensively? It doesn't read great IMO. > I would use something like > > # In the upgraded instance, the running status and failover option of the > # subscription with the failover option should have been prese

Re: Rename setup_cancel_handler in pg_dump

2024-01-31 Thread Yugo NAGATA
On Tue, 30 Jan 2024 13:44:28 +0100 Daniel Gustafsson wrote: > > On 26 Jan 2024, at 01:42, Yugo NAGATA wrote: > > > I am proposing it because there is a public function with > > the same name in fe_utils/cancel.c. I know pg_dump/parallel.c > > does not include fe_utils/cancel.h, so there is no c

RE: speed up a logical replica setup

2024-01-31 Thread Hayato Kuroda (Fujitsu)
Dear Fabrízio, Thanks for reporting. I understood that the issue occurred on v11 and v12. I will try to reproduce and check the reason. Best Regards, Hayato Kuroda FUJITSU LIMITED https://www.fujitsu.com/global/

Re: More new SQL/JSON item methods

2024-01-31 Thread Kyotaro Horiguchi
Thank you for the fix! At Tue, 30 Jan 2024 13:46:17 +0530, Jeevan Chalke wrote in > On Mon, Jan 29, 2024 at 11:03 AM Tom Lane wrote: > > > Kyotaro Horiguchi writes: > > > Having said that, I'm a bit concerned about the case where an overly > > > long string is given there. However, consideri

Re: More new SQL/JSON item methods

2024-01-31 Thread Kyotaro Horiguchi
At Thu, 01 Feb 2024 10:49:57 +0900 (JST), Kyotaro Horiguchi wrote in > By the way, while playing with this feature, I noticed the following > error message: > > > select jsonb_path_query('1.1' , '$.boolean()'); > > ERROR: numeric argument of jsonpath item method .boolean() is out of range > >

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Michael Paquier
On Wed, Jan 31, 2024 at 02:39:54PM +0900, Michael Paquier wrote: > Thanks, I'm looking into that now. I have much to say about the patch, but for now I have begun running some performance tests using the patches, because this thread won't get far until we are sure that the callbacks do not impact

set_cheapest without checking pathlist

2024-01-31 Thread James Coleman
Hello, Robert: I've taken the liberty of cc'ing you since you worked on most of this code. My apologies if that wasn't appropriate. While working on "Parallelize correlated subqueries that execute within each worker" [1] I noticed that while in the other call to set_cheapest (for partially_groupe

RE: speed up a logical replica setup

2024-01-31 Thread Hayato Kuroda (Fujitsu)
Dear Euler, Thanks for giving comments! I want to reply some of them. > I didn't provide the whole explanation. I'm envisioning the use case that pg_ctl doesn't reach the consistent state and the timeout is reached (the consequence is that pg_createsubscriber aborts the execution). It might occur

Re: Parallelize correlated subqueries that execute within each worker

2024-01-31 Thread James Coleman
On Wed, Jan 31, 2024 at 3:18 PM Robert Haas wrote: > > On Tue, Jan 30, 2024 at 9:56 PM James Coleman wrote: > > I don't follow the "Idle since July" since it just hasn't received > > review since then, so there's been nothing to reply to. > > It wasn't clear to me if you thought that the patch wa

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Euler Taveira
On Mon, Jan 29, 2024, at 10:17 AM, Zhijie Hou (Fujitsu) wrote: > Attach the V72-0001 which addressed above comments, other patches will be > rebased and posted after pushing first patch. Thanks Shveta for helping > address > the comments. While working on another patch I noticed a new NOTICE mess

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-01-31 Thread Richard Guo
On Wed, Jan 31, 2024 at 11:21 PM Tom Lane wrote: > Richard Guo writes: > > On Wed, Jan 31, 2024 at 5:12 AM Tom Lane wrote: > >> * Why is it okay to just use pull_varnos on a tablesample expression, > >> when contain_references_to() does something different? > > > Hmm, the main reason is that th

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Wed, Jan 31, 2024, at 11:09 PM, Hayato Kuroda (Fujitsu) wrote: > > > Why? Are you suggesting that the dry run mode covers just the verification > part? If so, it is not a dry run mode. I would expect it to run until the end > (or until it accomplish its goal) but *does not* modify data. For pg_r

Re: Flushing large data immediately in pqcomm

2024-01-31 Thread Andres Freund
Hi, On 2024-01-31 14:57:35 -0500, Robert Haas wrote: > > You're right and I'm open to doing more legwork. I'd also appreciate any > > suggestion about how to test this properly and/or useful scenarios to > > test. That would be really helpful. > > I think experimenting to see whether the long-shor

RE: speed up a logical replica setup

2024-01-31 Thread Hayato Kuroda (Fujitsu)
Dear Fabrízio, Euler, I think you set the primary_slot_name to the standby server, right? While reading codes, I found below line in v11-0001. ``` if (primary_slot_name != NULL) { conn = connect_database(dbinfo[0].pubconninfo); if (conn != NULL)

Re: set_cheapest without checking pathlist

2024-01-31 Thread Richard Guo
On Thu, Feb 1, 2024 at 10:04 AM James Coleman wrote: > I don't see any inherent reason why we must always assume that > gather_grouping_paths will always result in having at least one entry > in pathlist. If, for example, we've disabled incremental sort and the > cheapest partial path happens to

Re: set_cheapest without checking pathlist

2024-01-31 Thread David Rowley
On Thu, 1 Feb 2024 at 16:29, Richard Guo wrote: > > > On Thu, Feb 1, 2024 at 10:04 AM James Coleman wrote: >> >> I don't see any inherent reason why we must always assume that >> gather_grouping_paths will always result in having at least one entry >> in pathlist. If, for example, we've disabled

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Junwang Zhao
Hi Michael, On Thu, Feb 1, 2024 at 9:58 AM Michael Paquier wrote: > > On Wed, Jan 31, 2024 at 02:39:54PM +0900, Michael Paquier wrote: > > Thanks, I'm looking into that now. > > I have much to say about the patch, but for now I have begun running > some performance tests using the patches, becaus

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Michael Paquier
On Thu, Feb 01, 2024 at 10:57:58AM +0900, Michael Paquier wrote: > And here are the results I get for text and binary (ms, average of 15 > queries after discarding the three highest and three lowest values): > test | master | v7 | v10 > -++--+-- > fr

Re: More new SQL/JSON item methods

2024-01-31 Thread Jeevan Chalke
On Thu, Feb 1, 2024 at 7:20 AM Kyotaro Horiguchi wrote: > Thank you for the fix! > > At Tue, 30 Jan 2024 13:46:17 +0530, Jeevan Chalke < > jeevan.cha...@enterprisedb.com> wrote in > > On Mon, Jan 29, 2024 at 11:03 AM Tom Lane wrote: > > > > > Kyotaro Horiguchi writes: > > > > Having said that,

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Amit Kapila
On Wed, Jan 31, 2024 at 9:20 PM Masahiko Sawada wrote: > > On Wed, Jan 31, 2024 at 7:42 PM Amit Kapila wrote: > > > > > > Considering my previous where we don't want to restart for a required > > parameter change, isn't it better to avoid repeated restart (say when > > the user gave an invalid db

Re: More new SQL/JSON item methods

2024-01-31 Thread Jeevan Chalke
On Thu, Feb 1, 2024 at 7:24 AM Kyotaro Horiguchi wrote: > At Thu, 01 Feb 2024 10:49:57 +0900 (JST), Kyotaro Horiguchi < > horikyota@gmail.com> wrote in > > By the way, while playing with this feature, I noticed the following > > error message: > > > > > select jsonb_path_query('1.1' , '$.bool

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Michael Paquier
On Thu, Feb 01, 2024 at 11:43:07AM +0800, Junwang Zhao wrote: > The first 6 rounds are like 10% better than the later 9 rounds, is this > normal? Even with HEAD? Perhaps you have some OS cache eviction in play here? FWIW, I'm not seeing any of that with longer runs after 7~ tries in a loop of 15

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-31 Thread Junwang Zhao
On Thu, Feb 1, 2024 at 11:56 AM Michael Paquier wrote: > > On Thu, Feb 01, 2024 at 11:43:07AM +0800, Junwang Zhao wrote: > > The first 6 rounds are like 10% better than the later 9 rounds, is this > > normal? > > Even with HEAD? Perhaps you have some OS cache eviction in play here? > FWIW, I'm n

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Amit Kapila
On Thu, Feb 1, 2024 at 8:15 AM Euler Taveira wrote: > > On Mon, Jan 29, 2024, at 10:17 AM, Zhijie Hou (Fujitsu) wrote: > > Attach the V72-0001 which addressed above comments, other patches will be > rebased and posted after pushing first patch. Thanks Shveta for helping > address > the comments.

Re: pg_column_toast_chunk_id: a function to get a chunk ID of a TOASTed value

2024-01-31 Thread Yugo NAGATA
On Tue, 30 Jan 2024 14:57:20 +0800 jian he wrote: > On Tue, Jan 30, 2024 at 1:56 PM Yugo NAGATA wrote: > > > > I attached the correct one, v4. > > > > +-- Test pg_column_toast_chunk_id: > +-- Check if the returned chunk_id exists in the TOAST table > +CREATE TABLE test_chunk_id (v1 text, v2 tex

  1   2   >