Re: New Table Access Methods for Multi and Single Inserts

2024-03-25 Thread Bharath Rupireddy
On Sat, Mar 23, 2024 at 5:47 AM Jeff Davis wrote: > > Comments: Thanks for looking into it. > * Do I understand correctly that CMV, RMV, and CTAS experience a > performance benefit, but COPY FROM does not? And is that because COPY > already used table_multi_insert, whereas CMV and RMV did not?

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Nathan Bossart
On Mon, Mar 25, 2024 at 04:49:12PM +, Bertrand Drouvot wrote: > On Mon, Mar 25, 2024 at 12:25:37PM -0400, Robert Haas wrote: >> In the same vein, I think deactivated_at or inactive_since might be >> good names to consider. I think they get at the same thing as >> released_time, but they avoid i

Re: Popcount optimization using AVX512

2024-03-25 Thread Nathan Bossart
On Mon, Mar 25, 2024 at 06:42:36PM +, Amonson, Paul D wrote: > Ok, CI turned green after my re-post of the patches. Can this please get > merged? Thanks for the new patches. I intend to take another look soon. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Large block sizes support in Linux

2024-03-25 Thread Bruce Momjian
On Mon, Mar 25, 2024 at 02:53:56PM +0100, Pankaj Raghav wrote: > This is an excellent question that needs a bit of community discussion to > expose a device agnostic value that userspace can trust. > > There might be a talk this year at LSFMM about untorn writes[1] in buffered IO > path. I will ma

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 1:30 AM Nathan Bossart wrote: > > On Mon, Mar 25, 2024 at 04:49:12PM +, Bertrand Drouvot wrote: > > On Mon, Mar 25, 2024 at 12:25:37PM -0400, Robert Haas wrote: > >> In the same vein, I think deactivated_at or inactive_since might be > >> good names to consider. I think

Re: Large block sizes support in Linux

2024-03-25 Thread Thomas Munro
On Tue, Mar 26, 2024 at 3:34 AM Pankaj Raghav wrote: > One question: Does ZFS do something like FUA request to force the device > to clear the cache before it can update the node to point to the new page? > > If it doesn't do it, there is no guarantee from device to update the data > atomically un

Re: Why is parula failing?

2024-03-25 Thread David Rowley
On Thu, 21 Mar 2024 at 14:19, Tom Lane wrote: > > David Rowley writes: > > We could also do something like the attached just in case we're > > barking up the wrong tree. > > Yeah, checking indisvalid isn't a bad idea. I'd put another > one further down, just before the DROP of table ab, so we >

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Jelte Fennema-Nio
On Mon, 25 Mar 2024 at 20:16, Bruce Momjian wrote: > I am wondering if the fact that you would be able to do: > > ALTER SYSTEM SET externally_managed_configuration = false > > and then be unable to use ALTER SYSTEM to revert the change is > significant. This is not possible, due to the ex

Re: Add bump memory context type and use it for tuplesorts

2024-03-25 Thread David Rowley
On Tue, 26 Mar 2024 at 03:53, Tom Lane wrote: > I agree with this completely. However, the current design for chunk > headers is mighty restrictive about how many kinds of contexts we can > have. We need to open that back up. Andres mentioned how we could do this in [1]. One possible issue wit

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Bruce Momjian
On Mon, Mar 25, 2024 at 09:40:55PM +0100, Jelte Fennema-Nio wrote: > On Mon, 25 Mar 2024 at 20:16, Bruce Momjian wrote: > > I am wondering if the fact that you would be able to do: > > > > ALTER SYSTEM SET externally_managed_configuration = false > > > > and then be unable to use ALTER SYS

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-25 Thread Melanie Plageman
On Mon, Mar 25, 2024 at 2:29 AM Donghang Lin wrote: > > > > On Sat, Feb 17, 2024 at 2:31 PM Tomas Vondra > > wrote: > > 2) Leader vs. worker counters > > > > It seems to me this does nothing to add the per-worker values from "Heap > > Blocks" into the leader, which means we get stuff like this:

Re: add AVX2 support to simd.h

2024-03-25 Thread Nathan Bossart
Here is what I have staged for commit. One notable difference in this version of the patch is that I've changed + if (nelem <= nelem_per_iteration) + goto one_by_one; to + if (nelem < nelem_per_iteration) + goto one_by_one; I realized that there's no rea

Re: Add bump memory context type and use it for tuplesorts

2024-03-25 Thread Tom Lane
David Rowley writes: > On Tue, 26 Mar 2024 at 03:53, Tom Lane wrote: >> Could we move the knowledge of exactly which context type it is out >> of the per-chunk header and keep it in the block header? > I wasn't 100% clear on your opinion about using 010 vs expanding the > bit-space. Based on the

Re: Teach predtest about IS [NOT] proofs

2024-03-25 Thread Tom Lane
James Coleman writes: > [ v6 patchset ] I went ahead and committed 0001 after one more round of review statements; my bad). I also added the changes in test_predtest.c from 0002. I attach a rebased version of 0002, as well as 0003 which isn't changed, mainly to keep the cfbot happy. I'm still

Re: WIP Incremental JSON Parser

2024-03-25 Thread Jacob Champion
On Wed, Mar 20, 2024 at 11:56 PM Andrew Dunstan wrote: > Thanks, included that and attended to the other issues we discussed. I think > this is pretty close now. Okay, looking over the thread, there are the following open items: - extend the incremental test in order to exercise the semantic cal

Re: WIP Incremental JSON Parser

2024-03-25 Thread Andrew Dunstan
On Mon, Mar 25, 2024 at 6:15 PM Jacob Champion < jacob.champ...@enterprisedb.com> wrote: > On Wed, Mar 20, 2024 at 11:56 PM Andrew Dunstan > wrote: > > Thanks, included that and attended to the other issues we discussed. I > think this is pretty close now. > > Okay, looking over the thread, there

Re: WIP Incremental JSON Parser

2024-03-25 Thread Jacob Champion
On Mon, Mar 25, 2024 at 4:02 PM Andrew Dunstan wrote: > Well, what's the alternative? The current parser doesn't check stack depth in > frontend code. Presumably it too will eventually just run out of memory, > possibly rather sooner as the stack frames could be more expensive than the > incre

Re: session username in default psql prompt?

2024-03-25 Thread Andrew Dunstan
On Mon, Mar 25, 2024 at 9:14 AM Jelte Fennema-Nio wrote: > On Mon, 25 Mar 2024 at 14:06, Robert Haas wrote: > > On Mon, Mar 25, 2024 at 4:30 AM Jelte Fennema-Nio > wrote: > > > That problem seems easy to address by adding a newline into the > > > default prompt. > > > > Ugh. Please, no! > > I g

Re: WIP Incremental JSON Parser

2024-03-25 Thread Jacob Champion
On Mon, Mar 25, 2024 at 4:12 PM Jacob Champion wrote: > Stack size should be pretty limited, at least on the platforms I'm > familiar with. So yeah, the recursive descent will segfault pretty > quickly, but it won't repalloc() an unbounded amount of heap space. > The alternative would just be to g

Re: WIP Incremental JSON Parser

2024-03-25 Thread Andrew Dunstan
On Mon, Mar 25, 2024 at 7:12 PM Jacob Champion < jacob.champ...@enterprisedb.com> wrote: > On Mon, Mar 25, 2024 at 4:02 PM Andrew Dunstan > wrote: > > Well, what's the alternative? The current parser doesn't check stack > depth in frontend code. Presumably it too will eventually just run out of >

Re: WIP Incremental JSON Parser

2024-03-25 Thread Jacob Champion
On Mon, Mar 25, 2024 at 4:24 PM Andrew Dunstan wrote: > OK, so we invent a new error code and have the parser return that if the > stack depth gets too big? Yeah, that seems reasonable. I'd potentially be able to build on that via OAuth for next cycle, too, since that client needs to limit its

Re: SQL:2011 application time

2024-03-25 Thread jian he
On Sun, Mar 24, 2024 at 1:42 AM Paul Jungwirth wrote: > > v33 attached with minor changes. > > Okay, added those tests too. Thanks! > > Rebased to 697f8d266c. > hi. minor issues I found in v33-0003. there are 29 of {check_amproc_signature?.*false} only one {check_amproc_signature(procform->ampro

Re: make dist using git archive

2024-03-25 Thread Andres Freund
Hi, On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote: > Done and committed. This triggered a new warning for me: ../../../../../home/andres/src/postgresql/meson.build:3422: WARNING: Project targets '>=0.54' but uses feature introduced in '0.55.0': Passing executable/found program object to

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-25 Thread Melanie Plageman
Thanks for committing the new WAL format! On Mon, Mar 25, 2024 at 3:33 PM Heikki Linnakangas wrote: > > On 24/03/2024 18:32, Melanie Plageman wrote: > > On Thu, Mar 21, 2024 at 9:28 AM Heikki Linnakangas wrote: > >> > >> In heap_page_prune_and_freeze(), we now do some extra work on each live > >

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Amit Kapila
On Mon, Mar 25, 2024 at 9:55 PM Robert Haas wrote: > > On Mon, Mar 25, 2024 at 12:12 PM Bertrand Drouvot > wrote: > > > Would "released_time" sounds better? (at the end this is exactly what it > > does > > represent unless for the case where it is restored from disk for which the > > meaning >

Re: speed up a logical replica setup

2024-03-25 Thread vignesh C
On Mon, 25 Mar 2024 at 21:36, Hayato Kuroda (Fujitsu) wrote: > > Dear Bharath, Peter, > > > Looks like BF animals aren't happy, please check - > > > https://buildfarm.postgresql.org/cgi-bin/show_failures.pl. > > > > Looks like sanitizer failures. There were a few messages about that > > recently,

Re: Add new error_action COPY ON_ERROR "log"

2024-03-25 Thread Masahiko Sawada
On Mon, Mar 25, 2024 at 8:21 PM Bharath Rupireddy wrote: > > On Mon, Mar 25, 2024 at 10:42 AM Masahiko Sawada > wrote: > > > > The current approach, eliminating the duplicated information in > > CONTEXT, seems good to me. > > Thanks for looking into it. > > > One question about the latest (v8) p

Re: speed up a logical replica setup

2024-03-25 Thread Amit Kapila
On Mon, Mar 25, 2024 at 5:25 PM Peter Eisentraut wrote: > > I have committed your version v33. I did another pass over the > identifier and literal quoting. I added quoting for replication slot > names, for example, even though they can only contain a restricted set > of characters, but it felt

Re: RFC: Logging plan of the running query

2024-03-25 Thread Andres Freund
Hi, On 2024-03-13 15:33:02 -0400, Robert Haas wrote: > But also ... having to wrap the entire plan tree like this seems > pretty awful. I don't really like the idea of a large-scan plan > modification like this in the middle of the query. It's not great. But I also don't really see an alternative

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
On Mon, Mar 25, 2024, at 1:06 PM, Hayato Kuroda (Fujitsu) wrote: > ## Analysis for failure 1 > > The failure caused by a time lag between walreceiver finishes and > pg_is_in_recovery() > returns true. > > According to the output [1], it seems that the tool failed at > wait_for_end_recovery() >

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
On Mon, Mar 25, 2024, at 11:33 PM, Amit Kapila wrote: > On Mon, Mar 25, 2024 at 5:25 PM Peter Eisentraut wrote: > > > > I have committed your version v33. I did another pass over the > > identifier and literal quoting. I added quoting for replication slot > > names, for example, even though they

Re: Add new error_action COPY ON_ERROR "log"

2024-03-25 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 7:16 AM Masahiko Sawada wrote: > > > Please see the attached v9 patch set. > > Thank you for updating the patch! The patch mostly looks good to me. > Here are some minor comments: Thanks for looking into this. > --- > /* non-export function prototypes */ > -static char *

Re: Sync scan & regression tests

2024-03-25 Thread Andres Freund
Hi, On 2024-03-24 11:28:12 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 19/09/2023 01:57, Andres Freund wrote: > >> On 2023-09-18 13:49:24 +0300, Heikki Linnakangas wrote: > >>> d) Copy fewer rows to the table in the test. If we copy only 6 rows, for > >>> example, the table will ha

Re: Teach predtest about IS [NOT] proofs

2024-03-25 Thread Tom Lane
I wrote: > I went ahead and committed 0001 after one more round of review > > statements; my bad). I also added the changes in test_predtest.c from > 0002. I attach a rebased version of 0002, as well as 0003 which isn't > changed, mainly to keep the cfbot happy. [ squint.. ] Apparently I manag

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Andres Freund
Hi, On 2024-03-20 17:41:45 -0700, Andres Freund wrote: > On 2024-03-14 16:56:39 -0400, Tom Lane wrote: > > Also, this is probably not > > helping anything: > > > >'extra_config' => { > > ... > >

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Tom Lane
Andres Freund writes: > I think there must be some actual regression involved. The frequency of > failures on HEAD vs failures on 16 - both of which run the tests concurrently > via meson - is just vastly different. Are you sure it's not just that the total time to run the core regression tests h

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 12:43 PM shveta malik wrote: > > I have one concern, for synced slots on standby, how do we disallow > invalidation due to inactive-timeout immediately after promotion? > > For synced slots, last_inactive_time and inactive_timeout are both > set. Let's say I bring down prim

Re: speed up a logical replica setup

2024-03-25 Thread Amit Kapila
On Tue, Mar 26, 2024 at 8:27 AM Euler Taveira wrote: > > On Mon, Mar 25, 2024, at 11:33 PM, Amit Kapila wrote: > > On Mon, Mar 25, 2024 at 5:25 PM Peter Eisentraut wrote: > > > > I have committed your version v33. I did another pass over the > > identifier and literal quoting. I added quoting f

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread shveta malik
On Mon, Mar 25, 2024 at 9:54 PM Bertrand Drouvot wrote: > > Hi, > > On Mon, Mar 25, 2024 at 07:32:11PM +0530, Amit Kapila wrote: > > On Mon, Mar 25, 2024 at 6:57 PM Robert Haas wrote: > > > And I'm suspicious that having an exception for slots being synced is > > > a bad idea. That makes too much

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread shveta malik
On Tue, Mar 26, 2024 at 1:50 AM Bharath Rupireddy wrote: > > On Tue, Mar 26, 2024 at 1:30 AM Nathan Bossart > wrote: > > > > On Mon, Mar 25, 2024 at 04:49:12PM +, Bertrand Drouvot wrote: > > > On Mon, Mar 25, 2024 at 12:25:37PM -0400, Robert Haas wrote: > > >> In the same vein, I think deact

Re: Add new error_action COPY ON_ERROR "log"

2024-03-25 Thread Masahiko Sawada
On Tue, Mar 26, 2024 at 12:23 PM Bharath Rupireddy wrote: > > On Tue, Mar 26, 2024 at 7:16 AM Masahiko Sawada wrote: > > > > > Please see the attached v9 patch set. > > > > Thank you for updating the patch! The patch mostly looks good to me. > > Here are some minor comments: > > Thanks for lookin

RE: speed up a logical replica setup

2024-03-25 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Euler, > > This only drops the publications created by this tool, not the > pre-existing ones that we discussed in the link provided. Another concern around here is the case which primary subscribes changes from others. After the conversion, new subscriber also tries to connect to an

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Andres Freund
Hi, On 2024-03-26 00:00:38 -0400, Tom Lane wrote: > Andres Freund writes: > > I think there must be some actual regression involved. The frequency of > > failures on HEAD vs failures on 16 - both of which run the tests > > concurrently > > via meson - is just vastly different. > > Are you sure i

Re: Improve eviction algorithm in ReorderBuffer

2024-03-25 Thread Masahiko Sawada
On Thu, Mar 14, 2024 at 12:02 PM Masahiko Sawada wrote: > > > I've attached new version patches. Since the previous patch conflicts with the current HEAD, I've attached the rebased patches. Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com v10-0001-Make-binaryheap-enlar

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Amit Kapila
On Tue, Mar 26, 2024 at 1:24 AM Nathan Bossart wrote: > > > On Sun, Mar 24, 2024 at 03:05:44PM +0530, Bharath Rupireddy wrote: > > This commit particularly lets one specify the inactive_timeout for > > a slot via SQL functions pg_create_physical_replication_slot and > > pg_create_logical_replicati

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Tom Lane
Andres Freund writes: > On 2024-03-26 00:00:38 -0400, Tom Lane wrote: >> Are you sure it's not just that the total time to run the core >> regression tests has grown to a bit more than what the test timeout >> allows for? > You're right, that could be it - in a way at least, the issue is replay n

Re: altering a column's collation leaves an invalid foreign key

2024-03-25 Thread jian he
On Mon, Mar 25, 2024 at 2:47 PM Paul Jungwirth wrote: > > On 3/23/24 10:04, Paul Jungwirth wrote: > > Perhaps if the previous collation was nondeterministic we should force a > > re-check. > > Here is a patch implementing this. It was a bit more fuss than I expected, so > maybe someone has a > b

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 9:30 AM shveta malik wrote: > > On Mon, Mar 25, 2024 at 12:43 PM shveta malik wrote: > > > > I have one concern, for synced slots on standby, how do we disallow > > invalidation due to inactive-timeout immediately after promotion? > > > > For synced slots, last_inactive_ti

Re: pgsql: Track last_inactive_time in pg_replication_slots.

2024-03-25 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 9:38 AM shveta malik wrote: > > On Mon, Mar 25, 2024 at 9:54 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Mon, Mar 25, 2024 at 07:32:11PM +0530, Amit Kapila wrote: > > > On Mon, Mar 25, 2024 at 6:57 PM Robert Haas wrote: > > > > And I'm suspicious that having an ex

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 09:30:32AM +0530, shveta malik wrote: > On Mon, Mar 25, 2024 at 12:43 PM shveta malik wrote: > > > > I have one concern, for synced slots on standby, how do we disallow > > invalidation due to inactive-timeout immediately after promotion? > > > > For synced slots, last

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Amit Kapila
On Sun, Mar 24, 2024 at 3:05 PM Bharath Rupireddy wrote: > > I've attached the v18 patch set here. I've also addressed earlier > review comments from Amit, Ajin Cherian. Note that I've added new > invalidation mechanism tests in a separate TAP test file just because > I don't want to clutter or bl

Re: Add new error_action COPY ON_ERROR "log"

2024-03-25 Thread Bharath Rupireddy
On Tue, Mar 26, 2024 at 9:56 AM Masahiko Sawada wrote: > > > > errmsg("data type incompatibility at line %llu for column %s: \"%s\"", > > > > I guess it would be better to make the log message clearer to convey > > > what we did for the malformed row. For example, how about something > > > like "s

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread Bertrand Drouvot
Hi, On Tue, Mar 26, 2024 at 05:55:11AM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Mar 26, 2024 at 09:30:32AM +0530, shveta malik wrote: > > On Mon, Mar 25, 2024 at 12:43 PM shveta malik > > wrote: > > > > > > I have one concern, for synced slots on standby, how do we disallow > > > invali

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Tue, Mar 26, 2024 at 11:36 AM Bertrand Drouvot wrote: > > > > The issue that I can see with your proposal is: what if one synced the slots > > manually (with pg_sync_replication_slots()) but does not use the sync > > worker? > > Then I think ShutDownSlotSync() is not going to help in that case

Re: Propagate pathkeys from CTEs up to the outer query

2024-03-25 Thread Richard Guo
On Tue, Mar 26, 2024 at 1:39 AM Tom Lane wrote: > I got around to looking at this finally. I was a bit surprised by > your choice of data structure. You made a per-CTE-item cte_paths > list paralleling cte_plan_ids, but what I had had in mind was a > per-subplan list of paths paralleling glob->

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-25 Thread shveta malik
On Tue, Mar 26, 2024 at 11:08 AM Bharath Rupireddy wrote: > > On Tue, Mar 26, 2024 at 9:30 AM shveta malik wrote: > > > > On Mon, Mar 25, 2024 at 12:43 PM shveta malik > > wrote: > > > > > > I have one concern, for synced slots on standby, how do we disallow > > > invalidation due to inactive-t

Re: Functions to return random numbers in a given range

2024-03-25 Thread Dean Rasheed
On Tue, 27 Feb 2024 at 17:33, Dean Rasheed wrote: > > On Sat, 24 Feb 2024 at 17:10, Tomas Vondra > > > > I did a quick review and a little bit of testing on the patch today. I > > think it's a good/useful idea, and I think the code is ready to go (the > > code is certainly much cleaner than anythi

<    1   2