Re: Useless field ispartitioned in CreateStmtContext

2024-10-30 Thread Kirill Reshke
Hi On Thu, 24 Oct 2024, 18:08 hugo, <2689496...@qq.com> wrote: > Hi! > >When looking at the partition-related code, I found that the > ispartitioned > > field in CreateStmtContext is not used. It looks like we can safely remove > it and > > avoid invalid assignment logic. > > > > Here's a

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 07:26:07AM +, Bertrand Drouvot wrote: > 1 === Nit > > s/Permutations like this one/Permutations like the following commented out > one/ ? > > 2 === > > s/back its result/back its result once woken up/ ? > > 3 === > > s/faster than the wait/before the SQL function d

Consider pipeline implicit transaction as a transaction block

2024-10-30 Thread Anthonin Bonnefoy
Hi, When using pipelining with implicit transaction, a transaction will start from the first command and be committed with the Sync message. Functions like IsInTransactionBlock and PreventInTransactionBlock already assimilate this implicit transaction as a transaction block, relying on the XACT_FL

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Bertrand Drouvot
Hi, On Wed, Oct 30, 2024 at 09:09:54AM +0100, Peter Eisentraut wrote: > On 29.10.24 14:58, Bertrand Drouvot wrote: > > hi, > > > > On Tue, Oct 29, 2024 at 10:23:37AM +0100, Peter Eisentraut wrote: > > > On 29.10.24 08:54, Bertrand Drouvot wrote: > > > > +static inline bool > > > > +pg_mem_is_all_

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-30 Thread Bertrand Drouvot
Hi, On Wed, Oct 30, 2024 at 06:21:04PM +0900, Michael Paquier wrote: > On Wed, Oct 30, 2024 at 07:26:07AM +, Bertrand Drouvot wrote: > > 1 === Nit > > > > s/Permutations like this one/Permutations like the following commented out > > one/ ? > > > > 2 === > > > > s/back its result/back its

Re: Index AM API cleanup

2024-10-30 Thread Peter Eisentraut
I want to call out one particular aspect that is central to this patch series that needs more broader discussion and agreement. The problem is that btree indexes (and to a lesser extent hash indexes) are hard-coded in various places. This includes various places in the optimizer (see patch v17-0

Re: Fix for consume_xids advancing XIDs incorrectly

2024-10-30 Thread Yushi Ogiwara
I made a new patch (skip_xid_correctly.diff) that incorporates the points we discussed: 1. Fix the issue that consume_xids consumes nxids+1 XIDs. 2. Update lastxid when calling GetTopFullTransactionId() to support nxids==1 case. 3. Forbid consume_xids when nxids==0. 4. Add comments explaining

Re: Pgoutput not capturing the generated columns

2024-10-30 Thread Amit Kapila
On Tue, Oct 29, 2024 at 8:50 PM vignesh C wrote: > > Thank you for reporting this issue. The attached v46 patch addresses > the problem and includes some adjustments to the comments. Thanks to > Amit for sharing the comment changes offline. > Pushed. Kindly rebase and send the remaining patches.

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 07:49, Tatsuo Ishii wrote: > > I think one would have to define that somehow. If it's useful, the > > additional fields of both extensions could be appended, in some > > defined order. But this is an interesting question to think about. > > I think this kind of extension,

RE: Conflict detection for update_deleted in logical replication

2024-10-30 Thread Hayato Kuroda (Fujitsu)
Dear Hou, Thanks for updating the patch! Here are my comments. 01. CreateSubscription ``` +if (opts.detectupdatedeleted && !track_commit_timestamp) +ereport(ERROR, +errcode(ERRCODE_INVALID_PARAMETER_VALUE), +errmsg("detecting update_deleted conflicts re

Re: protocol-level wait-for-LSN

2024-10-30 Thread Tatsuo Ishii
>>> With this protocol extension, two things are changed: >>> >>> - The ReadyForQuery message sends back the current LSN. >> If other protocol extension X tries to add something to the >> ReadyForQuery message too, what would happen? > > I think one would have to define that somehow. If it's usef

Re: AIO writes vs hint bits vs checksums

2024-10-30 Thread Heikki Linnakangas
On 30/10/2024 04:21, Andres Freund wrote: Attached is a, unfortunately long, series of patches implementing what I described upthread. Review of the preparatory patches: 0001 Add very basic test for kill_prior_tuples We currently don't exercise this patch for gist and hash, which seems

Re: Conflict detection for update_deleted in logical replication

2024-10-30 Thread Michail Nikolaev
Hello Hayato! > Note that apply workers can stop due to some reasons (e.g., disabling subscriptions, > error out, deadlock...). In this case, the snapshot cannot eb registered by the > worker and index can be re-built during the period. However, the xmin of a slot affects replication_slot_xmin in

Re: Avoid detoast overhead when possible

2024-10-30 Thread Nikita Malakhov
Hi! Sorry for the long delay, other tasks required my attention and this project was postponed, but we have some recent requests from clients and this work has to be revived. 1) In short the idea behind the Json improvements is to introduce iterators for Json objects to extract these objects valu

Re: Reduce one comparison in binaryheap's sift down

2024-10-30 Thread Robert Haas
On Tue, Oct 29, 2024 at 11:00 AM Nathan Bossart wrote: > On Tue, Oct 29, 2024 at 11:21:15AM +0800, cca5507 wrote: > > Agree, new version patch is attached. > > I might editorialize a bit, but overall this looks pretty good to me. > Robert, would you like to commit this, or shall I? I would be del

Re: Alias of VALUES RTE in explain plan

2024-10-30 Thread Ashutosh Bapat
On Tue, Oct 29, 2024 at 10:49 PM Tom Lane wrote: > > Andrei Lepikhov writes: > > -- New behavior > > EXPLAIN (COSTS OFF, VERBOSE) > > SELECT * FROM (VALUES (4),(2),(3),(1) ORDER BY t1.x LIMIT 2) AS t1(x); > > SELECT * FROM (VALUES (4),(2),(3),(1) ORDER BY t1.x LIMIT 2) AS t1(x); > > After taking

Re: Eager aggregation, take 3

2024-10-30 Thread Robert Haas
On Sat, Oct 5, 2024 at 11:30 PM Richard Guo wrote: > Here’s a comparison of Execution Time and Planning Time for the seven > queries with eager aggregation disabled versus enabled (best of 3). > > Execution Time: > > EAGER-AGG-OFF EAGER-AGG-ON > > q4 105787.963 ms

Re: general purpose array_sort

2024-10-30 Thread Aleksander Alekseev
Hi, Thanks for the updated patch set. > > > +Datum > > > +array_sort_order(PG_FUNCTION_ARGS) > > > +{ > > > +return array_sort(fcinfo); > > > +} > > > + > > > +Datum > > > +array_sort_order_nulls_first(PG_FUNCTION_ARGS) > > > +{ > > > +return array_sort(fcinfo); > > > +} > > > > Any reaso

Re: detoast datum into the given buffer as a optimization.

2024-10-30 Thread Nikita Malakhov
Hi! Andy, I'm truly sorry, I removed quoted messages just to not make replies to walls of text. On detoast_attr_buffer topic - and again agree with reply above that supposes adding required data length as parameter, at least by doing so we should explicitly make the user of this API pay attention

Re: AIO writes vs hint bits vs checksums

2024-10-30 Thread Andres Freund
Hi, Thanks for the quick review! On 2024-10-30 14:16:51 +0200, Heikki Linnakangas wrote: > On 24/09/2024 18:55, Andres Freund wrote: > > What I'd instead like to propose is to implement the right to set hint bits > > as > > a bit in each buffer's state, similar to BM_IO_IN_PROGRESS. Tentatively

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 12:53, Heikki Linnakangas wrote: > We might have made a mistake by calling this mechanism "protocol > extensions", because it makes people think of user-defined extensions. I think this is a real problem, that's probably worth fixing. I created a separate thread to address

Re: general purpose array_sort

2024-10-30 Thread Junwang Zhao
Hi, On Wed, Oct 30, 2024 at 9:29 PM Aleksander Alekseev wrote: > > Hi, > > Thanks for the updated patch set. > > > > > +Datum > > > > +array_sort_order(PG_FUNCTION_ARGS) > > > > +{ > > > > +return array_sort(fcinfo); > > > > +} > > > > + > > > > +Datum > > > > +array_sort_order_nulls_first(PG

Re: Consider pipeline implicit transaction as a transaction block

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 10:15, Anthonin Bonnefoy wrote: > The attached patch adds the detection of implicit transactions started > by a pipeline in CheckTransactionBlock, avoiding warnings when > commands like `set local` are called within a pipeline, and making the > detection of transaction block

Re: [17] CREATE SUBSCRIPTION ... SERVER

2024-10-30 Thread Jeff Davis
On Fri, 2024-03-08 at 00:20 -0800, Jeff Davis wrote: > Implemented in v11, attached. Rebased, v12 attached. Regards, Jeff Davis From 5c2a8f5cb865becd70b08379d9fc72946be9a32a Mon Sep 17 00:00:00 2001 From: Jeff Davis Date: Tue, 2 Jan 2024 13:42:48 -0800 Subject: [PATCH v12] CREATE SUSBCR

Re: AIO writes vs hint bits vs checksums

2024-10-30 Thread Heikki Linnakangas
On 24/09/2024 18:55, Andres Freund wrote: What I'd instead like to propose is to implement the right to set hint bits as a bit in each buffer's state, similar to BM_IO_IN_PROGRESS. Tentatively I named this BM_SETTING_HINTS. It's only allowed to set BM_SETTING_HINTS when BM_IO_IN_PROGRESS isn't al

Re: [PATCH] Add array_reverse() function

2024-10-30 Thread Пополитов Владлен
On Tuesday, October 22, 2024 16:27 MSK, Aleksander Alekseev wrote:   Hi everyone, Thanks for the feedback! > > But it returns the input array as is. I think it should at least make > > a new copy of input array. > > I don't think that's really necessary. We have other functions that > will

Re: Reduce one comparison in binaryheap's sift down

2024-10-30 Thread Nathan Bossart
On Wed, Oct 30, 2024 at 08:09:17AM -0400, Robert Haas wrote: > I would be delighted if you could take care of it. Committed. -- nathan

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-10-30 Thread Bharath Rupireddy
Hi, Thanks for looking into this. On Mon, Oct 28, 2024 at 8:18 PM Jingtang Zhang wrote: > > Just found that the initialization > seems redundant since we have used palloc0? > > > +istate = (HeapInsertState *) palloc0(sizeof(HeapInsertState)); > > +istate->bistate = NULL; > > +

Re: Eager aggregation, take 3

2024-10-30 Thread Robert Haas
On Tue, Oct 29, 2024 at 3:57 AM Richard Guo wrote: > On Fri, Oct 18, 2024 at 10:22 PM jian he wrote: > > So overall I doubt here BTEQUALIMAGE_PROC flag usage is correct. > > The BTEQUALIMAGE_PROC flag is used to prevent eager aggregation for > types whose equality operators do not imply bitwise e

Re: Forbid to DROP temp tables of other sessions

2024-10-30 Thread Rafia Sabih
On Tue, 29 Oct 2024 at 07:22, Daniil Davydov <3daniss...@gmail.com> wrote: > Hi, > Thanks for your comments, I appreciate them. > > As I continued to deal with the topic of working with temp tables of > other sessions, I noticed something like a bug. For example > (REL_17_STABLE): > Session 1: > =

Re: Eager aggregation, take 3

2024-10-30 Thread Robert Haas
On Tue, Oct 29, 2024 at 3:51 AM Richard Guo wrote: > > 2. join type is FULL JOIN, (i am not sure about other Semijoins and > > anti-semijoins types). > > The presence of a FULL JOIN does not preclude the use of eager > aggregation. We still can push a partial aggregation down to a level > that i

Re: Psql meta-command conninfo+

2024-10-30 Thread Hunaid Sohail
Hi, I have attached a new patch that incorporates the approach suggested by David. The documentation has also been updated. $ bin/psql "port=5430 sslmode=disable dbname=postgres" -x -h localhost psql (18devel) Type "help" for help. postgres=# \conninfo+ Connection Information -[ RECORD 1 ]+

Re: Interrupts vs signals

2024-10-30 Thread Heikki Linnakangas
One remaining issue with these patches is backwards-compatibility for extensions: 1. Any extensions using WaitLatch, SetLatch, ResetLatch need to be changed to use WaitInterrupt, SetInterrupt/RaiseInterrupt, ResetInterrupt instead. 2. If an extension is defining its own Latch with InitLatch,

Re: AIO writes vs hint bits vs checksums

2024-10-30 Thread Andres Freund
Hi, On 2024-10-30 13:29:01 +0200, Heikki Linnakangas wrote: > On 30/10/2024 04:21, Andres Freund wrote: > > Attached is a, unfortunately long, series of patches implementing what I > > described upthread. > > Review of the preparatory patches: > > > 0001 Add very basic test for kill_prior_tuples

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jesper Pedersen
Hi, On 10/30/24 1:45 PM, Jelte Fennema-Nio wrote: On Wed, 30 Oct 2024 at 18:18, Ants Aasma wrote: The idea is great, I have been wanting something like this for a long time. For future proofing it might be a good idea to not require the communicated-waited value to be a LSN. Yours and Matthi

small pg_combinebackup improvements

2024-10-30 Thread Robert Haas
Hi, Here are two small patches to improve pg_combinebackup slightly. 0001: I noticed that there is some logic in reconstruct.c that constructs a pathname of the form a/b//c instead of a/b/c. AFAICT, this still works fine; it just looks ugly. It's possible to get one of these pathnames to show up

Re: Shave a few cycles off our ilog10 implementation

2024-10-30 Thread Heikki Linnakangas
On 30/10/2024 21:27, David Fetter wrote: Please find attached a patch to $Subject I've done some preliminary testing, and it appears to shave somewhere between 25-50% off the operations themselves, and these cascade into things like formatting result sets and COPY OUT. Impressive! What did you

Re: Proposal for Integrating Data Masking and anonymization into PostgreSQL

2024-10-30 Thread Stefan Keller
Hi Osman See postgresql_anonymizer https://gitlab.com/dalibo/postgresql_anonymizer/ and pg_anonymize https://github.com/rjuju/pg_anonymize , where the former has more functionality, like synthetic data generation (if you need that). Yours, Stefan Am Di., 29. Okt. 2024 um 10:01 Uhr schrieb Hosney

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 19:04, Jesper Pedersen wrote: > Having LSN would be nice, but to break all existing implementations, no. > Having to specify with startup parameters how a core message format > looks like sounds like a bad idea to me, It would really help if you would explain why you think

Re: Shave a few cycles off our ilog10 implementation

2024-10-30 Thread David Fetter
On Wed, Oct 30, 2024 at 09:54:20PM +0200, Heikki Linnakangas wrote: > On 30/10/2024 21:27, David Fetter wrote: > > Please find attached a patch to $Subject > > > > I've done some preliminary testing, and it appears to shave somewhere > > between 25-50% off the operations themselves, and these casc

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Wed, 30 Oct 2024 at 18:18, Ants Aasma wrote: > The idea is great, I have been wanting something like this for a long > time. For future proofing it might be a good idea to not require the > communicated-waited value to be a LSN. Yours and Matthias' feedback make total sense I think. From an im

Re: AIO writes vs hint bits vs checksums

2024-10-30 Thread Jeff Davis
On Tue, 2024-09-24 at 11:55 -0400, Andres Freund wrote: > What I suspect we might want instead is something inbetween a share > and an > exclusive lock, which is taken while setting a hint bit and which > conflicts > with having an IO in progress. I am starting to wonder if a shared content locks

Re: AIO writes vs hint bits vs checksums

2024-10-30 Thread Andres Freund
Hi, On 2024-10-30 10:47:35 -0700, Jeff Davis wrote: > On Tue, 2024-09-24 at 11:55 -0400, Andres Freund wrote: > > What I suspect we might want instead is something inbetween a share > > and an > > exclusive lock, which is taken while setting a hint bit and which > > conflicts > > with having an IO

Re: Increase of maintenance_work_mem limit in 64-bit Windows

2024-10-30 Thread Vladlen Popolitov
v.popoli...@postgrespro.ru писал(а) 2024-10-01 00:30: David Rowley писал(а) 2024-09-24 01:07: On Tue, 24 Sept 2024 at 02:47, Vladlen Popolitov wrote: I agree, it is better to fix all them together. I also do not like this hack, it will be removed from the patch, if I check and change all at

Re: Wrong result when enable_partitionwise_join is on if collation of PartitionKey and Column is different.

2024-10-30 Thread Junwang Zhao
On Wed, Oct 30, 2024 at 11:58 AM jian he wrote: > > I missed a case when column collation and partition key collation are > the same and indeterministic. > that should be fine for partition-wise join. > so v2 attached. > > have_partkey_equi_join, match_expr_to_partition_keys didn't do any > collat

Shave a few cycles off our ilog10 implementation

2024-10-30 Thread David Fetter
Please find attached a patch to $Subject I've done some preliminary testing, and it appears to shave somewhere between 25-50% off the operations themselves, and these cascade into things like formatting result sets and COPY OUT. Best, David. -- David Fetter http://fetter.org/ Phone: +1 415 235

Re: Interrupts vs signals

2024-10-30 Thread Robert Haas
On Wed, Oct 30, 2024 at 12:03 PM Heikki Linnakangas wrote: > 1. Any extensions using WaitLatch, SetLatch, ResetLatch need to be > changed to use WaitInterrupt, SetInterrupt/RaiseInterrupt, > ResetInterrupt instead. > > 2. If an extension is defining its own Latch with InitLatch, rather than > usin

Re: protocol-level wait-for-LSN

2024-10-30 Thread Ants Aasma
On Mon, 28 Oct 2024 at 17:51, Peter Eisentraut wrote: > This is something I hacked together on the way back from pgconf.eu. > It's highly experimental. > > The idea is to do the equivalent of pg_wal_replay_wait() on the protocol > level, so that it is ideally fully transparent to the application c

Re: Shave a few cycles off our ilog10 implementation

2024-10-30 Thread David Rowley
On Thu, 31 Oct 2024 at 09:02, David Fetter wrote: > This testing was basically just sending a bunch of appropriately sized > pseudo-random uints in a previously created array sent through a tight > loop that called the ilog10s and getting average execution times. > > Any suggestions for more thoro

Re: Large expressions in indexes can't be stored (non-TOASTable)

2024-10-30 Thread Nathan Bossart
On Fri, Oct 18, 2024 at 08:08:55AM +0900, Michael Paquier wrote: > 0002 and 0003 look sane enough to me as shaped. I'd need to spend a > few more hours on 0001 if I were to do a second round on it, but you > don't really need a second opinion, do you? :D I'll manage. 0001 was a doozy to back-pa

Re: Popcount optimization using AVX512

2024-10-30 Thread Raghuveer Devulapalli
BTW, I just realized function attributes for xsave and avx512 don't work on MSVC (see https://developercommunity.visualstudio.com/t/support-function-target-attribute-and-mutiversioning/10130630). Not sure if you care about it. Its an easy fix (see https://gcc.godbolt.org/z/Pebdj3vMx).

Re: [PATCH] Add array_reverse() function

2024-10-30 Thread Vladlen Popolitov
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed Content. The proposed function waited long to be implemented

Re: protocol-level wait-for-LSN

2024-10-30 Thread Heikki Linnakangas
On 30/10/2024 13:34, Tatsuo Ishii wrote: On Wed, 30 Oct 2024 at 07:49, Tatsuo Ishii wrote: I think one would have to define that somehow. If it's useful, the additional fields of both extensions could be appended, in some defined order. But this is an interesting question to think about. I

Re: protocol-level wait-for-LSN

2024-10-30 Thread Tatsuo Ishii
> On Wed, 30 Oct 2024 at 07:49, Tatsuo Ishii wrote: >> > I think one would have to define that somehow. If it's useful, the >> > additional fields of both extensions could be appended, in some >> > defined order. But this is an interesting question to think about. >> >> I think this kind of exte

Re: doc issues in event-trigger-matrix.html

2024-10-30 Thread jian he
On Tue, Oct 29, 2024 at 7:54 PM Peter Eisentraut wrote: > > I made a patch for this. I have expanded the narrative discussion on > what commands are supported for event triggers, also made a few > corrections/additions there, based on inspecting the source code. And > then removed the big matrix

Re: [PATCH] Add roman support for to_number function

2024-10-30 Thread Hunaid Sohail
Hi, I have attached a rebased patch if someone wants to review in the next CF. No changes as compared to v4. Regards, Hunaid Sohail > From 951f3e9620f0d50fcdaff095652a07d6304b490c Mon Sep 17 00:00:00 2001 From: Hunaid Sohail Date: Wed, 30 Oct 2024 12:00:47 +0500 Subject: [PATCH v5] Add roman su

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-30 Thread Bertrand Drouvot
Hi, On Wed, Oct 30, 2024 at 12:40:05PM +0900, Michael Paquier wrote: > On Tue, Oct 29, 2024 at 02:06:11PM +, Bertrand Drouvot wrote: > > Yeah, agree that it would make sense to document in the test what has been > > discovered here. > > How about something like the attached? Thanks! A few c

Use "protocol options" name instead of "protocol extensions" everywhere

2024-10-30 Thread Jelte Fennema-Nio
It was pointed out by Heikki in the thread around protocol-level wait-for-LSN that "protocol extensions" is a pretty confusing name, since it suggests a relation to Postgres extensions. Even though there is no such relationship at all. Attached is a small patch that aligns on the name "protocol opt

Re: freespace.c modifies buffer without any locks

2024-10-30 Thread Heikki Linnakangas
On 29/10/2024 02:50, Andres Freund wrote: Hi, I just noticed that fsm_vacuum_page() modifies a buffer without even holding a shared lock. That quite obviously seems like a violation of the buffer locking protocol: /* * Try to reset the next slot pointer. This encourages the us

Re: general purpose array_sort

2024-10-30 Thread Junwang Zhao
On Wed, Oct 30, 2024 at 10:17 PM Junwang Zhao wrote: > > Hi, > > On Wed, Oct 30, 2024 at 9:29 PM Aleksander Alekseev > wrote: > > > > Hi, > > > > Thanks for the updated patch set. > > > > > > > +Datum > > > > > +array_sort_order(PG_FUNCTION_ARGS) > > > > > +{ > > > > > +return array_sort(fcin

Re: protocol-level wait-for-LSN

2024-10-30 Thread Tatsuo Ishii
>> So yes, each protocol extension needs to know about all the other >> protocol extensions that it can be used with. In practice we'll avoid >> doing crazy stuff so that the protocol extensions are orthogonal > > Just as an example, let's say we add a server-side query time to the > protocol (whi

Re: Popcount optimization using AVX512

2024-10-30 Thread Nathan Bossart
On Wed, Oct 30, 2024 at 08:53:10PM +, Raghuveer Devulapalli wrote: > BTW, I just realized function attributes for xsave and avx512 don't work > on MSVC (see > https://developercommunity.visualstudio.com/t/support-function-target-attribute-and-mutiversioning/10130630). > Not sure if you care abo

Re: New "raw" COPY format

2024-10-30 Thread Masahiko Sawada
On Tue, Oct 29, 2024 at 9:48 AM Joel Jacobson wrote: > > > --- > > It's a bit odd to me to use the delimiter as a EOL marker in raw > > format, but probably it's okay. > > > > --- > > - if (cstate->opts.format != COPY_FORMAT_BINARY) > > + if (cstate->opts.format == COPY_FORMAT_

Re: CREATE OR REPLACE MATERIALIZED VIEW

2024-10-30 Thread Erik Wienhold
On 2024-09-05 22:33 +0200, Erik Wienhold wrote: > On 2024-07-27 02:45 +0200, Erik Wienhold wrote: > > On 2024-07-12 16:49 +0200, Said Assemlal wrote: > > > > My initial idea, while writing the patch, was that one could replace the > > > > matview without populating it and then run the concurrent re

Re: per backend I/O statistics

2024-10-30 Thread Bertrand Drouvot
Hi, On Tue, Oct 08, 2024 at 04:28:39PM +, Bertrand Drouvot wrote: > > > On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote: > > > > Okay, per the above and the persistency of the stats. > > Great, I'll work on an updated patch version then. > I spend some time on this during t

Re: Having problems generating a code coverage report

2024-10-30 Thread Peter Geoghegan
On Wed, Oct 30, 2024 at 5:21 PM Aleksander Alekseev wrote: > I'm using Xubuntu 24.04 LTS and my lcov version is: > > ``` > $ lcov --version > lcov: LCOV version 2.0-1 > ``` I use Debian unstable for most of my day to day work. Apparently Debian unstable has exactly the same version of lcov as Ubu

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Peter Smith
+/* + * Test if a memory region starting at p and of size len is full of zeroes. + */ +static inline bool +pg_mem_is_all_zeros(const void *ptr, size_t len) The function comment should say 'ptr' instead of 'p', right? == Kind Regards, Peter Smith. Fujitsu Australia

Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM

2024-10-30 Thread Jingtang Zhang
Hi~ I did some performance test these days, and I have some findings. From the archive months ago, I found there were discussions about which type of TupleTableSlot to use for buffering tuples. A single column mat view was used for evaluation. Finally we used virtual one. However when I test wit

Re: Large expressions in indexes can't be stored (non-TOASTable)

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 03:54:32PM -0500, Nathan Bossart wrote: > I'll manage. 0001 was a doozy to back-patch, and this obviously isn't a > terribly pressing issue, so I plan to wait until after the November minor > release to apply this. Okay by me. > I'm having second thoughts on 0002, so I'll

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 09:27:13AM +, Bertrand Drouvot wrote: > Thanks! LGTM. Thanks for the review. Applied that, then. -- Michael signature.asc Description: PGP signature

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Michael Paquier
On Thu, Oct 31, 2024 at 07:55:45AM +1100, Peter Smith wrote: > +/* > + * Test if a memory region starting at p and of size len is full of zeroes. > + */ > +static inline bool > +pg_mem_is_all_zeros(const void *ptr, size_t len) > > The function comment should say 'ptr' instead of 'p', right? Yes.

Re: Avoid detoast overhead when possible

2024-10-30 Thread Andy Fan
Nikita Malakhov writes: > 1) In short the idea behind the Json improvements is to introduce iterators > for Json objects to extract these objects value by value, OK here. > and modification > of in-memory and on-disk representations to hold partially-detoasted Json > objects (tree nodes with n

Re: Add ExprState hashing for GROUP BY and hashed SubPlans

2024-10-30 Thread David Rowley
On Tue, 29 Oct 2024 at 22:47, David Rowley wrote: > I've attached an updated patch with a few other fixes. Whilr checking > this tonight, noticed that master does not use > SubPlanState.tab_eq_funcs for anything. I resisted removing that in > this patch. Perhaps a follow-on patch can remove that.

Count and log pages set all-frozen by vacuum

2024-10-30 Thread Melanie Plageman
Vacuum currently counts and logs the number of pages of a relation with newly frozen tuples. It does not count the number of pages newly set all-frozen in the visibility map. The number of pages set all-frozen in the visibility map by a given vacuum can be useful when analyzing which normal vacuum

RE: Popcount optimization using AVX512

2024-10-30 Thread Devulapalli, Raghuveer
> Oh, good catch. IIUC we only need to check for #ifndef _MSC_VER in the > configure programs for meson. pg_attribute_target will be empty on MSVC, and > I > believe we only support meson builds there. Right. __has_attribute (target) produces a compiler warning on MSVC: https://gcc.godbolt.o

Re: Pgoutput not capturing the generated columns

2024-10-30 Thread Peter Smith
On Thu, Oct 31, 2024 at 1:14 PM vignesh C wrote: > > On Thu, 31 Oct 2024 at 04:42, Peter Smith wrote: > > > > On Thu, Oct 31, 2024 at 3:16 AM vignesh C wrote: > > > > > > On Wed, 30 Oct 2024 at 15:06, Amit Kapila wrote: > > > > > > > > On Tue, Oct 29, 2024 at 8:50 PM vignesh C wrote: > > > > >

Having problems generating a code coverage report

2024-10-30 Thread Aleksander Alekseev
Hi hackers, Recently I tried to build a code coverage report according to the documentation [1]. When using the following command: ``` time sh -c 'git clean -dfx && meson setup --buildtype debug -DPG_TEST_EXTRA="kerberos ldap ssl" -Db_coverage=true -Dldap=disabled -Dssl=openssl -Dcassert=true -Dt

Re: Interrupts vs signals

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 01:23:54PM -0400, Robert Haas wrote: > On Wed, Oct 30, 2024 at 12:03 PM Heikki Linnakangas wrote: >> We can provide backwards compatibility macros and a new facility to >> allocate custom interrupt bits. But how big of a problem is this anyway? >> In an in-person chat last

Re: Cleanup SubPlanstate

2024-10-30 Thread David Rowley
On Wed, 30 Oct 2024 at 03:34, Rafia Sabih wrote: > While reviewing a related patch, it came to the notice that tab_eq_funcs in > the SubPlanState is not used. Hence the patch. Thanks for picking that up. I didn't quite understand why you adjusted the header comment for the TupleHashEntryData str

RE: Conflict detection for update_deleted in logical replication

2024-10-30 Thread Hayato Kuroda (Fujitsu)
Dear Mikhail, Thanks for the reply! > > Anyway, this topic introduces huge complexity and is not mandatory for > > update_deleted > > detection. We can work on it in later versions based on the needs. > > From my perspective, this is critical for databases. REINDEX CONCURRENTLY is > typically r

Re: CREATE OR REPLACE MATERIALIZED VIEW

2024-10-30 Thread Michael Paquier
On Tue, Jul 02, 2024 at 01:46:21PM +0300, Aleksander Alekseev wrote: > I can imagine how this may impact many applications and upset many > software developers worldwide. Was there even a precedent (in the > recent decade or so) when PostgreSQL broke the SQL syntax? We're usually very careful abou

Re: [PATCH] Add array_reverse() function

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 10:11:20PM +0300, Пополитов Владлен wrote: > Makes me wonder if we should have an utility function which would > contain common code for array_shuffle() and array_reverse(). > Considering inconveniences such an approach caused in case of common > code for text and bytea type

Re: Pgoutput not capturing the generated columns

2024-10-30 Thread vignesh C
On Thu, 31 Oct 2024 at 04:42, Peter Smith wrote: > > On Thu, Oct 31, 2024 at 3:16 AM vignesh C wrote: > > > > On Wed, 30 Oct 2024 at 15:06, Amit Kapila wrote: > > > > > > On Tue, Oct 29, 2024 at 8:50 PM vignesh C wrote: > > > > > > > > Thank you for reporting this issue. The attached v46 patch

Re: Pgoutput not capturing the generated columns

2024-10-30 Thread Peter Smith
On Thu, Oct 31, 2024 at 3:16 AM vignesh C wrote: > > On Wed, 30 Oct 2024 at 15:06, Amit Kapila wrote: > > > > On Tue, Oct 29, 2024 at 8:50 PM vignesh C wrote: > > > > > > Thank you for reporting this issue. The attached v46 patch addresses > > > the problem and includes some adjustments to the c

Re: Add ExprState hashing for GROUP BY and hashed SubPlans

2024-10-30 Thread Andrei Lepikhov
On 10/31/24 08:16, David Rowley wrote: On Tue, 29 Oct 2024 at 22:47, David Rowley wrote: I've attached an updated patch with a few other fixes. Whilr checking this tonight, noticed that master does not use SubPlanState.tab_eq_funcs for anything. I resisted removing that in this patch. Perhaps a

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Bertrand Drouvot
Hi, On Thu, Oct 31, 2024 at 09:59:35AM +0900, Michael Paquier wrote: > On Thu, Oct 31, 2024 at 07:55:45AM +1100, Peter Smith wrote: > > +/* > > + * Test if a memory region starting at p and of size len is full of zeroes. > > + */ > > +static inline bool > > +pg_mem_is_all_zeros(const void *ptr, si

Re: Having problems generating a code coverage report

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 05:26:49PM -0400, Peter Geoghegan wrote: > I use Debian unstable for most of my day to day work. Apparently > Debian unstable has exactly the same version of lcov as Ubuntu 24.04. > > I've also been unable to generate coverage reports for some time (at > least on Debian, wit

Re: Allowing pg_recvlogical to create temporary replication slots

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 09:00:59AM +0100, Peter Eisentraut wrote: > I think you should explain a bit why you want to do that, what use case or > scenario this is meant to address. Indeed. Note also this page giving a couple more guidelines with more details: https://wiki.postgresql.org/wiki/Submi

Re: Inval reliability, especially for inplace updates

2024-10-30 Thread Noah Misch
On Mon, Oct 28, 2024 at 02:27:03PM +0530, Nitin Motiani wrote: > On Thu, Oct 24, 2024 at 8:24 AM Noah Misch wrote: > > With the releases wrapping in 2.5 weeks, I'm ambivalent about pushing this > > before the release or after. Pushing before means fewer occurrences of > > corruption, but pushing

Fix for Extra Parenthesis in pgbench progress message

2024-10-30 Thread Yushi Ogiwara
Hi, I noticed an issue in the pgbench progress message where an extra closing parenthesis )) appears, as shown below: 700 of 1000 tuples (70%) of pgbench_accounts done (elapsed 19.75 s, remaining 8.46 s)) This occurs when running commands like pgbench -i -s100 and is caused by left

Re: Segfault in jit tuple deforming on arm64 due to LLVM issue

2024-10-30 Thread Thomas Munro
On Thu, Oct 17, 2024 at 10:41 PM Thomas Munro wrote: > Thanks! I'm going to go ahead and commit this. (Sorry for the delay, I got distracted by pgconf.eu.) Today I set out to commit this patch, and wrote a proper commit message to explain the code provenance, circumstances that led to it, and t

Re: Consider pipeline implicit transaction as a transaction block

2024-10-30 Thread Michael Paquier
On Wed, Oct 30, 2024 at 04:06:20PM +0100, Jelte Fennema-Nio wrote: > On Wed, 30 Oct 2024 at 10:15, Anthonin Bonnefoy > wrote: >> The attached patch adds the detection of implicit transactions started >> by a pipeline in CheckTransactionBlock, avoiding warnings when >> commands like `set local` are

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Michael Paquier
On Thu, Oct 31, 2024 at 05:52:49AM +, Bertrand Drouvot wrote: > That makes sense to me, done that way in the attached. Seems kind of OK seen from here. I am wondering if others more comments about the name of the macro, its location, the fact that we still have pagebytes in bufpage.c, etc. --

Re: unicode test programs don't build with meson

2024-10-30 Thread Peter Eisentraut
On 18.10.24 15:17, Aleksander Alekseev wrote: 1) Add libintl directly to the affected programs, like case_test = executable('case_test', ['case_test.c'], - dependencies: [frontend_port_code, icu], + dependencies: [frontend_port_code, icu, libintl], include_directories: inc, l

Re: Allowing pg_recvlogical to create temporary replication slots

2024-10-30 Thread Peter Eisentraut
On 27.10.24 13:37, Torsten Förtsch wrote: This is my very first message to this mailing list. Please advise if I am making any mistakes in the procedure. Welcome! The attached patch enables pg_recvlogical to create a temporary slot. What is the next step in the process to get this change int

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Peter Eisentraut
On 29.10.24 14:58, Bertrand Drouvot wrote: hi, On Tue, Oct 29, 2024 at 10:23:37AM +0100, Peter Eisentraut wrote: On 29.10.24 08:54, Bertrand Drouvot wrote: +static inline bool +pg_mem_is_all_zeros(const char *p, size_t len) This should be a void * pointer please. Thanks for looking at it!

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Mon, 28 Oct 2024 at 16:51, Peter Eisentraut wrote: > Thoughts? + snprintf(xloc, sizeof(xloc), "%X/%X", LSN_FORMAT_ARGS(logptr)) + pq_sendstring(&buf, xloc); nit: I feel that sending the LSN as a string seems unnecessarily wasteful of bytes. I'd rather send

Re: protocol-level wait-for-LSN

2024-10-30 Thread Jelte Fennema-Nio
On Mon, 28 Oct 2024 at 17:58, Heikki Linnakangas wrote: > > - The Query message sends an LSN to wait for. (This doesn't handle the > > extended query protocol yet.) > > I'd suggest adding a new message type for this, so that it works the > same with simple and extended query. Or if you just want