Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Bertrand Drouvot
Hi, On Wed, Nov 13, 2024 at 09:25:37AM +0900, Michael Paquier wrote: > So that seems worth the addition, especially for > smaller sizes where this is 6 times faster here. So, something like v12 in pg_memory_is_all_zeros_v12() in allzeros_small.c attached? If so, that gives us: == with BLCKSZ 32

Re: Enhancing Memory Context Statistics Reporting

2024-11-12 Thread Rahila Syed
Hi, Thank you for the review. > > Hmm, would it make sene to use dynamic shared memory for this? The > publishing backend could dsm_create one DSM chunk of the exact size that > it needs, pass the dsm_handle to the consumer, and then have it be > destroy once it's been read. That way you don't

Re: Reordering DISTINCT keys to match input path's pathkeys

2024-11-12 Thread Richard Guo
On Mon, Oct 28, 2024 at 6:15 PM Andrei Lepikhov wrote: > On 6/7/24 16:46, Richard Guo wrote: > > This patch does not apply any more, so here is a new rebase, with some > > tweaks to the comments. > This patch needs a minor rebase again. > After skimming the code, I want to say that it looks good.

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Bertrand Drouvot
Hi, On Tue, Nov 12, 2024 at 01:32:36PM -0300, Ranier Vilela wrote: > It seems to me that it is enough to protect the SIMD loop when the size is > smaller. > > if (len > sizeof(size_t) * 8) > { > for (; p < aligned_end - (sizeof(size_t) * 7); p += sizeof(size_t) * > 8) > { >

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-11-12 Thread Amit Kapila
On Tue, Nov 12, 2024 at 7:05 PM Alvaro Herrera wrote: > > On 2024-Nov-12, Amit Kapila wrote: > > > I think we still need a fix for the master for the case when generated > > columns are not published but are part of REPLICA IDENTITY as that > > could lead to failures in applying UPDATE and DELETE

Re: Statistics Import and Export

2024-11-12 Thread Corey Huinker
> > We should probably not allow that, because you cannot ANALYZE system > columns: > Makes sense, and the fix is changing a single character (unless we think it warrants a test case).

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-11-12 Thread Shlok Kyal
Thanks for providing the comments. On Tue, 12 Nov 2024 at 12:52, Zhijie Hou (Fujitsu) wrote: > > On Friday, November 8, 2024 7:06 PM Shlok Kyal > wrote: > > > > Hi Amit, > > > > On Thu, 7 Nov 2024 at 11:37, Amit Kapila wrote: > > > > > > On Tue, Nov 5, 2024 at 12:53 PM Shlok Kyal > > wrote: >

Re: logical replication: restart_lsn can go backwards (and more), seems broken since 9.4

2024-11-12 Thread Ashutosh Bapat
On Tue, Nov 12, 2024 at 4:55 PM Tomas Vondra wrote: > > > > On 11/12/24 10:37, Ashutosh Bapat wrote: > > On Tue, Nov 12, 2024 at 4:54 AM Tomas Vondra wrote: > >> > >> > >> > >> On 11/11/24 23:41, Masahiko Sawada wrote: > >>> On Mon, Nov 11, 2024 at 6:17 AM Tomas Vondra wrote: > >>> > >>> Which m

RE: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Hayato Kuroda (Fujitsu)
Dear Andres, > I don't think the suggested workload is useful here. > pg_logical_emit_message(transactional = false) > does insert the WAL without the commit, i.e., xlcommitrecis always NULL. > This means backends won't go through added codes in > ReserveXLogInsertLocation(). Just in case I want

Re: Virtual generated columns

2024-11-12 Thread jian he
in transformColumnDefinition we can add parser_errposition for the error report. if (column->is_not_null && column->generated == ATTRIBUTE_GENERATED_VIRTUAL) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("not-null constra

Re: Skip collecting decoded changes of already-aborted transactions

2024-11-12 Thread vignesh C
On Mon, 11 Nov 2024 at 23:30, Masahiko Sawada wrote: > > On Sun, Nov 10, 2024 at 11:24 PM Peter Smith wrote: > > > > Hi Sawada-San, here are some review comments for the patch v5-0001. > > > > Thank you for reviewing the patch! > > > == > > Commit message. > > > > 1. > > This commit introduce

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-11-12 Thread Masahiro Ikeda
On 2024-11-13 00:55, Peter Geoghegan wrote: On Sun, Nov 10, 2024 at 11:36 PM Masahiro Ikeda wrote: Thanks! The change made it easier for me to understand. As follow-up to all of the recent work in this area, I'd like to add this wrapper function to return the next item from so->currPos. The

RE: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Hayato Kuroda (Fujitsu)
Dear Andres, Thanks for giving comments for my test! > This is not a useful measurement for overhead introduced in > ReserveXLogInsertLocation(). What you're measuring here is the number of > commits/second, not the WAL insertion rate. The number of commits/second is > largely determined by your

Re: Fix to increment the index scan counter for the bloom filter index

2024-11-12 Thread Masahiro Ikeda
On 2024-11-13 06:23, Peter Geoghegan wrote: On Tue, Nov 12, 2024 at 5:01 AM Masahiro Ikeda wrote: I noticed that the bloom filter index forgets to increment the index scan counter while reviewing the skip scan patch [1]. It seems this was simply overlooked in the implementation. What do you thi

Re: Fix to increment the index scan counter for the bloom filter index

2024-11-12 Thread Peter Geoghegan
On Tue, Nov 12, 2024 at 7:45 PM Masahiro Ikeda wrote: > On 2024-11-13 06:23, Peter Geoghegan wrote: > > I'll commit this and backpatch once the release tags for the pending > > set of point releases are pushed. > > OK, thanks. Pushed. Thanks -- Peter Geoghegan

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Ranier Vilela
Em ter., 12 de nov. de 2024 às 21:33, Michael Paquier escreveu: > On Tue, Nov 12, 2024 at 01:32:36PM -0300, Ranier Vilela wrote: > > See v1_allzeros_small.c attached. > > In your pg_memory_is_all_zeros_v11: > while (((uintptr_t) p & (sizeof(size_t) - 1)) != 0) > { > if (p == end)

Re: Conflict detection for update_deleted in logical replication

2024-11-12 Thread Masahiko Sawada
On Tue, Nov 12, 2024 at 2:19 AM Zhijie Hou (Fujitsu) wrote: > > On Friday, October 18, 2024 5:45 PM Amit Kapila > wrote: > > On Tue, Oct 15, 2024 at 5:03 PM Amit Kapila > > wrote: > > > > > > On Mon, Oct 14, 2024 at 9:09 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > We thought of few op

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Michael Paquier
On Tue, Nov 12, 2024 at 01:32:36PM -0300, Ranier Vilela wrote: > See v1_allzeros_small.c attached. In your pg_memory_is_all_zeros_v11: while (((uintptr_t) p & (sizeof(size_t) - 1)) != 0) { if (p == end) return true; if (*p++ != 0) return false;

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Michael Paquier
On Tue, Nov 12, 2024 at 10:56:20AM +, Bertrand Drouvot wrote: > I think that depends of the memory area size. If the size is small enough > then the > byte per byte can be good enough. > > For example, with the allzeros_small.c attached: > > == with BLCKSZ 32 > > $ /usr/local/gcc-14.1.0/bin

Re: Vacuum statistics

2024-11-12 Thread Jim Nasby
On Nov 10, 2024, at 2:09 PM, Alena Rybakina wrote: On 08.11.2024 22:34, Jim Nasby wrote: On Nov 2, 2024, at 7:22 AM, Alena Rybakina wrote:

Re: [PoC] XMLCast (SQL/XML X025)

2024-11-12 Thread Jim Jones
On 12.11.24 15:59, Robert Haas wrote: > Those are good things to check, but we also need to consider how it > interacts with features PostgreSQL itself already has. I totally agree. It just didn't occur to me to check how XMLTABLE() deals with these conversions :) > In particular, > I'm conce

Re: Fix array access (src/bin/pg_dump/pg_dump.c)

2024-11-12 Thread Ranier Vilela
Em ter., 12 de nov. de 2024 às 16:11, Alvaro Herrera < alvhe...@alvh.no-ip.org> escreveu: > On 2024-Nov-12, Ranier Vilela wrote: > > > Per Coverity. > > > > The function *determineNotNullFlags* has a little oversight. > > The struct field *notnull_islocal* is an array. > > > > I think this is a si

Re: [PoC] Federated Authn/z with OAUTHBEARER

2024-11-12 Thread Jacob Champion
On Fri, Nov 8, 2024 at 1:21 AM Peter Eisentraut wrote: > Assorted review comments from me: Thank you! I will cherry-pick some responses here and plan to address the rest in a future patchset. > trust_validator_authz: Personally, I'm not a fan of the "authz" and > "authn" abbreviations. I know t

Re: doc: pgevent.dll location

2024-11-12 Thread Andres Freund
Hi, On 2024-11-12 16:11:44 -0500, Robert Haas wrote: > On Tue, Nov 12, 2024 at 3:06 PM Peter Eisentraut wrote: > > In this context, a shared module is something like plpgsql or hstore > > that you dlopen, and a shared library is something like libpq or libecpg > > that you -l at build time. > > T

Re: Fix to increment the index scan counter for the bloom filter index

2024-11-12 Thread Peter Geoghegan
On Tue, Nov 12, 2024 at 5:01 AM Masahiro Ikeda wrote: > I noticed that the bloom filter index forgets to increment the index > scan counter > while reviewing the skip scan patch [1]. It seems this was simply > overlooked in > the implementation. What do you think? I think that you're right. I'll

Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE

2024-11-12 Thread Robert Haas
On Tue, Nov 12, 2024 at 4:02 PM Guillaume Lelarge wrote: > Sure looks easy enough to do (though it still lacks doc and tests changes). > See patch attached. Yep, that's very small. I'm a bit wondering if it's too small, though. standard_ExplainOneQuery() seems to do some stuff with es->buffers e

Re: Statistics Import and Export

2024-11-12 Thread Heikki Linnakangas
On 23/10/2024 01:27, Jeff Davis wrote: I've taken most of Jeff's work, reincorporated it into roughly the same patch structure as before, and am posting it now. I committed 0001-0004 with significant revision. This just caught my eye: postgres=# select pg_set_attribute_stats('foo', 'xmin', f

Re: doc: pgevent.dll location

2024-11-12 Thread Robert Haas
On Tue, Nov 12, 2024 at 3:06 PM Peter Eisentraut wrote: > On 12.11.24 18:27, Robert Haas wrote: > > On Tue, Nov 12, 2024 at 11:52 AM Peter Eisentraut > > wrote: > >> So the most straightforward way to "make it work like it used to" would > >> be to change src/bin/pgevent/meson.build to use share

Re: Improve error messages for database object stats manipulation functions during recovery

2024-11-12 Thread Heikki Linnakangas
On 25/10/2024 20:07, Fujii Masao wrote: Hi, When database object stats manipulation functions like pg_set_relation_stats() are run, they currently produce the following error and hint messages, which are "internal" and make it hard for users to understand the issue:   ERROR:  cannot acq

Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE

2024-11-12 Thread Guillaume Lelarge
Le mar. 12 nov. 2024 à 16:35, Guillaume Lelarge a écrit : > Le mar. 12 nov. 2024 à 16:21, Robert Haas a > écrit : > >> On Mon, Nov 11, 2024 at 3:59 PM Guillaume Lelarge >> wrote: >> > Agreed. Having an "EXPLAIN (ALL)" would be a great addition. I could >> tell a customer to do an "EXPLAIN (ALL)

Re: Showing primitive index scan count in EXPLAIN ANALYZE (for skip scan and SAOP scans)

2024-11-12 Thread Peter Geoghegan
On Sun, Nov 10, 2024 at 2:00 PM Alena Rybakina wrote: > Or maybe I was affected by fatigue, but I don’t understand this point, to be > honest. I see from the documentation and your first letter that it specifies > how many times in total the tuple search would be performed during the index > ex

Re: doc: pgevent.dll location

2024-11-12 Thread Peter Eisentraut
On 12.11.24 18:27, Robert Haas wrote: On Tue, Nov 12, 2024 at 11:52 AM Peter Eisentraut wrote: So the most straightforward way to "make it work like it used to" would be to change src/bin/pgevent/meson.build to use shared_module() instead of shared_library(). Based on the explanation in the do

Re: logical replication: restart_lsn can go backwards (and more), seems broken since 9.4

2024-11-12 Thread Masahiko Sawada
On Tue, Nov 12, 2024 at 4:08 AM Ashutosh Bapat wrote: > > On Tue, Nov 12, 2024 at 4:55 PM Tomas Vondra wrote: > > > > but I'm still wondering if the current coding was intentional and we're > > just missing why it was written like this. > > Interestingly, the asymmetry between the functions is ad

Re: pg_rewind WAL segments deletion pitfall

2024-11-12 Thread Alvaro Herrera
Hello After reading the whole thread a couple of times to make sure I understood the problem correctly, I think the approach in the v10 patch is a reasonable one. I agree that it's better for maintainability to keep a separate hash table. I made some cosmetic adjustments -- didn't find any fault

Re: Fix array access (src/bin/pg_dump/pg_dump.c)

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-12, Ranier Vilela wrote: > Per Coverity. > > The function *determineNotNullFlags* has a little oversight. > The struct field *notnull_islocal* is an array. > > I think this is a simple typo. > Fix using array notation access. Yeah, thanks, I had been made aware of this bug. Before

Re: doc: pgevent.dll location

2024-11-12 Thread Robert Haas
On Tue, Nov 12, 2024 at 11:52 AM Peter Eisentraut wrote: > So the most straightforward way to "make it work like it used to" would > be to change src/bin/pgevent/meson.build to use shared_module() instead > of shared_library(). > > Based on the explanation in the documentation, this file is really

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Andres Freund
Hi, On 2024-11-12 11:40:39 -0500, Jan Wieck wrote: > On 11/12/24 10:34, Andres Freund wrote: > > I have working code - pretty ugly at this state, but mostly needs a fair bit > > of elbow grease not divine inspiration... It's not a trivial change, but > > entirely doable. > > > > The short summar

Re: Parallel workers stats in pg_stat_database

2024-11-12 Thread Benoit Lobréau
On 11/12/24 16:24, Michael Banck wrote: I am not sure "backend state" is a good reason (unless it is exposed somewhere to users?), but the point about utilities does make sense I guess. We only track parallel workers used by queries right now. Parallel index builds (btree & brin) and vacuum

Fix array access (src/bin/pg_dump/pg_dump.c)

2024-11-12 Thread Ranier Vilela
Hi. Per Coverity. The function *determineNotNullFlags* has a little oversight. The struct field *notnull_islocal* is an array. I think this is a simple typo. Fix using array notation access. Trivial patch attached. best regards, Ranier Vilela fix_array_access_pg_dump.patch Description: Binar

Re: doc: pgevent.dll location

2024-11-12 Thread Peter Eisentraut
On 12.11.24 17:02, Robert Haas wrote: On Wed, Nov 6, 2024 at 11:11 AM Peter Eisentraut wrote: I don't have Windows handy to test it out, but looking at the respective build system source files, in master, pgevent is built and installed like a normal shared library in both meson.build and Makefi

Re: Virtual generated columns

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-12, Peter Eisentraut wrote: > On 12.11.24 09:49, jian he wrote: > > > On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut > > > wrote: > > check_modified_virtual_generated, we can replace fastgetattr to > > heap_attisnull? like: > > // boolisnull; > >

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Jan Wieck
Hello, On 11/12/24 10:34, Andres Freund wrote: I have working code - pretty ugly at this state, but mostly needs a fair bit of elbow grease not divine inspiration... It's not a trivial change, but entirely doable. The short summary of how it works is that it uses a single 64bit atomic that is

Re: Extract constants from EXECUTE queries

2024-11-12 Thread Tomas Vondra
On 11/12/24 12:28, ISHAN CHHANGANI wrote: > Hi hackers, I wanted to extract constants hard coded within prepared > statements. > > ex:- > >   > > PREPARE stmt(text, int) as SELECT * FROM test_table WHERE a = $1 AND b = > 99 AND c = $2; > > EXECUTE stmt('abc', 1); > >   > > I can easily get th

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Ranier Vilela
Em ter., 12 de nov. de 2024 às 07:56, Bertrand Drouvot < bertranddrouvot...@gmail.com> escreveu: > Hi, > > On Tue, Nov 12, 2024 at 03:56:13PM +0900, Michael Paquier wrote: > > On Tue, Nov 12, 2024 at 06:09:04AM +, Bertrand Drouvot wrote: > > > I think that the 64b len check done in v11 is mand

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 10.11.24 04:16, Amit Kapila wrote: The possible idea to replicate virtual generated columns is to compute the corresponding expression before sending the data to the client. If we can allow it in the row filter than why not to publish it as well. Row filters have pretty strong restrictions f

Extract constants from EXECUTE queries

2024-11-12 Thread ISHAN CHHANGANI
Hi hackers, I wanted to extract constants hard coded within prepared statements. ex:- PREPARE stmt(text, int) as SELECT * FROM test_table WHERE a = $1 AND b = 99 AND c = $2; EXECUTE stmt('abc', 1); I can easily get the parameter values ('abc' and 1) from queryDesc->params, but I need to also ex

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 11.11.24 06:51, vignesh C wrote: The patch needs to be rebased due to a recent commit 14e87ffa5c5. done in v9 I have verified the behavior of logical replication of row filters on the virtual generated column, and everything appears to be functioning as expected. One suggestion would be to

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 12.11.24 09:49, jian he wrote: On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: RelationBuildPartitionKey if (!isnull) { char *exprString; Node *expr; exprString = TextDatumGetCString(datum); expr = stringToNode(exprString); pfr

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 11.11.24 12:37, jian he wrote: On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: New patch version. I've gone through the whole thread again and looked at all the feedback and various bug reports and test cases and made sure they are all addressed in the latest patch version. (I'll

Re: Virtual generated columns

2024-11-12 Thread Peter Eisentraut
On 07.11.24 11:02, Dean Rasheed wrote: On Tue, 5 Nov 2024 at 16:17, Peter Eisentraut wrote: New patch version. In expand_generated_columns_in_expr(): +/* + * XXX For the benefit of triggers, make two passes, so it covers + * PRS2_OLD_VARNO and PRS2_NEW_VARNO. +

Re: doc: pgevent.dll location

2024-11-12 Thread Robert Haas
On Wed, Nov 6, 2024 at 11:11 AM Peter Eisentraut wrote: > I don't have Windows handy to test it out, but looking at the respective > build system source files, in master, pgevent is built and installed > like a normal shared library in both meson.build and Makefile, so it > should end up somewhere

Re: Avoiding superfluous buffer locking during nbtree backwards scans

2024-11-12 Thread Peter Geoghegan
On Sun, Nov 10, 2024 at 11:36 PM Masahiro Ikeda wrote: > Thanks! The change made it easier for me to understand. As follow-up to all of the recent work in this area, I'd like to add this wrapper function to return the next item from so->currPos. The wrapper function has extra assertions, compare

Re: [PATCH] Add sortsupport for range types and btree_gist

2024-11-12 Thread Bernd Helmle
Am Montag, dem 11.11.2024 um 23:03 +0500 schrieb Andrey M. Borodin: > Some nitpicking: > 0. postgres % git apply ~/Downloads/v7.3-Add-GIST-sortsupport-* > /Users/x4mmm/Downloads/v7.3-Add-GIST-sortsupport-btree-gist.patch:19: > space before tab in indent. I obviously shouldn't do patches after a lo

Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE

2024-11-12 Thread Guillaume Lelarge
Le mar. 12 nov. 2024 à 16:21, Robert Haas a écrit : > On Mon, Nov 11, 2024 at 3:59 PM Guillaume Lelarge > wrote: > > Agreed. Having an "EXPLAIN (ALL)" would be a great addition. I could > tell a customer to do an "EXPLAIN (ALL)", rather than first asking the > PostgreSQL release installed on the

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Andres Freund
Hi, On 2024-11-12 10:12:40 -0500, Jan Wieck wrote: > On 11/12/24 08:55, Andres Freund wrote: > > Hi, > > > > On 2024-11-12 08:51:49 -0500, Jan Wieck wrote: > > > On 11/11/24 23:21, Amit Kapila wrote: > > > > As the inversion issue can mainly hamper logical replication-based > > > > solutions we ca

Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE

2024-11-12 Thread Robert Haas
On Mon, Nov 11, 2024 at 3:59 PM Guillaume Lelarge wrote: > Agreed. Having an "EXPLAIN (ALL)" would be a great addition. I could tell a > customer to do an "EXPLAIN (ALL)", rather than first asking the PostgreSQL > release installed on the server and after that, giving the correct options > for

Re: Parallel workers stats in pg_stat_database

2024-11-12 Thread Michael Banck
Hi, On Tue, Nov 12, 2024 at 03:56:11PM +0100, Benoit Lobréau wrote: > On 11/12/24 15:05, Michael Banck wrote: > > I was wondering about the weird new column name workers_to_launch when I > > read the commit message - AFAICT this has been an internal term so far, > > and this is the first time we e

Re: meson and check-tests

2024-11-12 Thread jian he
On Mon, Nov 11, 2024 at 4:14 PM Nazir Bilal Yavuz wrote: > > Hi, > > Thanks for the feedback! > > On Mon, 11 Nov 2024 at 06:34, jian he wrote: > > > > hi. > > Actually, there is a difference compared to make. > > make works fine with many whitespaces. but meson you can only have one > > whitespac

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Jan Wieck
Hello, On 11/12/24 08:55, Andres Freund wrote: Hi, On 2024-11-12 08:51:49 -0500, Jan Wieck wrote: On 11/11/24 23:21, Amit Kapila wrote: > As the inversion issue can mainly hamper logical replication-based > solutions we can do any of this additional work under spinlock only > when the current

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2024-11-12 Thread Michail Nikolaev
Hello, everyone! With winter approaching, it’s the perfect time to dive back into work on this patch! :) The first attached patch implements Matthias's idea of periodically resetting the snapshot during the initial heap scan. The next step will be to add support for parallel builds. Additionally

Re: [PoC] XMLCast (SQL/XML X025)

2024-11-12 Thread Robert Haas
Hi Jim, On Mon, Nov 11, 2024 at 2:43 PM Jim Jones wrote: > > The only thing I found during a quick perusal of the documentation was > > XMLTABLE(), which seems a bit baroque if you just want to convert one > > value. Is this intended to plug that gap? Is there any other current > > way of doing i

Re: Parallel workers stats in pg_stat_database

2024-11-12 Thread Benoit Lobréau
On 11/12/24 15:05, Michael Banck wrote: I was wondering about the weird new column name workers_to_launch when I read the commit message - AFAICT this has been an internal term so far, and this is the first time we expose it to users? I personally find (parallel_)workers_planned/launched clearer

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Jan Wieck
On 11/11/24 23:22, Amit Kapila wrote: On Mon, Nov 11, 2024 at 9:05 PM Tomas Vondra wrote: Alternatively, we could simply stop relying on the timestamps recorded in the commit, and instead derive "monotonic" commit timestamps after the fact. For example, we could track timestamps for some subse

Re: RFC: Additional Directory for Extensions

2024-11-12 Thread David E. Wheeler
On Nov 12, 2024, at 08:25, Peter Eisentraut wrote: > No, you can also install them into a common directory and mount that one. > For example, you install the extension at build time into > /tmp/foo/{lib,share/extension}, you package that up as a disk image, mount it > at /opt/extensions/myext

Re: [PATCH] Refactor SLRU to always use long file names

2024-11-12 Thread Aleksander Alekseev
Hi again, Just a quick follow-up. > (*) BTW I noticed a mistake in the commented code. The condition > should be `>=`, not `<`, i.e: > > ``` > if(new_cluster.controldata.cat_ver >= SLRU_SEG_FILENAMES_CHANGE_CAT_VER) > return; > ``` The concentration of caffeine in my blood is a bit l

Re: Allowing pg_recvlogical to create temporary replication slots

2024-11-12 Thread Dickson S. Guedes
Em ter., 12 de nov. de 2024 às 07:54, Torsten Förtsch escreveu: > Hi Dickson, > > [ ... ] > > If the slot existed before, the existing slot will be used, no error is > generated. In that case the slot will keep existing when the connection is > dropped. > > The following situation can happen: >

Re: [PATCH] Refactor SLRU to always use long file names

2024-11-12 Thread Aleksander Alekseev
Hi Michael, Thanks for your feedback! > Your patch is just doing a rename() of the files from short to long > names. How about adding a new TAP script in pg_upgrade that creates a > couple of empty files with short files names in each path that needs > to do the transfer? Then the test could ru

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Andres Freund
Hi, On 2024-11-11 09:49:19 +, Hayato Kuroda (Fujitsu) wrote: > I've done performance tests compared with master vs. v2 patch. > It showed that for small transactions cases, the performance difference was > 0-2%, > which was almost the same of the run-by-run variation. > > We may completely c

Re: Parallel workers stats in pg_stat_database

2024-11-12 Thread Michael Banck
Hi, On Fri, Oct 11, 2024 at 09:33:48AM +0200, Guillaume Lelarge wrote: > FWIW, with the recent commits of the pg_stat_statements patch, you need a > slight change in the patch I sent on this thread. You'll find a patch > attached to do that. You need to apply it after a rebase to master. > > -

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Andres Freund
Hi, On 2024-11-12 08:51:49 -0500, Jan Wieck wrote: > On 11/11/24 23:21, Amit Kapila wrote: > > As the inversion issue can mainly hamper logical replication-based > > solutions we can do any of this additional work under spinlock only > > when the current record is a commit record (which the curren

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Jan Wieck
On 11/11/24 23:21, Amit Kapila wrote: As the inversion issue can mainly hamper logical replication-based solutions we can do any of this additional work under spinlock only when the current record is a commit record (which the currently proposed patch is already doing) and "wal_level = logical" a

Re: Parametrization minimum password lenght

2024-11-12 Thread Tomas Vondra
Hi, On 11/12/24 14:41, Emanuele Musella wrote: > The goal about this patch is to parameterize the minimum password lenght > on users database and apply it on the general code. > The patch is applicable to the master branch. > We already tested it: it build and works as expected and nothing is > fo

Parametrization minimum password lenght

2024-11-12 Thread Emanuele Musella
The goal about this patch is to parameterize the minimum password lenght on users database and apply it on the general code. The patch is applicable to the master branch. We already tested it: it build and works as expected and nothing is found broken, Settings in postgresql.conf parametrization l

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-12, Amit Kapila wrote: > I think we still need a fix for the master for the case when generated > columns are not published but are part of REPLICA IDENTITY as that > could lead to failures in applying UPDATE and DELETE on subscriber. Ah, I thought that was already in place. > Am, I

Re: RFC: Additional Directory for Extensions

2024-11-12 Thread Peter Eisentraut
On 11.11.24 19:15, David E. Wheeler wrote: In fact, most of the patch is refactoring the routines in dfmgr.c to not hardcode dynamic_library_path but allow searching for any file in any path. Once a control file is found, the other extension support files (script files and auxiliary control f

Re: logical replication: restart_lsn can go backwards (and more), seems broken since 9.4

2024-11-12 Thread Tomas Vondra
On 11/12/24 13:19, Amit Kapila wrote: > On Tue, Nov 12, 2024 at 4:55 PM Tomas Vondra wrote: >> >> There's also the question of backpatching - the simpler the better, and >> this I think just resetting the fields wins in this regard. The main >> question is whether it's correct - I think it is.

CREATE SCHEMA ... CREATE DOMAIN support

2024-11-12 Thread Kirill Reshke
Hi hackers! This copy of my reply mail from pgsql-general[0], & [1] which was held for moderation for some reason. Here it goes as-is : == begin Hi Álvaro, thanks for the detailed explanation. So, IIUC you are suggesting to support SQL standard features before any work with PostgreSQL extension

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-12, Amit Kapila wrote: > On Tue, Nov 12, 2024 at 2:15 PM Alvaro Herrera > wrote: > > So, another option is to do nothing for stable branches. > > Fair enough. The other point in favor of that option is that nobody > has reported this problem yet but my guess is that they would have

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-11-12 Thread Amit Kapila
On Tue, Nov 12, 2024 at 5:37 PM Alvaro Herrera wrote: > > On 2024-Nov-12, Amit Kapila wrote: > > > > > It's not clear to me why doesn't pgoutput cope with generated columns in > > > replica identities. Maybe that can be reconsidered? > > > > In stable branches, we intentionally skip publishing ge

Re: logical replication: restart_lsn can go backwards (and more), seems broken since 9.4

2024-11-12 Thread Amit Kapila
On Tue, Nov 12, 2024 at 4:55 PM Tomas Vondra wrote: > > There's also the question of backpatching - the simpler the better, and > this I think just resetting the fields wins in this regard. The main > question is whether it's correct - I think it is. I'm not too worried > about efficiency very muc

Re: logical replication: restart_lsn can go backwards (and more), seems broken since 9.4

2024-11-12 Thread Ashutosh Bapat
On Tue, Nov 12, 2024 at 4:55 PM Tomas Vondra wrote: > > > A candidate_restart_lsn and candidate_restart_valid pair just tells > > that we may set slot's data.restart_lsn to candidate_restart_lsn when > > the downstream confirms an LSN >= candidate_restart_valid. That pair > > can never get inaccu

Re: Commit Timestamp and LSN Inversion issue

2024-11-12 Thread Tomas Vondra
On 11/12/24 04:05, Amit Kapila wrote: > On Mon, Nov 11, 2024 at 9:05 PM Tomas Vondra wrote: >> >> On 11/11/24 09:19, Amit Kapila wrote: >>> >>> I can't think of a solution other than the current proposal where we >>> do both the operations (reserve WAL space for commit and adjust >>> commit_timest

Re: logical replication: restart_lsn can go backwards (and more), seems broken since 9.4

2024-11-12 Thread Tomas Vondra
On 11/12/24 10:37, Ashutosh Bapat wrote: > On Tue, Nov 12, 2024 at 4:54 AM Tomas Vondra wrote: >> >> >> >> On 11/11/24 23:41, Masahiko Sawada wrote: >>> On Mon, Nov 11, 2024 at 6:17 AM Tomas Vondra wrote: >>> >>> Which made me re-investigate the issue and thought that it doesn't >>> necessaril

Re: Parallel heap vacuum

2024-11-12 Thread vignesh C
On Wed, 30 Oct 2024 at 22:48, Masahiko Sawada wrote: > > > I've attached new version patches that fixes failures reported by > cfbot. I hope these changes make cfbot happy. > I just started reviewing the patch and found the following comments while going through the patch: 1) I felt we should add

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-09, Amit Kapila wrote: > On Fri, Nov 8, 2024 at 5:17 PM Alvaro Herrera wrote: > > > > On 2024-Nov-07, Amit Kapila wrote: > > > > > BTW, I was thinking as to how to fix it on back branches and it seems > > > we should restrict to define REPLICA IDENTITY on stored generated > > > column

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Bertrand Drouvot
Hi, On Tue, Nov 12, 2024 at 03:56:13PM +0900, Michael Paquier wrote: > On Tue, Nov 12, 2024 at 06:09:04AM +, Bertrand Drouvot wrote: > > I think that the 64b len check done in v11 is mandatory for safety reasons. > > > > The loop above reads 64 bytes at once, so would read beyond the memory a

Re: Allowing pg_recvlogical to create temporary replication slots

2024-11-12 Thread Torsten Förtsch
Hi Dickson, > > While reviewing your patch for pg_recvlogical, I’m curious about the > expected behavior when using --if-not-exists in conjunction with > --temporary-slot. > > If an existing slot is reused under these options, will it behave as > a temporary slot and be removed when the connection

Re: explain plans for foreign servers

2024-11-12 Thread Ashutosh Bapat
On Mon, Nov 11, 2024 at 9:12 PM dinesh salve wrote: > > > Hi Hackers, > > I am working on a feature in postgres_fdw extension to show plans used by > remote postgresql servers in the output of the EXPLAIN command. > I think this will help end users understand query execution plans used by > remo

Re: not null constraints, again

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-09, Alvaro Herrera wrote: > I notice I cargo-culted a "free de-toasted copy", but I think it's > impossible to end up with a toasted datum here, because the column is > guaranteed to have only one element, so not a candidate for toasting. > But also, if we don't free it (in case somebo

Re: Add html-serve target to autotools and meson

2024-11-12 Thread Ashutosh Bapat
On Mon, Nov 11, 2024 at 10:41 PM Jacob Champion wrote: > > On Mon, Nov 11, 2024 at 7:50 AM Andres Freund wrote: > > If you just want that we could print out a file:// url or even open it? > > Rather > > than an http server? > > +1. The coverage-html recipe is a nice example of this; just > contr

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-11-12 Thread Amit Kapila
On Tue, Nov 12, 2024 at 2:15 PM Alvaro Herrera wrote: > > On 2024-Nov-09, Amit Kapila wrote: > > > On Fri, Nov 8, 2024 at 5:17 PM Alvaro Herrera > > wrote: > > > > > > On 2024-Nov-07, Amit Kapila wrote: > > > > > > > BTW, I was thinking as to how to fix it on back branches and it seems > > > > w

Re: Separate memory contexts for relcache and catcache

2024-11-12 Thread Ashutosh Bapat
On Tue, Nov 12, 2024 at 2:57 AM Jeff Davis wrote: > > On Mon, 2024-11-11 at 17:05 +0530, Ashutosh Bapat wrote: > > It will be good > > to move ahead with the ones we all agree for now. Looking at all the > > emails, those will be CatCacheContext, > > RelCacheContext, PlanCacheContext, TypCacheCont

Re: NOT ENFORCED constraint feature

2024-11-12 Thread Peter Eisentraut
I started reviewing patch 0001 for check constraints. I think it's a good idea how you structured it so that we can start with this relatively simple feature and get all the syntax parsing etc. right. I also looked over the remaining patches a bit. The general structure looks right to me. But

Re: Identify huge pages accessibility using madvise

2024-11-12 Thread Peter Eisentraut
On 08.11.24 09:54, Dmitry Dolgov wrote: Looks like there is a plot twist. After talking to Gabriele off list and testing on an EKS, I've discovered that since 5.7 Linux kernel supports hugetlb reservation via hugetlbfs [1]. That means that together with the original limitation at page fault time

Fix to increment the index scan counter for the bloom filter index

2024-11-12 Thread Masahiro Ikeda
Hi, I noticed that the bloom filter index forgets to increment the index scan counter while reviewing the skip scan patch [1]. It seems this was simply overlooked in the implementation. What do you think? # A test case: ## HEAD: 3f323eba89fb CREATE EXTENSION bloom ; CREATE TABLE tbloom AS

Re: Update Unicode data to Unicode 16.0.0

2024-11-12 Thread Laurenz Albe
On Mon, 2024-11-11 at 14:52 -0500, Joe Conway wrote: > On 11/11/24 01:27, Peter Eisentraut wrote: > > Here is the patch to update the Unicode data to version 16.0.0. > > > > Normally, this would have been routine, but a few months ago there was > > some debate about how this should be handled. [0]

Re: logical replication: restart_lsn can go backwards (and more), seems broken since 9.4

2024-11-12 Thread Ashutosh Bapat
On Tue, Nov 12, 2024 at 4:54 AM Tomas Vondra wrote: > > > > On 11/11/24 23:41, Masahiko Sawada wrote: > > On Mon, Nov 11, 2024 at 6:17 AM Tomas Vondra wrote: > > > > Which made me re-investigate the issue and thought that it doesn't > > necessarily need to clear these candidate values in memory o

Re: Support LIKE with nondeterministic collations

2024-11-12 Thread Peter Eisentraut
On 11.11.24 14:25, Heikki Linnakangas wrote: Sadly the algorithm is O(n^2) with non-deterministic collations.Is there any way this could be optimized? We make no claims on how expensive any functions or operators are, so I suppose a slow implementation is nevertheless better than throwing an er

Re: Virtual generated columns

2024-11-12 Thread jian he
> On Wed, Nov 6, 2024 at 12:17 AM Peter Eisentraut wrote: > > > > New patch version. I've gone through the whole thread again and looked > > at all the feedback and various bug reports and test cases and made sure > > they are all addressed in the latest patch version. (I'll send some > > separa

Re: doc fail about ALTER TABLE ATTACH re. NO INHERIT

2024-11-12 Thread Alvaro Herrera
On 2024-Nov-11, Amit Langote wrote: > The restriction also exists in the CREATE TABLE PARTITION OF path: > > [...] > > I think we can remove the restriction at least for the leaf partition > case just like in the ATTACH PARTITION path. Sure, WFM. -- Álvaro Herrera 48°01'N 7°57'E

  1   2   >