Re: allowing extensions to control planner behavior

On Sat, Oct 19, 2024 at 6:00 AM Andrei Lepikhov wrote: > Generally, a hash value doesn't 100% guarantee the uniqueness of a node > identification. Also, RelOptInfo corresponds to a subtree in the final > plan, and sometimes, it takes work to find which node in the partially > executed plan corresp

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

On Tue, 22 Oct 2024 16:32:33 +0200 Alvaro Herrera wrote: > On 2024-Oct-22, Jehan-Guillaume de Rorthais wrote: > > > On Fri, 18 Oct 2024 16:50:59 +0200 > > Alvaro Herrera wrote: > > > > For branch 14, I opted to make it delete the constraints on detach. > > > This isn't ideal but unless someb

Remove unnecessary word in a comment

Hi, I think we should apply the attached which does this: --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -758,7 +758,7 @@ ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos) * ExecGetRangeTableRelation * Open the Relation for a range ta

Re: Set query_id for query contained in utility statement

On Wed, Oct 23, 2024 at 2:10 PM Michael Paquier wrote: > > On Tue, Oct 22, 2024 at 11:34:55AM +0200, Anthonin Bonnefoy wrote: > > On Tue, Oct 22, 2024 at 7:06 AM Michael Paquier wrote: > >> Something that > >> worries me a bit is that this changes makes the code less clean, by > >> having a SELEC

Re: Pgoutput not capturing the generated columns

On Wed, Oct 23, 2024 at 12:26 PM Peter Smith wrote: > > On Wed, Oct 23, 2024 at 5:21 PM Amit Kapila wrote: > > > Additional comment on the 0003 patch > > +# > > = > > +# Misc test. > > +# > > +# A "normal -> generated" r

Re: Set query_id for query contained in utility statement

On Wed, Oct 23, 2024 at 8:10 AM Michael Paquier wrote: > > I have some more minor comments. > > -if (@$ < 0) /* see comments for YYLLOC_DEFAULT */ > -@$ = @2; > > With 14e5680eee19 now in the tree (interesting timing as this did not > exist until yesterday), it looks li

Re: Make default subscription streaming option as Parallel

On Wed, Oct 23, 2024 at 1:21 AM Masahiko Sawada wrote: > > On Tue, Oct 22, 2024 at 2:17 AM Amit Kapila wrote: > > > > On Mon, Oct 21, 2024 at 8:40 PM vignesh C wrote: > > > > > > Attached v3 version patch has a caution added for the same. > > > > > > > Thanks, the patch looks good to me and I am

Re: allowing extensions to control planner behavior

On 10/23/24 15:05, Robert Haas wrote: On Sat, Oct 19, 2024 at 6:00 AM Andrei Lepikhov wrote: Generally, a hash value doesn't 100% guarantee the uniqueness of a node identification. Also, RelOptInfo corresponds to a subtree in the final plan, and sometimes, it takes work to find which node in th

Re: Remove unnecessary word in a comment

On Wed, Oct 23, 2024 at 5:29 PM Amit Langote wrote: > I think we should apply the attached which does this: > > --- a/src/backend/executor/execUtils.c > +++ b/src/backend/executor/execUtils.c > @@ -758,7 +758,7 @@ ExecInitRangeTable(EState *estate, List > *rangeTable, List *permInfos) > * ExecGe

Refine comments on usage WL_POSTMASTER_DEATH vs WL_EXIT_ON_PM_DEATH

Hi, Hackers! Current comments on the usage of WL_POSTMASTER_DEATH state that it should be used for scenarios of finishing other than immediately i.e. returning values and waiting for postmaster dies. In fact, in parts of the code, it's currently used to immediately exit or throw FATAL (in the wals

Re: Refactor to use common function 'get_publications_str'.

On Wed, Oct 23, 2024 at 5:23 PM Michael Paquier wrote: > > On Wed, Oct 23, 2024 at 03:44:03PM +1100, Peter Smith wrote: > > During a code review, it was noticed that there are several places > > within logical replication where a comma-separated list of publication > > names is built explicitly. T

Re: allowing extensions to control planner behavior

On Wed, Oct 23, 2024 at 11:51 AM Andrei Lepikhov wrote: > Parse tree may be implemented with multiple execution plans. Even > clauses can be transformed during optimisation (Remember OR -> ANY). > Also, the cardinality of a middle-tree join depends on the inner and > outer subtrees. Because of tha

Proposal for Integrating Data Masking and anonymization into PostgreSQL

Dear PostgreSQL Development Team, I am writing to propose the development of native data masking and anonymization features within PostgreSQL. As a long-time user of PostgreSQL, I have observed a growing need for efficient and secure data handling, particularly in compliance with regulations like

Re: msvc directory missing in PostgreSQL 17.0

Check out this: https://wiki.postgresql.org/wiki/Meson There's a table showing the setup & build commands using the old way and the new way (meson). > On Oct 21, 2024, at 4:12 PM, Mark Hill wrote: > > Thanks Bill! Do you have a sample meson command for building that you could > share? > >

Re: Using read_stream in index vacuum

On Wed, Oct 23, 2024 at 9:32 AM Andrey M. Borodin wrote: > > > On 22 Oct 2024, at 16:42, Melanie Plageman > > wrote: > > > > Ah, right, the callback might return InvalidBlockNumber far before > > we've actually read (and vacuumed) the blocks it is specifying. > > I've discussed the issue with Th

Re: use a non-locking initial test in TAS_SPIN on AArch64

On Wed, Oct 23, 2024 at 09:46:56AM -0500, Nathan Bossart wrote: > I have a couple of Apple M-series machines I can test, too. After some preliminary tests on an M3, I'm not seeing any gains outside the noise range. That's not too surprising because it's likely more difficult to create a lot of sp

Re: Using Expanded Objects other than Arrays from plpgsql

Michel Pelletier writes: > Here's another example: > CREATE OR REPLACE FUNCTION test2(graph matrix) > RETURNS bigint LANGUAGE plpgsql AS > $$ > BEGIN > perform set_element(graph, 1, 1, 1); > RETURN nvals(graph); > end; > $$; > CREATE FUNCTION > postgres=# select test2(

Wrong result when enable_partitionwise_join is on if collation of PartitionKey and Column is different.

Hi all, I find another issue as $SUBJECT when I work on [1]. I will use [1] SQL to show the problem. CREATE COLLATION case_insensitive ( provider = icu, locale = 'und-u-ks-level2', deterministic = false ); CREATE TABLE pagg_tab (c text collate case_insensitive) PARTITION BY LIST(c collate "C"); CR

Re: Using Expanded Objects other than Arrays from plpgsql

I wrote: > One idea I was toying with is that it doesn't matter if f() > throws an error so long as the plpgsql function is not executing > within an exception block: if the error propagates out of the plpgsql > function then we no longer care about the value of the variable. > That would very subs

Re: Using read_stream in index vacuum

On Wed, Oct 23, 2024 at 9:32 PM Andrey M. Borodin wrote: > > > > > On 22 Oct 2024, at 16:42, Melanie Plageman > > wrote: > > > > Ah, right, the callback might return InvalidBlockNumber far before > > we've actually read (and vacuumed) the blocks it is specifying. > > I've discussed the issue wit

Re: use a non-locking initial test in TAS_SPIN on AArch64

On Wed, Oct 23, 2024 at 11:01:05AM +0800, Jingtang Zhang wrote: > The result looks great, but the discussion in [0] shows that the result may > vary among different ARM chips. Could you provide the chip model of this > test? So that we can do a cross validation of this patch. This is on a c8g.24xl

Re: Inconsistent use of relpages = -1

Jeff Davis writes: > It bothers me somewhat that views still have relpages=0, because they > also don't have storage. Thoughts? I think it's fine, because they also have reltuples=0. The combination of those two zeroes indicates ignorance as to the rel's contents. If either field is not zero, t

Re: Proposal for Integrating Data Masking and anonymization into PostgreSQL

Hi Hosney On Wed, 23 Oct 2024, 15:17 Hosney Osman, wrote: > Dear PostgreSQL Development Team, > > I am writing to propose the development of native data masking and > anonymization features within PostgreSQL. As a long-time user of > PostgreSQL, > > I have observed a growing need for efficient a

Re: Unexpected table size usage for small composite arrays

Thank you both for explaining this! Tom, your statement sounds like a good solution: " In principle we could squeeze those out and store them only once per array, but we don't." On Tue, Oct 22, 2024 at 8:44 PM Tom Lane wrote: > "David G. Johnston" writes: > > On Tue, Oct 22, 2024 at 4:40 PM E

Re: Using read_stream in index vacuum

> On 22 Oct 2024, at 16:42, Melanie Plageman wrote: > > Ah, right, the callback might return InvalidBlockNumber far before > we've actually read (and vacuumed) the blocks it is specifying. I've discussed the issue with Thomas on PGConf.eu and he proposed to use stream reset. PFA v3. Best re

Re: general purpose array_sort

Sorry for the late reply. On Mon, Oct 14, 2024 at 4:10 AM Tom Lane wrote: > > Junwang Zhao writes: > > It seems there is not much meaning to sort composite type, > > so are you proposing we should error on that? > > It's hardly "general purpose" if it randomly refuses to > sort certain types. I

Re: Using read_stream in index vacuum

Thanks! > On 23 Oct 2024, at 18:17, Melanie Plageman wrote: > > > Note that you don't check if buf is valid here and break out of the > inner loop when it is invalid. I've added two asserts to clarify expectations. > > By doing read_stream_reset() before you first invoke > read_stream_next_b

Re: Statistics Import and Export

On Tue, 2024-10-22 at 23:58 +, Shinoda, Noriyoshi (SXD Japan FSIP) wrote: > Thanks for developing good features. I tried the patch that was > committed right away. > It seems that the implementation and documentation differ on the > return value of the pg_clear_attribute_stats function. > The a

Re: Inconsistent use of relpages = -1

On Wed, 2024-10-23 at 04:47 +0200, Laurenz Albe wrote: > On Tue, 2024-10-22 at 10:41 -0700, Jeff Davis wrote: > > I attached a patch that creates partitioned tables with relpages=- > > 1, > > and updates the docs. > > Does this need any changes for pg_upgrade? pg_upgrade would go through the same

Re: Refine comments on usage WL_POSTMASTER_DEATH vs WL_EXIT_ON_PM_DEATH

Hi, Heikki! On Wed, 23 Oct 2024 at 21:00, Heikki Linnakangas wrote: > On 23/10/2024 12:18, Pavel Borisov wrote: > > Hi, Hackers! > > > > Current comments on the usage of WL_POSTMASTER_DEATH state that it > > should be used for scenarios of finishing other than immediately i.e. > > returning va

Re: Refactor to use common function 'get_publications_str'.

On Thu, Oct 24, 2024 at 8:26 AM Masahiko Sawada wrote: Thanks for your feedback! > > While the changes look good to me, the comment of GetPublicationsStr() > seems not match what the function actually does: > > +/* > + * Add publication names from the list to a string. > + */ > +void > +GetPubli

Re: Refactor to use common function 'get_publications_str'.

On Wed, Oct 23, 2024 at 3:26 PM Peter Smith wrote: > > On Thu, Oct 24, 2024 at 8:26 AM Masahiko Sawada wrote: > > Thanks for your feedback! > > > > > While the changes look good to me, the comment of GetPublicationsStr() > > seems not match what the function actually does: > > > > +/* > > + * Add

Re: Set query_id for query contained in utility statement

On Wed, Oct 23, 2024 at 04:36:42PM +0800, jian he wrote: > I am not sure of the meaning of "@$", though. Please feel free to look at the upstream docs about that: https://www.gnu.org/software/bison/manual/bison.html#Locations "the location of the whole grouping is @$". -- Michael signature.asc D

Re: [PoC] Federated Authn/z with OAUTHBEARER

On Thu, Oct 17, 2024 at 10:51 PM Antonin Houska wrote: > This is the 1st round, based on reading the code. I'll continue paying > attention to the project and possibly post some more comments in the future. Thanks again for the reviews! > * Information on the new method should be added to pg_hba

Re: Trigger more frequent autovacuums of heavy insert tables

On Tue, Oct 22, 2024 at 3:12 PM Melanie Plageman wrote: > > The attached patch does this. I realized that I broke relation_statistics_update(). Attached v2 is fixed. > I've estimated the unfrozen percentage of the table by adding a new > field to pg_class, relallfrozen, which is updated in the s

Can rs_cindex be < 0 for bitmap heap scans?

Hi, HeapScanDescData->rs_cindex (the current index into the array of visible tuples stored in the heap scan descriptor) is used for multiple scan types, but for bitmap heap scans, AFAICT, it should never be < 0. As such, I find this test in heapam_scan_bitmap_next_tuple() pretty confusing. if (

Re: DOCS - pg_replication_slot . Fix the 'inactive_since' description

Hi. Here are a couple of minor comments. 1. +The time slot synchronization (see ) +was most recently stopped. /The time slot/The time when slot/ ~~~ 2. - /* The time since the slot has become inactive */ + /* The time slot sychronized was stopped. */ Maybe just make this comme

Re: Refine comments on usage WL_POSTMASTER_DEATH vs WL_EXIT_ON_PM_DEATH

On 23/10/2024 20:29, Pavel Borisov wrote: Hi, Heikki! On Wed, 23 Oct 2024 at 21:00, Heikki Linnakangas > wrote: On 23/10/2024 12:18, Pavel Borisov wrote: > Hi, Hackers! > > Current comments on the usage of WL_POSTMASTER_DEATH state that it > sh

Re: Refine comments on usage WL_POSTMASTER_DEATH vs WL_EXIT_ON_PM_DEATH

On 23/10/2024 12:18, Pavel Borisov wrote: Hi, Hackers! Current comments on the usage of WL_POSTMASTER_DEATH state that it should be used for scenarios of finishing other than immediately i.e. returning values and waiting for postmaster dies. In fact, in parts of the code, it's currently used t

Re: Fix typo in tidstore.h

On Thu, 24 Oct 2024 at 09:53, Masahiko Sawada wrote: > I've attached the patch to fix a typo in tidstore.h. It's an oversight > in commit f6bef362c. Looks good. David

Re: Using read_stream in index vacuum

On Wed, Oct 23, 2024 at 4:29 PM Andrey M. Borodin wrote: > > > On 23 Oct 2024, at 20:57, Andrey M. Borodin wrote: > > > > I'll think how to restructure flow there... > > OK, I've understood how it should be structured. PFA v5. Sorry for the noise. I think this would be a bit nicer: while (Buffe

Re: Refine comments on usage WL_POSTMASTER_DEATH vs WL_EXIT_ON_PM_DEATH

Heikki Linnakangas writes: > On 23/10/2024 20:29, Pavel Borisov wrote: >> That was exactly my point. Actually the caller should not wait, it could >> do whatever it wants contrary to the existing comments: >>> WL_POSTMASTER_DEATH: Wait for postmaster to die >> I don't insist on this patch, but ex

Re: New "raw" COPY format

Hi, On Sat, Oct 19, 2024 at 8:33 AM Joel Jacobson wrote: > > On Sat, Oct 19, 2024, at 12:13, jian he wrote: > > We already make RAW and can only have one column. > > if RAW has no default delimiter, then COPY FROM a text file will > > become one datum value; > > which makes it looks like importin

Re: Fix typo in tidstore.h

On Wed, Oct 23, 2024 at 3:01 PM David Rowley wrote: > > On Thu, 24 Oct 2024 at 09:53, Masahiko Sawada wrote: > > I've attached the patch to fix a typo in tidstore.h. It's an oversight > > in commit f6bef362c. > > Looks good. Thank you for reviewing the patch. Pushed. Regards, -- Masahiko Sawa

Re: Using read_stream in index vacuum

> On 23 Oct 2024, at 20:57, Andrey M. Borodin wrote: > > I'll think how to restructure flow there... OK, I've understood how it should be structured. PFA v5. Sorry for the noise. Best regards, Andrey Borodin. v5-0001-Prototype-B-tree-vacuum-streamlineing.patch Description: Binary data

Re: Fix for consume_xids advancing XIDs incorrectly

On Tue, Oct 22, 2024 at 11:47 PM Fujii Masao wrote: > > > > On 2024/10/18 14:57, Yushi Ogiwara wrote: > > In conclusion, I agree that: > > > > * Update lastxid with GetTopTransactionId(). > > * consume_xids with 0 should raise an error. > > > > I have attached a new patch that incorporates you

Re: Refine comments on usage WL_POSTMASTER_DEATH vs WL_EXIT_ON_PM_DEATH

On Wednesday, October 23, 2024, Pavel Borisov wrote: > > In fact, in parts of the code, it's currently used to immediately exit or > throw FATAL (in the walsender and in libpq). > Maybe submit a patch to fix these parts of the code that should apparently be using WL_EXIT_ON_PM_DEATH to do so ins

Re: Logical Replication of sequences

On Tue, Oct 8, 2024 at 2:46 AM vignesh C wrote: > > On Fri, 4 Oct 2024 at 15:39, shveta malik wrote: > > > > On Sun, Sep 29, 2024 at 12:34 PM vignesh C wrote: > > > > > > On Thu, 26 Sept 2024 at 11:07, shveta malik > > > wrote: > > > > > > > > On Fri, Sep 20, 2024 at 9:36 AM vignesh C wrote:

Fix typo in tidstore.h

Hi, I've attached the patch to fix a typo in tidstore.h. It's an oversight in commit f6bef362c. Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com v1-0001-Fix-typo-in-tidstore.h.patch Description: Binary data

Re: POC: make mxidoff 64 bits

After a bit of thought, I've realized that to be conservative here is the way to go. We can reuse a maximum of existing logic. I mean, we can remove offset wraparound "error logic" and reuse "warning logic". But set the threshold for "warning logic" to a much higher value. For now, I choose 2^32-1

Re: Refactor to use common function 'get_publications_str'.

On Wed, Oct 23, 2024 at 12:25 AM Peter Smith wrote: > > On Wed, Oct 23, 2024 at 5:23 PM Michael Paquier wrote: > > > > On Wed, Oct 23, 2024 at 03:44:03PM +1100, Peter Smith wrote: > > > During a code review, it was noticed that there are several places > > > within logical replication where a com

Re: Using Expanded Objects other than Arrays from plpgsql

Michel Pelletier writes: > On Wed, Oct 23, 2024 at 9:04 AM Tom Lane wrote: >> For stuff like your vxm() function, that'd be annoying. But functions >> that need that and are willing to deal with the aliasing hazard could >> still provide a prosupport function that promises it's okay. What >> we

Commutation of array SOME/ANY and ALL operators

Assume the following table structure for my examples. CREATE TABLE my_table ( id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY, int_array integer[], text_array text[] ); INSERT INTO my_table (id, int_array, text_array) VALUES (1, '{5,10}', '{"1234%", "%5678"}'), (2, '{8,15}', '{"

Re: Using Expanded Objects other than Arrays from plpgsql

On Wed, Oct 23, 2024 at 9:04 AM Tom Lane wrote: > I wrote: > > One idea I was toying with is that it doesn't matter if f() > > throws an error so long as the plpgsql function is not executing > > within an exception block: if the error propagates out of the plpgsql > > function then we no longer

Re: Refactor GetLockStatusData() by skipping unused backends and groups

Hi, On Wed, Oct 23, 2024 at 01:19:37AM +0900, Fujii Masao wrote: > > I understand that PGPROC entries with pid=0 are typically those not yet > allocated to > any backends. Yes, as you mentioned, prepared transactions also have pid=0. > However, > GetLockStatusData() loops up to ProcGlobal->allP

Re: Wrong result when enable_partitionwise_join is on if collation of PartitionKey and Column is different.

Tender Wang 于2024年10月23日周三 21:48写道： > Hi all, > > > I think the root cause of this thread and [1] are same. We don't use the > Partition Key collation but column's > collation to fill the RelOptInfo partexprs field in > set_baserel_partition_key_exprs(). > If the Partition Key definition is same

Re: Using Expanded Objects other than Arrays from plpgsql

On Wed, Oct 23, 2024 at 8:21 AM Tom Lane wrote: > Michel Pelletier writes: > > Here's another example: > > > CREATE OR REPLACE FUNCTION test2(graph matrix) > > RETURNS bigint LANGUAGE plpgsql AS > > $$ > > BEGIN > > perform set_element(graph, 1, 1, 1); > > RETURN nvals(graph)

Re: Set query_id for query contained in utility statement

On Wed, Oct 23, 2024 at 11:24:11AM +0200, Anthonin Bonnefoy wrote: > This also answers another issue I was wondering about. Should the > child's parsestate inherit the location information when > make_parsestate is called? That would be incorrect since this is used > for sub-statement, pstate shoul

Re: Refactor to use common function 'get_publications_str'.

On Wed, Oct 23, 2024 at 03:40:19PM -0700, Masahiko Sawada wrote: > > Now, I've changed the function comment to: > > /* > > * Add a comma-separated list of publication names to the 'dest' string. > > */ > > Thank you for updating the patch. The patch looks good to me. + /* Build the pub_names

Re: Commutation of array SOME/ANY and ALL operators

Matthew Morrissette Vance writes: > If instead, PostgreSQL could support the commutation of the `SOME/ANY` and > `ALL` operators so that the `ANY(array)` could be on both sides of the > provided operator, it would allow for this kind of searching natively. > Firstly, would a PR that enhanced Post

Re: Missing installation of Kerberos.pm and AdjustUpgrade.pm

On Tue, Jul 09, 2024 at 04:46:23PM +0900, Michael Paquier wrote: > I cannot say much in favor of Kerberos.pm, but AdjustUpgrade.pm could > be really useful IMO when running the main regression test suite in an > out-of-core module for upgrade scenarios. Not to mention that it > includes a set of f

Re: Refactor to use common function 'get_publications_str'.

On Thu, Oct 24, 2024 at 3:17 PM Michael Paquier wrote: > > On Wed, Oct 23, 2024 at 03:40:19PM -0700, Masahiko Sawada wrote: > > > Now, I've changed the function comment to: > > > /* > > > * Add a comma-separated list of publication names to the 'dest' string. > > > */ > > > > Thank you for updat

Re: Refactor to use common function 'get_publications_str'.

On Thu, Oct 24, 2024 at 5:41 PM Michael Paquier wrote: > > On Thu, Oct 24, 2024 at 05:27:25PM +1100, Peter Smith wrote: > > Yes, well spotted -- I was aware of that. Originally I had coded a >= > > PG15 check for that pub_names assignment. e.g. > > > > if (walrcv_server_version(LogRepWorkerWalRcvC

Re: New "raw" COPY format

On Thu, Oct 24, 2024, at 03:54, Masahiko Sawada wrote: > I have one question: > > From the 0001 patch's commit message: > > No behavioral changes are intended; this is a pure refactoring to improve code > clarity and maintainability. > > Does the reorganization of the option validation done by this

Re: Refactor to use common function 'get_publications_str'.

On Thu, Oct 24, 2024 at 05:27:25PM +1100, Peter Smith wrote: > Yes, well spotted -- I was aware of that. Originally I had coded a >= > PG15 check for that pub_names assignment. e.g. > > if (walrcv_server_version(LogRepWorkerWalRcvConn) >= 15) > GetPublicationsStr(MySubscription->publications