RE: Avoid distributing new catalog snapshot again for the transaction being decoded.

2022-11-29 Thread wangw.f...@fujitsu.com
On Sat, Nov 26, 2022 at 19:50 PM Amit Kapila wrote: > On Fri, Nov 25, 2022 at 5:30 PM Ashutosh Bapat > wrote: > > > > Hi Hou, > > Thanks for the patch. With a simple condition, we can eliminate the > > need to queueing snapshot change in the current transaction and then > > applying it. Saves som

Re: [PoC] Reducing planning time when tables have many partitions

2022-11-29 Thread Yuya Watari
Dear Andrey and Thom, Thank you for reviewing and testing the patch. I really apologize for my late response. On Tue, Nov 8, 2022 at 8:31 PM Andrey Lepikhov wrote: > Looking into find_em_for_rel() changes I see that you replaced > if (bms_is_subset(em->em_relids, rel->relids) > with assertion st

Re: Non-decimal integer literals

2022-11-29 Thread Dean Rasheed
On Wed, 23 Nov 2022 at 08:56, David Rowley wrote: > > On Wed, 23 Nov 2022 at 21:54, David Rowley wrote: > > I wonder if you'd be better off with something like: > > > > while (*ptr && isxdigit((unsigned char) *ptr)) > > { > > if (unlikely(tmp & UINT64CONST(0xF0

Re: Reducing power consumption on idle servers

2022-11-29 Thread Simon Riggs
On Mon, 28 Nov 2022 at 23:16, Thomas Munro wrote: > > I found some more comments and some documentation to word-smith very > lightly, and pushed. Thanks > The comments were stray references to the > trigger file. It's > a little confusing because the remaining mechanism also uses a file, > but

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-29 Thread Nikita Malakhov
Hi! I'm working on this issue according to the plan Tom proposed above - >I agree that we can't simply widen varatt_external to use 8 bytes for >the toast ID in all cases. Also, I now get the point about avoiding >use of globally assigned OIDs here: if the counter starts from zero >for each tabl

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-29 Thread Simon Riggs
On Mon, 28 Nov 2022 at 23:40, Nathan Bossart wrote: > > Okay, here is a new patch set. 0004 adds logic to prevent custodian tasks > from delaying shutdown. That all seems good, thanks. The last important point for me is tests, in src/test/modules probably. It might be possible to reuse the fina

Re: Support logical replication of DDLs

2022-11-29 Thread li jie
I will continue to give feedback for this patch. 1. LIKE STORAGE ``` CREATE TABLE ctlt (a text, c text); ALTER TABLE ctlt ALTER COLUMN c SET STORAGE EXTERNAL; CREATE TABLE ctlt_storage (LIKE ctlt INCLUDING STORAGE); ``` postgres=# \d+ ctlt_storage Table "pub

Re: Reducing power consumption on idle servers

2022-11-29 Thread Simon Riggs
On Sun, 20 Nov 2022 at 20:00, Simon Riggs wrote: > > On Thu, 24 Mar 2022 at 16:21, Robert Haas wrote: > > > > On Thu, Mar 24, 2022 at 12:02 PM Simon Riggs > > > > What changes will be acceptable for bgwriter, walwriter and logical > > > worker? > > > > Hmm, I think it would be fine to introduce

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-29 Thread Amit Kapila
On Tue, Nov 29, 2022 at 10:18 AM houzj.f...@fujitsu.com wrote: > > Attach the new version patch which addressed all comments. > Review comments on v53-0001* == 1. Subscription *MySubscription = NULL; -static bool MySubscriptionValid = false; +bool MySubscriptionValid = fa

Re: Patch: Global Unique Index

2022-11-29 Thread Ilya Anfimov
On Fri, Nov 18, 2022 at 12:03:53PM +0300, Sergei Kornilov wrote: > Hello > Do we need new syntax actually? I think that a global unique index can be > created automatically instead of raising an error "unique constraint on > partitioned table must include all partitioning columns" I may suggest

Re: GUC for temporarily disabling event triggers

2022-11-29 Thread Daniel Gustafsson
> On 3 Nov 2022, at 21:47, Daniel Gustafsson wrote: > The patch adds a new GUC, ignore_event_trigger with two option values, 'all' > and 'none' (the login event patch had 'login' as well). The attached v2 fixes a small bug which caused testfailures the CFBot. -- Daniel Gustafsson

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Simon Riggs
On Thu, 17 Nov 2022 at 17:29, Simon Riggs wrote: > (yes, the last line shows x10 performance patched, that is not a typo) New version of patch, now just a one-line patch! Results show it's still a good win for XidInMVCCSnapshot(). -- Simon Riggshttp://www.EnterpriseDB.com/ s

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Amit Kapila
On Mon, Nov 28, 2022 at 11:29 PM Robert Haas wrote: > > On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > > I think we should consider deprecating the pg_stat_bgwriter columns but > > leaving them in place for a few years. New stuff should only be added to > > pg_stat_checkpointer, but we do

Re: Bug in wait time when waiting on nested subtransaction

2022-11-29 Thread Simon Riggs
On Mon, 28 Nov 2022 at 21:10, Robert Haas wrote: > > On Mon, Nov 28, 2022 at 3:01 PM Tom Lane wrote: > > One thing we need to be pretty careful of here is to not break the > > promise of atomic commit. At topmost commit, all subxacts must > > appear committed simultaneously. It's not quite clea

Re: Patch: Global Unique Index

2022-11-29 Thread Vik Fearing
On 11/24/22 19:15, Cary Huang wrote: On Thu, 24 Nov 2022 08:00:59 -0700 Thomas Kellerer wrote --- > Pavel Stehule schrieb am 24.11.2022 um 07:03: > > There are many Oracle users that find global indexes useful despite > > their disadvantages. > > > > I have seen thi

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Chris Travers
Hi; I suppose I must not have been clear in what I am suggesting we do and why. I will try to answer specific points below and then restate what I think the problem is, and what I think should be done about it. On Mon, Nov 28, 2022 at 5:53 PM Robert Haas wrote: > On Sat, Nov 26, 2022 at 4:08

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Tomas Vondra
On 11/29/22 13:49, Simon Riggs wrote: > On Thu, 17 Nov 2022 at 17:29, Simon Riggs > wrote: > >> (yes, the last line shows x10 performance patched, that is not a typo) > > New version of patch, now just a one-line patch! > > Results show it's still a good win for XidInMVCCSnapshot(). > I'm a

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Julien Tachoires
Hi Tomas, Le mar. 29 nov. 2022 à 14:06, Tomas Vondra a écrit : > > On 11/29/22 13:49, Simon Riggs wrote: > > On Thu, 17 Nov 2022 at 17:29, Simon Riggs > > wrote: > > > >> (yes, the last line shows x10 performance patched, that is not a typo) > > > > New version of patch, now just a one-line pat

Re: Logical Replication Custom Column Expression

2022-11-29 Thread Ashutosh Bapat
On Fri, Nov 25, 2022 at 4:13 PM Stavros Koureas wrote: > > Yes, if the property is on the subscription side then it should be applied > for all the tables that the connected publication is exposing. > So if the property is enabled you should be sure that this origin column > exists to all of the

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Chris Travers
On Mon, Nov 28, 2022 at 11:06 PM Peter Geoghegan wrote: > On Mon, Nov 28, 2022 at 1:52 PM Bruce Momjian wrote: > > I think the problem is that we still have bloat with 64-bit XIDs, > > specifically pg_xact and pg_multixact files. Yes, that bloat is less > > serious, but it is still an issue wor

'Flexible "partition pruning" hook' redux?

2022-11-29 Thread Ted Toth
There's an old thread that interests me but which ended without any resolution/solution: https://www.postgresql.org/messageid/CA%2BHiwqGpoKmDg%2BTJdMfoeu3k4VQnxcfBon4fwBRp8ex_CTCsnA%40mail.gmail.com Some of our basic requirements are: 1) All data must be labeled at a specific level (using SELinux

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Drouvot, Bertrand
Hi, On 11/28/22 6:58 PM, Robert Haas wrote: On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: I think we should consider deprecating the pg_stat_bgwriter columns but leaving them in place for a few years. New stuff should only be added to pg_stat_checkpointer, but we don't need to break old

Re: Privileges on PUBLICATION

2022-11-29 Thread Antonin Houska
Antonin Houska wrote: > Peter Eisentraut wrote: > > > On 04.11.22 08:28, Antonin Houska wrote: > > > I thought about the whole concept a bit more and I doubt if the > > > PUBLICATION > > > privilege is the best approach. In particular, the user specified in > > > CREATE > > > SUBSCRIPTION ...

Re: Support logical replication of DDLs

2022-11-29 Thread vignesh C
On Tue, 29 Nov 2022 at 17:51, li jie wrote: > > I will continue to give feedback for this patch. Thanks a lot, that will be very helpful for us. > 1. LIKE STORAGE > ``` > CREATE TABLE ctlt (a text, c text); > ALTER TABLE ctlt ALTER COLUMN c SET STORAGE EXTERNAL; > CREATE TABLE ctlt_storage (LIK

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 02:35:20PM +0100, Chris Travers wrote: > So I think the problem is that PostgreSQL is becoming more and more scalabile, > hardware is becoming more capable, and certain use cases are continuing to > scale up.  Over time, we tend to find ourselves approaching the end of the >

Re: Fix database creation during installchecks for ICU cluster

2022-11-29 Thread Nazir Bilal Yavuz
Hi, Thanks for the patch! On 10/29/22 12:54, Marina Polyakova wrote: 1) The ECPG tests fail because they use the SQL_ASCII encoding [2], the database template0 uses the ICU locale provider and SQL_ASCII is not supported by ICU: $ make -C src/interfaces/ecpg/ installcheck ... =

Re: Support load balancing in libpq

2022-11-29 Thread Jelte Fennema
Attached is a new version with the tests cleaned up a bit (more comments mostly). @Michael, did you have a chance to look at the last version? Because I feel that the patch is pretty much ready for a committer to look at, at this point. v5-0001-Support-load-balancing-in-libpq.patch Description

Re: fixing CREATEROLE

2022-11-29 Thread David G. Johnston
On Tue, Nov 29, 2022 at 12:32 AM wrote: > > Is there any other argument to be made against ADP? > These aren't privileges, they are memberships. The pg_default_acl catalog is also per-data while these settings should be present in a catalog which, like pg_authid, is catalog-wide. This latter p

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Jan Wieck
On 11/29/22 09:46, Bruce Momjian wrote: As far as I know, all our freeze values are focused on avoiding XID wraparound. If XID wraparound is no longer an issue, we might find that our freeze limits can be much higher than they are now. I'd be careful in that direction as the values together w

Re: fixing CREATEROLE

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 2:32 AM wrote: > I propose a slightly different syntax instead: > > ALTER DEFAULT PRIVILEGES GRANT CREATED ROLE TO role_specification WITH ...; > > This, together with the proposal above regarding the grantor, should be > consistent. I think that is more powerful than what

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread SATYANARAYANA NARLAPURAM
On Sun, Nov 27, 2022 at 10:33 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Mon, Nov 28, 2022 at 12:57 AM Andrey Borodin > wrote: > > > > Some funny stuff. If a user tries to cancel a non-replicated transaction > > Azure Postgres will answer: "user requested cancel wh

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Robert Haas
On Mon, Nov 28, 2022 at 4:53 PM Peter Geoghegan wrote: > Imagine if we actually had 64-bit XIDs -- let's assume for a moment > that it's a done deal. This raises a somewhat awkward question: do you > just let the system get further and further behind on freezing, > forever? We can all agree that 2

Re: Collation version tracking for macOS

2022-11-29 Thread Joe Conway
On 11/28/22 14:11, Robert Haas wrote: On Wed, Nov 23, 2022 at 12:09 AM Thomas Munro wrote: OK. Time for a new list of the various models we've discussed so far: 1. search-by-collversion: We introduce no new "library version" concept to COLLATION and DATABASE object and little or no new synt

Re: Patch: Global Unique Index

2022-11-29 Thread Laurenz Albe
On Tue, 2022-11-29 at 13:58 +0100, Vik Fearing wrote: > I disagree.  A user does not need to know that a table is partitionned, > and if the user wants a unique constraint on the table then making them > type an extra word to get it is just annoying. Hmm. But if I created a primary key without

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM wrote: > 2. Process proc die immediately when a backend is waiting for sync > replication acknowledgement, as it does today, however, upon restart, > don't open up for business (don't accept ready-only connections) >

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 8:03 AM Chris Travers wrote: > To be clear, I never suggested shutting down the database. What I have > suggested is that repurposing the current approaching-xid-wraparound warnings > to start complaining loudly when a threshold is exceeded would be helpful. > I think

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread SATYANARAYANA NARLAPURAM
On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: > On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM wrote: > > 2. Process proc die immediately when a backend is waiting for sync > > replication acknowledgement, as it does today, however, upon restart, > > don't o

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 11:41:07AM -0500, Robert Haas wrote: > My argument is that removing xidStopLimit is totally fine, because it > only serves to stop the database. What to do about xidWarnLimit is a > slightly more complex question. Certainly it can't be left untouched, > because warning that

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread SATYANARAYANA NARLAPURAM
On Tue, Nov 29, 2022 at 8:42 AM SATYANARAYANA NARLAPURAM < satyanarlapu...@gmail.com> wrote: > > > On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: > >> On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM wrote: >> > 2. Process proc die immediately when a backend is waiting

Re: Collation version tracking for macOS

2022-11-29 Thread Robert Haas
On Mon, Nov 28, 2022 at 11:49 PM Jeff Davis wrote: > Not necessarily, #2-4 (at least as implemented in v7) can only load one > major version at a time, so can't specify minor versions: > https://www.postgresql.org/message-id/9f8e9b5a3352478d4cf7d6c0a5dd7e82496be4b6.ca...@j-davis.com > > With #1, y

Re: Collation version tracking for macOS

2022-11-29 Thread Jeremy Schneider
On 11/28/22 6:54 PM, Jeff Davis wrote: > > =# select * from pg_icu_collation_versions('en_US') order by > icu_version; > icu_version | uca_version | collator_version > -+-+-- > ... > 67.1| 13.0| 153.14 > 68.2| 13.0| 153.

Re: ExecRTCheckPerms() and many prunable partitions

2022-11-29 Thread Alvaro Herrera
On 2022-Nov-29, Amit Langote wrote: > Maybe, we should have the following there so that the PlannedStmt's > contents don't point into the Query? > > newperminfo = copyObject(perminfo); Hmm, I suppose if we want a separate RTEPermissionInfo node, we should instead do GetRTEPermissionInfo(rte)

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 7:03 AM Jeremy Schneider wrote: > It seems to me that the collator_version field is not a good version > identifier to use. > > Just taking a quick glance at the ICU home page right now, it shows that > all of the last 5 versions of ICU have included "additions and > correc

Re: Fix for visibility check on 14.5 fails on tpcc with high concurrency

2022-11-29 Thread Dimos Stamatakis
[External Email] On 2022-Nov-25, Dimos Stamatakis wrote: > So does this mean there is no race condition in this case and that > this error is redundant? No, it means I believe a bug exists but that I haven't spent enough time on it to understand what it is. Great! Please keep me posted and le

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Tom Lane
Simon Riggs writes: > New version of patch, now just a one-line patch! Of course, there's all the documentation and comments that you falsified. Also, what of the SubTransSetParent call in ProcessTwoPhaseBuffer? (The one in ProcArrayApplyXidAssignment is actually okay, though the comment making

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Andres Freund
Hi, On 2022-11-29 13:30:02 -0500, Tom Lane wrote: > I still wonder whether we'll regret losing information about the > subtransaction tree structure, as discussed in the other thread [1]. > That seems like the main barrier to proceeding with this. Yea, this has me worried. I suspect that we have

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 1:35 PM Andres Freund wrote: > On 2022-11-29 13:30:02 -0500, Tom Lane wrote: > > I still wonder whether we'll regret losing information about the > > subtransaction tree structure, as discussed in the other thread [1]. > > That seems like the main barrier to proceeding with

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 12:32 -0500, Robert Haas wrote: > You know more about this than I do, for sure, so don't let my vote > back the project into a bad spot. I'm going back and forth myself. I haven't found a great answer here yet. > But, yeah, the thing you mention > here is what I'm worried a

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread Andrey Borodin
On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: > > On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM wrote: > > 2. Process proc die immediately when a backend is waiting for sync > > replication acknowledgement, as it does today, however, upon restart, > > don't

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 11:27 -0500, Joe Conway wrote: > My vote is for something like #5. The collversion should indicate a > specific immutable ordering behavior. Easier said than done: https://www.postgresql.org/message-id/abddc35a7a447d93e2b8371a1a9052cb48866070.ca...@j-davis.com Even pointin

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 07:18 +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 7:03 AM Jeremy Schneider > wrote: > > It seems to me that the collator_version field is not a good > > version > > identifier to use. > > > > Just taking a quick glance at the ICU home page right now, it shows > > th

Re: Add non-blocking version of PQcancel

2022-11-29 Thread Daniel Gustafsson
> On 15 Nov 2022, at 12:38, Jelte Fennema wrote: > Here's the correct one.<0001-Add-non-blocking-version-of-PQcancel.patch> This version of the patch no longer applies, a rebased version is needed. -- Daniel Gustafsson https://vmware.com/

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread SATYANARAYANA NARLAPURAM
On Tue, Nov 29, 2022 at 10:52 AM Andrey Borodin wrote: > On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: > > > > On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM wrote: > > > 2. Process proc die immediately when a backend is waiting for sync > > > replication ackno

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Robert Haas
On Fri, Sep 10, 2021 at 2:39 AM Noah Misch wrote: > Revoke PUBLIC CREATE from public schema, now owned by pg_database_owner. > > This switches the default ACL to what the documentation has recommended > since CVE-2018-1058. Upgrades will carry forward any old ownership and > ACL. Sites that decl

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Justin Pryzby
On Tue, Nov 29, 2022 at 02:22:59PM -0500, Robert Haas wrote: > Here's a proposed patch to do that. If I'm not wrong, you message includes a diffstat but without the patch itself.

Re: Collation version tracking for macOS

2022-11-29 Thread Joe Conway
On 11/29/22 13:59, Jeff Davis wrote: On Tue, 2022-11-29 at 11:27 -0500, Joe Conway wrote: My vote is for something like #5. The collversion should indicate a specific immutable ordering behavior. Easier said than done: https://www.postgresql.org/message-id/abddc35a7a447d93e2b8371a1a9052cb48866

Re: pgsql: Revoke PUBLIC CREATE from public schema, now owned by pg_databas

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 2:32 PM Justin Pryzby wrote: > On Tue, Nov 29, 2022 at 02:22:59PM -0500, Robert Haas wrote: > > Here's a proposed patch to do that. > > If I'm not wrong, you message includes a diffstat but without the patch > itself. D'oh. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Mon, 2022-11-28 at 19:36 -0800, Jeff Davis wrote: > On Mon, 2022-11-28 at 21:57 -0500, Robert Haas wrote: > > That is ... astonishingly bad. > > https://unicode-org.atlassian.net/browse/CLDR-16175 Oops, reported in CLDR instead of ICU. Moved to: https://unicode-org.atlassian.net/browse/ICU-22

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-29 Thread SATYANARAYANA NARLAPURAM
On Tue, Nov 29, 2022 at 11:20 AM SATYANARAYANA NARLAPURAM < satyanarlapu...@gmail.com> wrote: > > > On Tue, Nov 29, 2022 at 10:52 AM Andrey Borodin > wrote: > >> On Tue, Nov 29, 2022 at 8:29 AM Bruce Momjian wrote: >> > >> > On Tue, Nov 29, 2022 at 08:14:10AM -0800, SATYANARAYANA NARLAPURAM >> w

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > One bit of weirdness is that I may have found another ICU problem. Reported as: https://unicode-org.atlassian.net/browse/ICU-22216 -- Jeff Davis PostgreSQL Contributor Team - AWS

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 8:03 AM Jeff Davis wrote: > On Wed, 2022-11-30 at 07:18 +1300, Thomas Munro wrote: > > I think it also includes the CLDR version for *some* locales. From a > > quick look, that includes 'ar', 'ru', 'tr', 'zh'. Jeff, would you > > mind sharing the same table for one of tho

Re: Collation version tracking for macOS

2022-11-29 Thread Robert Haas
On Tue, Nov 29, 2022 at 1:59 PM Jeff Davis wrote: > 6. Create a new concept of a "locked down collation" that points at > some specific collation code (identified by some combination of library > version and collation version or whatever else can be used to identify > it). If a collation is locked

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 8:52 AM Robert Haas wrote: > On Tue, Nov 29, 2022 at 1:59 PM Jeff Davis wrote: > > 6. Create a new concept of a "locked down collation" that points at > > some specific collation code (identified by some combination of library > > version and collation version or whatever

Re: Non-decimal integer literals

2022-11-29 Thread David Rowley
On Tue, 29 Nov 2022 at 23:11, Dean Rasheed wrote: > > On Wed, 23 Nov 2022 at 08:56, David Rowley wrote: > > > > On Wed, 23 Nov 2022 at 21:54, David Rowley wrote: > > > I wonder if you'd be better off with something like: > > > > > > while (*ptr && isxdigit((unsigned char) *ptr)) > > >

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Tue, 2022-11-29 at 14:34 -0500, Joe Conway wrote: > I understand that it is not easily done, but if the combination of > collprovider + collversion does not represent specific immutable > ordering behavior for a given locale Given the u_versionToString() bug, we know the version string could

Re: Non-decimal integer literals

2022-11-29 Thread David Rowley
On Tue, 29 Nov 2022 at 03:00, Peter Eisentraut wrote: > Fixed in new patch. There seems to be a small bug in the pg_strtointXX functions in the code that checks that there's at least 1 digit. This causes 0x to be a valid representation of zero. That does not seem to be allowed by the parser, so

Re: User functions for building SCRAM secrets

2022-11-29 Thread Daniel Gustafsson
> On 27 Nov 2022, at 06:21, Jonathan S. Katz wrote: > On 11/26/22 2:53 PM, Jonathan S. Katz wrote: >> On 11/16/22 10:09 PM, Michael Paquier wrote: > >>> git diff --check reports some whitespaces. >> Ack. Will fix on the next pass. (I've been transitioning editors, which >> could have resulted in

Re: Missing update of all_hasnulls in BRIN opclasses

2022-11-29 Thread Justin Pryzby
On Mon, Nov 28, 2022 at 01:13:14AM +0100, Tomas Vondra wrote: > Opinions? Considering this will need to be backpatches, it'd be good to > get some feedback on the approach. I think it's fine, but it would be > unfortunate to fix one issue but break BRIN in a different way. > --- a/contrib/pageinsp

Re: Slow standby snapshot

2022-11-29 Thread Tom Lane
I wrote: > That seems like a fairly bad idea: it will add extra contention > on ProcArrayLock, and I see no real strong argument that the path > can't get traversed often enough for that to matter. It would > likely be better for KnownAssignedXidsCompress to obtain the lock > for itself, only afte

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 08:41 +1300, Thomas Munro wrote: > In terms of user experience, I think that might mean that users of > 'zh' who encounter warnings after a minor upgrade would therefore > really only have the options of REFRESHing and rebuilding, or > downgrading the package, because there's

Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-11-29 Thread Jacob Champion
On 11/23/22 19:45, Andrey Chudnovsky wrote: > Mostly terminology questions here. OAUTHBEARER SASL appears to be the > spec about using OAUTH2 tokens for Authentication. > While any OAUTH2 can generally work, we propose to specifically > highlight that only OIDC providers can be supported, as we nee

Re: [PoC] Federated Authn/z with OAUTHBEARER

2022-11-29 Thread Jacob Champion
On 11/24/22 00:20, mahendrakar s wrote: > I had validated Github by skipping the discovery mechanism and letting > the provider extension pass on the endpoints. This is just for > validation purposes. > If it needs to be supported, then need a way to send the discovery > document from extension. Y

Re: Slow standby snapshot

2022-11-29 Thread Michail Nikolaev
Hello, Tom. > Since we're running out of time in the current commitfest, > I went ahead and changed that, and made the cosmetic fixes > I wanted, and pushed. Great, thanks! The small thing I was thinking to add in KnownAssignedXidsCompress is the assertion like Assert(MyBackendType == B_STARTUP

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 9:59 AM Jeff Davis wrote: > Here's what I found for the 'ar' locale (firstminor/lastminor are the > icu library versions, firstcollversion/lastcollversion are their > respective collation versions for the given locale): > > firstminor | lastminor | firstcollversion | lastc

Re: Removing another gen_node_support.pl special case

2022-11-29 Thread Tom Lane
I wrote: > I notice that EquivalenceClass is already marked as no_copy_equal, > which means that gen_node_support.pl can know that emitting a > recursive node-copy or node-compare request is a bad idea. What > do you think of using the patch as it stands, plus a cross-check > that we don't emit CO

Re: Slow standby snapshot

2022-11-29 Thread Tom Lane
Michail Nikolaev writes: > The small thing I was thinking to add in KnownAssignedXidsCompress is > the assertion like > Assert(MyBackendType == B_STARTUP); Mmm ... given where the call sites are, we have got lots more problems than this if some non-startup process reaches them. I'm not sure thi

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 09:00 +1300, Thomas Munro wrote: > I'm struggling to understand what's new about proposal #6. Perhaps it's just a slight variant; I'm not sure. It's not a complete proposal yet. The difference I had in mind is that it would treat the built-in ICU differently from what is fou

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 8:38 AM Jeff Davis wrote: > On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > > One bit of weirdness is that I may have found another ICU problem. > > Reported as: > > https://unicode-org.atlassian.net/browse/ICU-22216 I'm no expert on loader/linker arcana but I have

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Greg Stark
On Mon, 28 Nov 2022 at 13:00, Robert Haas wrote: > > On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > I vote to just remove them. I think that most people won't update > their queries until they are forced to do so. I don't think it > matters very much when we force them to do that. I wo

Re: Patch: Global Unique Index

2022-11-29 Thread Greg Stark
On Fri, 25 Nov 2022 at 20:03, David Zhang wrote: > > Hi Bruce, > > Thank you for helping review the patches in such detail. > > On 2022-11-25 9:48 a.m., Bruce Momjian wrote: > > Looking at the patch, I am unclear how the the patch prevents concurrent > duplicate value insertion during the partitio

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Greg Stark
On Mon, 28 Nov 2022 at 16:53, Peter Geoghegan wrote: > > Imagine if we actually had 64-bit XIDs -- let's assume for a moment > that it's a done deal. This raises a somewhat awkward question: do you > just let the system get further and further behind on freezing, > forever? We can all agree that

Re: Patch: Global Unique Index

2022-11-29 Thread Tom Lane
Greg Stark writes: > If I understand correctly you're going to insert into the local index > for the partition using the normal btree uniqueness implementation. > Then while holding an exclusive lock on the index do lookups on every > partition for the new key. Effectively serializing inserts to t

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 10:52 +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 8:38 AM Jeff Davis wrote: > > On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > > > One bit of weirdness is that I may have found another ICU > > > problem. > > > > Reported as: > > > > https://unicode-org.atla

Re: Introduce a new view for checkpointer related stats

2022-11-29 Thread Andres Freund
Hi, On 2022-11-28 12:58:48 -0500, Robert Haas wrote: > On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > > I think we should consider deprecating the pg_stat_bgwriter columns but > > leaving them in place for a few years. New stuff should only be added to > > pg_stat_checkpointer, but we don

Re: Collation version tracking for macOS

2022-11-29 Thread Jeff Davis
On Wed, 2022-11-30 at 10:29 +1300, Thomas Munro wrote: > On Wed, Nov 30, 2022 at 9:59 AM Jeff Davis wrote: > > Here's what I found for the 'ar' locale (firstminor/lastminor are > > the > > icu library versions, firstcollversion/lastcollversion are their > > respective collation versions for the gi

Re: Add LZ4 compression in pg_dump

2022-11-29 Thread Michael Paquier
On Tue, Nov 29, 2022 at 12:10:46PM +, gkokola...@pm.me wrote: > Thank you. Please advice if is preferable to split 0002 in two parts. > I think not but I will happily do so if you think otherwise. This one makes me curious. What kind of split are you talking about? If it makes the code review

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 1:32 PM Jeff Davis wrote: > On Wed, 2022-11-30 at 10:29 +1300, Thomas Munro wrote: > > On Wed, Nov 30, 2022 at 9:59 AM Jeff Davis wrote: > > > Here's what I found for the 'ar' locale (firstminor/lastminor are > > > the > > > icu library versions, firstcollversion/lastcollv

Re: Collation version tracking for macOS

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 1:25 PM Jeff Davis wrote: > On Wed, 2022-11-30 at 10:52 +1300, Thomas Munro wrote: > > On Wed, Nov 30, 2022 at 8:38 AM Jeff Davis wrote: > > > On Tue, 2022-11-29 at 10:46 -0800, Jeff Davis wrote: > > > https://unicode-org.atlassian.net/browse/ICU-22216 > > > > I'm no exper

Re: Collation version tracking for macOS

2022-11-29 Thread Michael Paquier
On Wed, Nov 30, 2022 at 01:50:51PM +1300, Thomas Munro wrote: > The new character added to Version 12.1 is: > > U+32FF SQUARE ERA NAME REIWA > > Version 12.1 adds that single character to enable software to be > rapidly updated to support the new Japanese era name in calendrical > systems and dat

Re: User functions for building SCRAM secrets

2022-11-29 Thread Michael Paquier
On Tue, Nov 29, 2022 at 09:32:34PM +0100, Daniel Gustafsson wrote: > On the whole I tend to agree with Jacob upthread, while this does provide > consistency it doesn't seem to move the needle for best practices. Allowing > scram_build_secret_sha256('password', 'a', 1); with the password potentiall

Re: Strange failure on mamba

2022-11-29 Thread Tom Lane
I wrote: > Thomas Munro writes: >> On Fri, Nov 18, 2022 at 11:08 AM Tom Lane wrote: >>> mamba has been showing intermittent failures in various replication >>> tests since day one. >> I wonder if it's a runtime variant of the other problem. We do >> load_file("libpqwalreceiver", false) before u

Re: New docs chapter on Transaction Management and related changes

2022-11-29 Thread Bruce Momjian
On Fri, Nov 18, 2022 at 02:11:50PM -0500, Bruce Momjian wrote: > On Mon, Nov 7, 2022 at 11:04:46PM +0100, Laurenz Albe wrote: > > On Sat, 2022-11-05 at 10:08 +, Simon Riggs wrote: > > > Agreed; new compilation patch attached, including mine and then > > > Robert's suggested rewordings. > > >

Re: Patch: Global Unique Index

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 06:13:56PM -0500, Tom Lane wrote: > Greg Stark writes: > > If I understand correctly you're going to insert into the local index > > for the partition using the normal btree uniqueness implementation. > > Then while holding an exclusive lock on the index do lookups on every

Re: Patch: Global Unique Index

2022-11-29 Thread Tom Lane
Bruce Momjian writes: > On Tue, Nov 29, 2022 at 06:13:56PM -0500, Tom Lane wrote: >> ... not to mention creating a high probability of deadlocks between >> concurrent insertions to different partitions. If they each >> ex-lock their own partition's index before starting to look into >> other part

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-29 Thread Chris Travers
On Tue, Nov 29, 2022 at 5:57 PM Bruce Momjian wrote: > On Tue, Nov 29, 2022 at 11:41:07AM -0500, Robert Haas wrote: > > My argument is that removing xidStopLimit is totally fine, because it > > only serves to stop the database. What to do about xidWarnLimit is a > > slightly more complex question

Re: Patch: Global Unique Index

2022-11-29 Thread Bruce Momjian
On Tue, Nov 29, 2022 at 09:16:23PM -0500, Tom Lane wrote: > Assuming that you are inserting into index X, and you've checked > index Y to find that it has no conflicts, what prevents another > backend from inserting a conflict into index Y just after you look? > AIUI the idea is to prevent that by

Re: Strange failure on mamba

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 2:44 PM Tom Lane wrote: > Now, we certainly cannot think that these are occurring early in > postmaster startup. In the wake of 8acd8f869, we should expect > that there's no further need to call rtld_bind at all in the > postmaster, but seemingly that's not so. It's very

Re: Strange failure on mamba

2022-11-29 Thread Thomas Munro
On Wed, Nov 30, 2022 at 3:43 PM Thomas Munro wrote: > sigaction(0, NULL, &mask) s/sigaction/sigprocmask/

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-11-29 Thread Justin Pryzby
On Mon, Nov 28, 2022 at 09:08:36PM -0500, Melanie Plageman wrote: > > +pgstat_io_op_stats_collected(BackendType bktype) > > +{ > > + return bktype != B_INVALID && bktype != B_ARCHIVER && bktype != > > B_LOGGER && > > + bktype != B_WAL_RECEIVER && bktype != B_WAL_WRITER; > > > >

Re: Make mesage at end-of-recovery less scary.

2022-11-29 Thread Kyotaro Horiguchi
At Tue, 22 Nov 2022 16:04:56 -0600, Justin Pryzby wrote in > On Fri, Nov 18, 2022 at 05:25:37PM +0900, Kyotaro Horiguchi wrote: > > + while (*p == 0 && p < pe) > > + p++; > > The bug reported by Andres/cfbot/ubsan is here. > > Fixed in attached. Ur..ou.. -

  1   2   >