Re: [Proposal] Add foreign-server health checks infrastructure

2023-01-25 Thread Katsuragi Yuta
On 2023-01-23 14:40, Hayato Kuroda (Fujitsu) wrote: Dear Ted, Thanks for reviewing! PSA new version. For v25-0001-Add-PQConnCheck-and-PQCanConnCheck-to-libpq.patch , `pqConnCheck_internal` only has one caller which is quite short. Can pqConnCheck_internal and PQConnCheck be merged into one fu

Re: Operation log for major operations

2023-01-25 Thread Dmitry Koval
Hi! Maybe another discussion thread can be created for the consolidation of XX_file_exists functions. Usually XX_file_exists functions are simple. They contain single call stat() or open() and specific error processing after this call. Likely the unified function will be too complex to cove

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-25 Thread Kyotaro Horiguchi
At Wed, 25 Jan 2023 12:30:19 +0530, Amit Kapila wrote in > On Wed, Jan 25, 2023 at 11:57 AM Kyotaro Horiguchi > wrote: > > > > At Tue, 24 Jan 2023 12:19:04 +, "Takamichi Osumi (Fujitsu)" > > wrote in > > > Attached the patch v20 that has incorporated all comments so far. > > > ... > > > >

Re: document the need to analyze partitioned tables

2023-01-25 Thread David Rowley
On Wed, 25 Jan 2023 at 19:46, Laurenz Albe wrote: > Did you see Justin's wording suggestion in > https://postgr.es/m/20230118174919.GA9837%40telsasoft.com ? > He didn't attach it as a patch, so you may have missed it. > I was pretty happy with that. I didn't pay too much attention as I tend to ap

Re: old_snapshot_threshold bottleneck on replica

2023-01-25 Thread Maxim Orlov
On Tue, 24 Jan 2023 at 18:46, Robert Haas wrote: > > (1) that mutex also protects something else and the existing comment > is wrong, or > > (2) the mutex should have been removed but the patch neglected to do so, or > > (3) the mutex is still needed for some reason, in which case either > (3a) t

Re: plpython vs _POSIX_C_SOURCE

2023-01-25 Thread Andres Freund
Hi, On 2023-01-24 23:37:44 -0500, Tom Lane wrote: > Andres Freund writes: > > Patches attached. > > +1 for 0001. Cool, will push tomorrow. > I'm still nervous about 0002. However, maybe the cases that we had trouble > with are legacy issues that nobody cares about anymore in 2023. We can >

Re: to_hex() for negative inputs

2023-01-25 Thread Aleksander Alekseev
Hi Dean, > I don't see how a couple of extra arguments will expand to hundreds. Maybe I was exaggerating, but the point is that adding extra flags for every possible scenario is a disadvantageous approach in general. There is no need to increase the code base, the amount of test cases and the amo

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread David Rowley
On Wed, 25 Jan 2023 at 13:55, Melanie Plageman wrote: > David Rowley and I were discussing how to test the > NoMovementScanDirection case for heapgettup() and heapgettup_pagemode() > in [1] (since there is not currently coverage). We are actually > wondering if it is dead code (in core). Yeah, so

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-25 Thread Andres Freund
Hi, On 2023-01-24 20:59:04 -0800, Andres Freund wrote: > I find this to be awkward code. The booleans are kinda pointless, and the > tableagevac case is hard to follow because trigger is set elsewhere. > > I can give reformulating it a go. Need to make some food first. Here's a draft of what I am

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-01-25 Thread Dean Rasheed
On Wed, 18 Jan 2023 at 15:25, Justin Pryzby wrote: > > TBH, I think the best approach is what I did in: > 0001-report-top-parent-progress-for-CREATE-INDEX.txt > > That's a minimal patch, ideal for backpatching. > > ..which defines/clarifies that the progress reporting is only for > *direct* childr

Re: to_hex() for negative inputs

2023-01-25 Thread Dean Rasheed
On Wed, 25 Jan 2023 at 09:02, Aleksander Alekseev wrote: > > > I don't see how a couple of extra arguments will expand to hundreds. > > Maybe I was exaggerating, but the point is that adding extra flags for > every possible scenario is a disadvantageous approach in general. > There is no need to i

Re: to_hex() for negative inputs

2023-01-25 Thread Aleksander Alekseev
Hi Dean, > Of course there is. This is easy to code in C using unsigned ints, > without resorting to abs() (yes, I'm aware that abs() is undefined for > INT_MIN). So in your opinion what is the expected result of to_hex(INT_MIN, with_sign => true)? -- Best regards, Aleksander Alekseev

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-25 Thread Hayato Kuroda (Fujitsu)
Dear Katsuragi-san, Thank you for reading the patch! PSA new version. > Thank you for updating the patch! > > +/* Check whether the postgres server is still alive or not */ > +extern int PQConnCheck(PGconn *conn); > +extern int PQCanConnCheck(void); > > Aren't these PQconnCheck and PQcanConnChe

Re: to_hex() for negative inputs

2023-01-25 Thread Dean Rasheed
On Wed, 25 Jan 2023 at 10:57, Aleksander Alekseev wrote: > > > Of course there is. This is easy to code in C using unsigned ints, > > without resorting to abs() (yes, I'm aware that abs() is undefined for > > INT_MIN). > > So in your opinion what is the expected result of to_hex(INT_MIN, > with_si

Re: Logical replication timeout problem

2023-01-25 Thread Amit Kapila
On Tue, Jan 24, 2023 at 8:15 AM wangw.f...@fujitsu.com wrote: > > Attach the new patch. > I think the patch missed to handle the case of non-transactional messages which was previously getting handled. I have tried to address that in the attached. Is there a reason that shouldn't be handled? Apar

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
> After looking at this, it seemed to me that the factorization > wasn't quite right after all: specifically, the new function > could be used in several more places if it confines itself to > being a privilege check and doesn't consider GUC_NO_SHOW_ALL. > So more like the attached. > > You could a

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
>>> Also, I intentionally dropped the GUC_NO_SHOW_ALL check in >>> get_explain_guc_options, because it seems redundant given >>> the preceding GUC_EXPLAIN check. It's unlikely we'd ever have >>> a variable that's marked both GUC_EXPLAIN and GUC_NO_SHOW_ALL ... >>> but if we did, shouldn't the form

Re: drop postmaster symlink

2023-01-25 Thread Devrim Gündüz
Hi, On Wed, 2023-01-25 at 08:54 +0100, Peter Eisentraut wrote: > > Apart from your concerns, it appears there is consensus for making > this change.  The RPM packaging scripts can obviously be fixed > easily for this.  Do you have an objection to making this change? I'm inclined to create the

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-01-25 Thread shveta malik
On Mon, Jan 23, 2023 at 6:30 PM Melih Mutlu wrote: > > Hi, > > Thanks for your review. > Attached updated versions of the patches. > Hello, I am still in the process of reviewing the patch, before that I tried to run below test: --publisher create table tab1(id int , name varchar); create table

Re: Fix to enum hashing for dump and restore

2023-01-25 Thread Andrew
Those are excellent points. We will investigate adjusting pg_dump behavior, as this is primarily a dump+restore issue. Thank you! -Andrew J Repp (VMware) On Tue, Jan 24, 2023, at 9:56 PM, Tom Lane wrote: > Andrew writes: > > I have discovered a bug in one usage of enums. If a table with hash

CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
This works in PG 15: CREATE ROLE service CREATEROLE; CREATE ROLE service1 WITH LOGIN IN ROLE service; SET SESSION AUTHORIZATION service; CREATE ROLE service2 WITH LOGIN IN ROLE service; but generates an error in git master: CREATE ROLE service CREATEROLE;

Re: plpython vs _POSIX_C_SOURCE

2023-01-25 Thread Robert Haas
On Tue, Jan 24, 2023 at 11:37 PM Tom Lane wrote: > Andres Freund writes: > > Patches attached. > > +1 for 0001. I'm still nervous about 0002. However, maybe the > cases that we had trouble with are legacy issues that nobody cares > about anymore in 2023. We can always look for another answer i

Syncrep and improving latency due to WAL throttling

2023-01-25 Thread Jakub Wartak
Hi, attached is proposal idea by Tomas (in CC) for protecting and prioritizing OLTP latency on syncrep over other heavy WAL hitting sessions. This is the result of internal testing and research related to the syncrep behavior with Tomas, Alvaro and me. The main objective of this work-in-progress/c

Re: CREATE ROLE bug?

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 8:29 AM Bruce Momjian wrote: > This works in PG 15: > > CREATE ROLE service CREATEROLE; > CREATE ROLE service1 WITH LOGIN IN ROLE service; > SET SESSION AUTHORIZATION service; > CREATE ROLE service2 WITH LOGIN IN ROLE service; > > but generat

Re: old_snapshot_threshold bottleneck on replica

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 3:52 AM Maxim Orlov wrote: > But in TransactionIdLimitedForOldSnapshots these variable is using > conjointly. So, I'm not > sure, is it completely safe to remove mutex. Well, that's something we - and ideally you, as the patch author - need to analyze and figure out. We c

More pgindent tweaks

2023-01-25 Thread Andrew Dunstan
After I committed 1249371632 I thought that I should really go ahead and do what I suggested and allow multiple exclude pattern files for pgindent. One obvious case is to exclude an in tree meson build directory. I also sometimes have other in tree objects I'd like to be able exclude. The attached

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread songjinzhou
Hello, this usage scenario is from Oracle's PL/SQL language (I have been doing the function development of PL/SQL language for some time). I think this patch is very practical and will expand our for loop scenario. In short, I look forward to your reply. Happy Chinese New Year! songjinzhou(2

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread Pavel Stehule
Hi st 25. 1. 2023 v 15:18 odesílatel songjinzhou <2903807...@qq.com> napsal: > Hello, this usage scenario is from Oracle's PL/SQL language (I have been > doing the function development of PL/SQL language for some time). I think > this patch is very practical and will expand our for loop scenario

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-25 Thread Takamichi Osumi (Fujitsu)
On Wednesday, January 25, 2023 3:27 PM Kyotaro Horiguchi wrote: > At Tue, 24 Jan 2023 12:19:04 +, "Takamichi Osumi (Fujitsu)" > wrote in > > Attached the patch v20 that has incorporated all comments so far. > > Thanks! I looked thourgh the documentation part. Thank you for your review ! >

RE: Perform streaming logical transactions by background workers and parallel apply

2023-01-25 Thread houzj.f...@fujitsu.com
On Wednesday, January 25, 2023 7:30 AM Peter Smith wrote: > > Here are my review comments for patch v87-0002. Thanks for your comments. > == > doc/src/sgml/config.sgml > > 1. > > -Allows streaming or serializing changes immediately in > logical decoding. > The al

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-25 Thread Takamichi Osumi (Fujitsu)
On Wednesday, January 25, 2023 3:55 PM Amit Kapila wrote: > On Wed, Jan 25, 2023 at 11:23 AM Takamichi Osumi (Fujitsu) > wrote: > > > > > > Thank you for checking the patch ! > > On Wednesday, January 25, 2023 10:17 AM Kyotaro Horiguchi > wrote: > > > In short, I'd like to propose renaming the

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 08:47:14AM -0500, Robert Haas wrote: > > I am not sure if the behavior is wrong, the error message is wrong, or > > it is working as expected. > > It is indeed related to that discussion and change. In existing > released branches, a CREATEROLE user can make any role a memb

Re: CREATE ROLE bug?

2023-01-25 Thread David G. Johnston
On Wed, Jan 25, 2023 at 7:35 AM Bruce Momjian wrote: > > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission? Are we adding any security by > preventing this? > > As an encouraged design choice you wouldn't. You'd create a new group and

Re: More pgindent tweaks

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 08:59:44AM -0500, Andrew Dunstan wrote: > After I committed 1249371632 I thought that I should really go ahead and > do what I suggested and allow multiple exclude pattern files for > pgindent. One obvious case is to exclude an in tree meson build > directory. I also sometim

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Tom Lane
David Rowley writes: > Does anyone know of any reason why we shouldn't ditch the nomovement > code in heapgettup/heapgettup_pagemode? AFAICS, the remaining actual use-case for NoMovementScanDirection is that defined by ExecutorRun: *If direction is NoMovementScanDirection then nothing i

Re: Decoupling antiwraparound autovacuum from special rules around auto cancellation

2023-01-25 Thread Robert Haas
On Tue, Jan 24, 2023 at 3:33 PM Peter Geoghegan wrote: > Sure, it's possible that such a cancellable aggressive autovacuum was > indeed cancelled, and that that factor made the crucial difference. > But I find it far easier to believe that there simply was no such > aggressive autovacuum in the fi

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 07:38:51AM -0700, David G. Johnston wrote: > On Wed, Jan 25, 2023 at 7:35 AM Bruce Momjian wrote: > > > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission?  Are we adding any security by > preventing t

[PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi hackers, Currently we allow self-conflicting inserts for ON CONFLICT DO NOTHING: ``` CREATE TABLE t (a INT UNIQUE, b INT); INSERT INTO t VALUES (1,1), (1,2) ON CONFLICT DO NOTHING; -- succeeds, inserting the first row and ignoring the second ``` ... but not for ON CONFLICT .. DO UPDATE: ``` I

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-01-25 Thread Andrei Zubkov
Hi, I've updated this patch for the current master. Also I have some additional explanations.. On Wed, 2023-01-18 at 17:29 +0100, Tomas Vondra wrote: > 1) I'm not sure why the patch is adding tests of permissions on the > pg_stat_statements_reset function? I've fixed that > > 2) If we want the

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Israel Barth Rubio
Hello Jim/Jacob, > > I do not think it is worth it to change the current behavior of > PostgreSQL > > in that sense. > > Well, I am not suggesting to change the current behavior of PostgreSQL in > that matter. Quite the contrary, I find this feature very convenient, > specially when you need to de

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread Matthias van de Meent
On Tue, 24 Jan 2023 at 23:50, Peter Geoghegan wrote: > > On Mon, Jan 16, 2023 at 5:55 PM Peter Geoghegan wrote: > > 0001 (the freezing strategies patch) is now committable IMV. Or at > > least will be once I polish the docs a bit more. I plan on committing > > 0001 some time next week, barring an

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Tom Lane
Nitin Jadhav writes: > I agree that the developer can use both GUC_NO_SHOW_ALL and > GUC_EXPLAIN knowingly or unknowingly for a single GUC. If used by > mistake then according to the existing code (without patch), > GUC_NO_SHOW_ALL takes higher precedence whether it is marked first or > last in th

Re: More pgindent tweaks

2023-01-25 Thread Andrew Dunstan
On 2023-01-25 We 09:41, Bruce Momjian wrote: > On Wed, Jan 25, 2023 at 08:59:44AM -0500, Andrew Dunstan wrote: >> After I committed 1249371632 I thought that I should really go ahead and >> do what I suggested and allow multiple exclude pattern files for >> pgindent. One obvious case is to exclud

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread songjinzhou
>Hi >st 25. 1. 2023 v 16:39 odesílatel songjinzhou <2903807...@qq.com> napsal: >Hello, my personal understanding is that you can use multiple iterative >controls (as a merge) in a fo loop, otherwise we can only separate these >iterative controls, but in fact, they may do the same thing. >1. p

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Tom Lane
Kyotaro Horiguchi writes: > At Tue, 24 Jan 2023 10:42:17 -0800, Nathan Bossart > wrote in >> Here is a first attempt at a patch. I scanned through all the existing >> uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything >> else that needed adjusting. > There seems to be

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread Pavel Stehule
st 25. 1. 2023 v 17:22 odesílatel songjinzhou <2903807...@qq.com> napsal: > > >Hi > > >st 25. 1. 2023 v 16:39 odesílatel songjinzhou <2903807...@qq.com> > napsal: Hello, my personal understanding is that you can use multiple > iterative controls (as a merge) in a fo loop, otherwise we can only > s

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Jacob Champion
On Wed, Jan 25, 2023 at 7:47 AM Israel Barth Rubio wrote: > I imagine more people might have already hit a similar situation too. While > the > workaround can seem a bit weird, in my very humble opinion the user/client is > somehow still the one to blame in this case as it is providing the "wrong

Re: CREATE ROLE bug?

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 9:35 AM Bruce Momjian wrote: > So, how would someone with CREATEROLE permission add people to their own > role, without superuser permission? Are we adding any security by > preventing this? They can't, because a role can't ever have ADMIN OPTION on itself, and you need A

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Tomas Vondra
On 1/25/23 16:37, gkokola...@pm.me wrote: > > > > > > --- Original Message --- > On Wednesday, January 25th, 2023 at 2:42 AM, Justin Pryzby > wrote: > > >> >> >> On Tue, Jan 24, 2023 at 03:56:20PM +, gkokola...@pm.me wrote: >> >>> On Monday, January 23rd, 2023 at 7:00 PM, Jus

Re: CREATE ROLE bug?

2023-01-25 Thread Bruce Momjian
On Wed, Jan 25, 2023 at 12:21:14PM -0500, Robert Haas wrote: > But it does seem pretty important to keep talking about these things, > because there's definitely no guarantee whatsoever that all of the > commits I've made to master in this area are without problems. If we > find important cases tha

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier wrote: > Rename contrib module basic_archive to basic_wal_module FWIW, I find this new name much less clear than the old one. If we want to provide a basic_archive module and a basic_recovery module, that seems fine. Why merge them? -- Robert Ha

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 04:12:00PM +0900, Kyotaro Horiguchi wrote: > At Tue, 24 Jan 2023 10:42:17 -0800, Nathan Bossart > wrote in >> Here is a first attempt at a patch. I scanned through all the existing >> uses of InvalidDsaPointer and DSM_HANDLE_INVALID and didn't notice anything >> else tha

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 11:49:27AM -0500, Tom Lane wrote: > Right. I fixed some other infelicities and pushed it. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread Justin Pryzby
On Wed, Jan 25, 2023 at 03:37:12PM +, gkokola...@pm.me wrote: > Of course, one can throw the error before entering the loop, yet I think > that it does not help the readability of the code. IMHO it is easier to > follow if the error is thrown once during that check. > If anything, I can sugges

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 12:49:45PM -0500, Robert Haas wrote: > On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier wrote: >> Rename contrib module basic_archive to basic_wal_module > > FWIW, I find this new name much less clear than the old one. > > If we want to provide a basic_archive module and

Re: Authentication fails for md5 connections if ~/.postgresql/postgresql.{crt and key} exist

2023-01-25 Thread Israel Barth Rubio
Hello Jacob, > I'm not sure how helpful it is to assign "blame" here. I think the > requested improvement is reasonable -- it should be possible to > override the default for a particular connection, without having to > pick a junk value that you hope doesn't match up with an actual file > on the

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 10:02:28 -0500, Tom Lane wrote: > David Rowley writes: > > Does anyone know of any reason why we shouldn't ditch the nomovement > > code in heapgettup/heapgettup_pagemode? +1 Because I dug it up yesterday. There used to be callers of heap* with NoMovement. But they were unused

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 18:45:12 +0300, Aleksander Alekseev wrote: > Currently we allow self-conflicting inserts for ON CONFLICT DO NOTHING: > > ``` > CREATE TABLE t (a INT UNIQUE, b INT); > INSERT INTO t VALUES (1,1), (1,2) ON CONFLICT DO NOTHING; > -- succeeds, inserting the first row and ignoring th

pg_upgrade from PG-14.5 to PG-15.1 failing due to non-existing function

2023-01-25 Thread Dimos Stamatakis
Hi hackers, I attempted to perform an upgrade from PG-14.5 to PG-15.1 with pg_upgrade and unfortunately it errors out because of a function that does not exist anymore in PG-15.1. The function is ‘pg_catalog.close_lb’ and it exists in 14.5 but not in 15.1. In our scenario we changed the permissi

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 07.01.23 01:34, Justin Pryzby wrote: "ON (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END = t.oid)\n" This breaks interoperability with older servers: ERROR: column a.attrealtypid does not exist Same in describe.c Find attached some typos and bad indentation. I'm se

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 12.01.23 17:32, Peter Eisentraut wrote: Can we do anything about the attack vector wherein a malicious DBA simply copies the encrypted datum from one row to another? We discussed this earlier [0].  This patch is not that feature.  We could get there eventually, but it would appear to be an

Re: Transparent column encryption

2023-01-25 Thread Peter Eisentraut
On 19.01.23 21:48, Jacob Champion wrote: I like the existing "caveats" documentation, and I've attached a sample patch with some more caveats documented, based on some of the upthread conversation: - text format makes fixed-length columns leak length information too - you only get partial protec

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi Andres, > I don't think I agree with this being a bug. Perhaps that's not a bug especially considering the fact that the documentation describes this behavior, but in any case the fact that: ``` INSERT INTO t VALUES (1,1) ON CONFLICT (a) DO UPDATE SET b = 0; INSERT INTO t VALUES (1,2) ON CONF

Re: Syncrep and improving latency due to WAL throttling

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 14:32:51 +0100, Jakub Wartak wrote: > In other words it allows slow down of any backend activity. Any feedback on > such a feature is welcome, including better GUC name proposals ;) and > conditions in which such feature should be disabled even if it would be > enabled globally (

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Robert Haas
On Wed, Jan 25, 2023 at 1:17 PM Nathan Bossart wrote: > On Wed, Jan 25, 2023 at 12:49:45PM -0500, Robert Haas wrote: > > On Wed, Jan 25, 2023 at 12:37 AM Michael Paquier > > wrote: > >> Rename contrib module basic_archive to basic_wal_module > > > > FWIW, I find this new name much less clear tha

Re: What object types should be in schemas?

2023-01-25 Thread Peter Eisentraut
On 12.01.23 18:41, Alvaro Herrera wrote: I think one important criterion to think about is how does encryption work when you have per-customer (or per-whatever) schemas. Is the concept of a column encryption [objtype] a thing that you would like to set up per customer? In that case, you will pr

Re: pg_upgrade from PG-14.5 to PG-15.1 failing due to non-existing function

2023-01-25 Thread Christoph Moench-Tegeder
## Dimos Stamatakis (dimos.stamata...@servicenow.com): > In our scenario we changed the permissions of this function in PG14.5 > (via an automated tool) and then pg_upgrade tries to change the > permissions in PG15.1 as well. Given that this function wasn't even documented and did nothing but thr

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Peter Geoghegan
On Wed, Jan 25, 2023 at 11:01 AM Aleksander Alekseev wrote: > Just to make sure we are on the same page. The patch doesn't break the > current DO NOTHING behavior but rather makes DO UPDATE work the same > way DO NOTHING does. It also makes DO UPDATE not work the same way as either UPDATE itself

Re: Proposal: Support custom authentication methods using hooks

2023-01-25 Thread Andrey Chudnovsky
Greetings, Want to resurface the OAUTH support topic in the context of the concerns raised here. > How about- if we just added OAUTH support directly into libpq and the > backend, would that work with Azure's OIDC provider? If not, why not? > If it does, then what's the justification for trying t

Re: Implement missing join selectivity estimation for range types

2023-01-25 Thread Mahmoud Sakr
Hi Tomas, > I finally had time to properly read the paper today - the general > approach mostly matches how I imagined the estimation would work for > inequalities, but it's definitely nice to see the algorithm properly > formalized and analyzed. Awesome, thanks for this interest! > What seems a

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Aleksander Alekseev
Hi Peter, > It also makes DO UPDATE not work the same way as either UPDATE itself > (which will silently skip a second or subsequent update of the same > row by the same UPDATE statement in RC mode), or MERGE (which has > similar cardinality violations). That's true. On the flip side, UPDATE and

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread gkokolatos
--- Original Message --- On Wednesday, January 25th, 2023 at 6:28 PM, Tomas Vondra wrote: > > > > On 1/25/23 16:37, gkokola...@pm.me wrote: > > > --- Original Message --- > > On Wednesday, January 25th, 2023 at 2:42 AM, Justin Pryzby > > pry...@telsasoft.com wrote: >

Re: to_hex() for negative inputs

2023-01-25 Thread Aleksander Alekseev
Hi Dean, > > So in your opinion what is the expected result of to_hex(INT_MIN, > > with_sign => true)? > > > > "-8000" or "-0x8000", depending on whether the prefix is > requested. Whether this is the right result is very debatable. 0x8000 is a binary representation of -2147483648: `

Re: Add LZ4 compression in pg_dump

2023-01-25 Thread gkokolatos
--- Original Message --- On Wednesday, January 25th, 2023 at 7:00 PM, Justin Pryzby wrote: > > > On Wed, Jan 25, 2023 at 03:37:12PM +, gkokola...@pm.me wrote: > > While looking at this, I realized that commit 5e73a6048 introduced a > regression: > > @@ -3740,19 +3762,24 @

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-25 Thread Andres Freund
Hi, On 2023-01-14 12:34:03 -0800, Andres Freund wrote: > On 2023-01-14 00:48:52 -0800, Jeff Davis wrote: > > On Mon, 2022-12-26 at 14:20 +0530, Bharath Rupireddy wrote: > > > Please review the attached v2 patch further. > > > > I'm still unclear on the performance goals of this patch. I see that

Re: GUCs to control abbreviated sort keys

2023-01-25 Thread Jeff Davis
On Tue, 2023-01-24 at 21:42 -0500, Robert Haas wrote: > I find it a bit premature to include this comment in the very first > email what if other people don't like the idea? The trust_strxfrm GUC was pulled from the larger collation refactoring patch, which has been out for a while. The sort_a

Set arbitrary GUC options during initdb

2023-01-25 Thread Tom Lane
The attached patch responds to the discussion at [1] about how we ought to offer a way to set any server GUC from the initdb command line. Currently, if for some reason the server won't start with default parameters, the only way to get through initdb is to change the installed version of postgres

Re: plpython vs _POSIX_C_SOURCE

2023-01-25 Thread Andres Freund
Hi, Pushed the patches. So far no fallout, and hoverfly recovered. I just checked a few of the more odd animals (Illumos, Solaris, old OpenBSD, AIX) that already ran without finding new warnings. There's a few more animals to run before I'll fully relax though. On 2023-01-25 08:31:23 -0500, Ro

Re: heapgettup() with NoMovementScanDirection unused in core?

2023-01-25 Thread Tom Lane
Andres Freund writes: > On 2023-01-25 10:02:28 -0500, Tom Lane wrote: >> We must have the NoMovementScanDirection option because count = 0 >> does not mean "do nothing", and I noted at least two call sites >> that require it. > I wonder if we'd be better off removing NoMovementScanDirection, and

Re: GUCs to control abbreviated sort keys

2023-01-25 Thread Jeff Davis
On Tue, 2023-01-24 at 19:43 -0600, Justin Pryzby wrote: > I think "an optimization, if applicable" is either too terse, or > somehow > wrong.  Maybe: > > > Enables or disables the use of abbreviated keys, a sort > > optimization... Done. > > +    optimization could return wrong results. Set

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 02:05:39PM -0500, Robert Haas wrote: > On Wed, Jan 25, 2023 at 1:17 PM Nathan Bossart > wrote: >> I wanted to merge basic_archive and basic_recovery because there's a decent >> chunk of duplicated code. Perhaps that is okay, but I would rather just >> have one test module

Re: to_hex() for negative inputs

2023-01-25 Thread Peter Eisentraut
On 24.01.23 14:10, Dean Rasheed wrote: I also think it might be useful for it to gain a couple of boolean options: 1). An option to output a signed value (defaulting to false, to preserve the current two's complement output). I find the existing behavior so strange, I would rather give up and

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Tom Lane
Nathan Bossart writes: > I wanted to merge basic_archive and basic_recovery because there's a decent > chunk of duplicated code. Would said code likely be duplicated into non-test uses of this feature? If so, maybe you ought to factor it out into a common location. I agree with Robert's point th

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-01-25 Thread SATYANARAYANA NARLAPURAM
On Sat, Jan 14, 2023 at 12:34 PM Andres Freund wrote: > Hi, > > On 2023-01-14 00:48:52 -0800, Jeff Davis wrote: > > On Mon, 2022-12-26 at 14:20 +0530, Bharath Rupireddy wrote: > > > Please review the attached v2 patch further. > > > > I'm still unclear on the performance goals of this patch. I se

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 14:05:39 -0500, Robert Haas wrote: > > I wanted to merge basic_archive and basic_recovery because there's a decent > > chunk of duplicated code. Perhaps that is okay, but I would rather just > > have one test module. AFAICT the biggest reason to split it is because we > > can't

Re: Set arbitrary GUC options during initdb

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 16:25:19 -0500, Tom Lane wrote: > The attached patch responds to the discussion at [1] about how > we ought to offer a way to set any server GUC from the initdb > command line. Are you thinking of backpatching this, to offer the people affected by the issue in [1] a way out? >

Re: Re: Support plpgsql multi-range in conditional control

2023-01-25 Thread Isaac Morland
On Wed, 25 Jan 2023 at 12:02, Pavel Stehule wrote: > > > st 25. 1. 2023 v 17:22 odesílatel songjinzhou <2903807...@qq.com> napsal: > >> >> As follows, we can only repeat the for statement before we use such SQL: >> >> begin >> for i in 10..20 loop >> raise notice '%', i; -- Things to do >> end lo

Re: Set arbitrary GUC options during initdb

2023-01-25 Thread Tom Lane
Andres Freund writes: > On 2023-01-25 16:25:19 -0500, Tom Lane wrote: >> The attached patch responds to the discussion at [1] about how >> we ought to offer a way to set any server GUC from the initdb >> command line. > Are you thinking of backpatching this, to offer the people affected by the >

Re: [PATCH] Make ON CONFLICT DO NOTHING and ON CONFLICT DO UPDATE consistent

2023-01-25 Thread Andres Freund
Hi, On 2023-01-25 22:00:50 +0300, Aleksander Alekseev wrote: > Perhaps that's not a bug especially considering the fact that the > documentation describes this behavior, but in any case the fact that: > > ``` > INSERT INTO t VALUES (1,1) ON CONFLICT (a) DO UPDATE SET b = 0; > INSERT INTO t VALUES

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 04:50:22PM -0500, Tom Lane wrote: > Nathan Bossart writes: >> I wanted to merge basic_archive and basic_recovery because there's a decent >> chunk of duplicated code. > > Would said code likely be duplicated into non-test uses of this feature? > If so, maybe you ought to f

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Nathan Bossart
On Wed, Jan 25, 2023 at 01:58:01PM -0800, Andres Freund wrote: > I think it'd be just fine to keep the name as basic_archive and use it for > both archiving and restoring. Restoring from an archive still deals with > archiving. This is my preference. If Michael and Robert are okay with it, I thin

Re: pgsql: Rename contrib module basic_archive to basic_wal_module

2023-01-25 Thread Michael Paquier
On Wed, Jan 25, 2023 at 02:41:18PM -0800, Nathan Bossart wrote: > This is my preference. If Michael and Robert are okay with it, I think > this is what we should do. Else, I'll create separate basic_archive and > basic_restore modules. Grouping both things into the same module has the advantage

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-01-25 Thread Jacob Champion
On 1/24/23 12:04, Robert Haas wrote: > I find the concept of "ambient authentication" problematic. I don't > know exactly what you mean by it. I hope you'll tell me, Sure: Ambient authority [1] means that something is granted access based on some aspect of its existence that it can't remove (or ev

[BUG] pg_stat_statements and extended query protocol

2023-01-25 Thread Imseih (AWS), Sami
Doing some work with extended query protocol, I encountered the same issue that was discussed in [1]. It appears when a client is using extended query protocol and sends an Execute message to a portal with max_rows, and a portal is executed multiple times, pg_stat_statements does not correctly trac

Re: [PATCH] Clarify the behavior of the system when approaching XID wraparound (stop telling users to "vacuum that database in single-user mode")

2023-01-25 Thread Justin Pryzby
On Mon, Jan 16, 2023 at 03:50:57PM +0300, Aleksander Alekseev wrote: > Hi hackers, > > > The proposed patchset changes the documentation and the error messages > > accordingly, making them less misleading. 0001 corrects the > > documentation but doesn't touch the code. 0002 and 0003 correct the >

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Nathan Bossart
On Tue, Jan 24, 2023 at 06:45:08PM -0500, Tom Lane wrote: > I took a look through this, and have a number of mostly-cosmetic > issues: Thanks for the detailed review. > * It seems wrong that next_sync_start isn't handled as one of the > wakeup[NUM_LRW_WAKEUPS] entries. I see that it needs to be

Re: drop postmaster symlink

2023-01-25 Thread Karl O. Pinc
Hello, Somehow I missed the email changing the status of this back to "needs review". Buried in https://www.postgresql.org/message-id/20230107165942.748ccf4e%40slate.karlpinc.com is the one change I see that should be made. > In doc/src/sgml/ref/allfiles.sgml at line 222 there is an ENTITY > def

improving user.c error messages

2023-01-25 Thread Nathan Bossart
moving this discussion to a new thread... On Thu, Jan 19, 2023 at 10:20:33AM -0500, Robert Haas wrote: > On Wed, Jan 18, 2023 at 6:17 PM Nathan Bossart > wrote: >> However, as the attribute >> system becomes more sophisticated, I think we ought to improve the error >> messages in user.c. IMHO m

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Thomas Munro
On Thu, Jan 26, 2023 at 12:50 PM Nathan Bossart wrote: > I did this in v3. I noticed that many of your comments also applied to the > similar patch that was recently applied to walreceiver.c, so I created > another patch to fix that up. Can we also use TimestampDifferenceMilliseconds()? It know

Re: suppressing useless wakeups in logical/worker.c

2023-01-25 Thread Nathan Bossart
On Thu, Jan 26, 2023 at 01:23:41PM +1300, Thomas Munro wrote: > Can we also use TimestampDifferenceMilliseconds()? It knows about > rounding up for WaitLatch(). I think we might risk overflowing "long" when all the wakeup times are DT_NOEND: * This is typically used to calculate a wait

  1   2   >