Re: An inefficient query caused by unnecessary PlaceHolderVar

2023-07-18 Thread Richard Guo
On Fri, Jun 2, 2023 at 1:33 AM James Coleman wrote: > On Wed, May 31, 2023 at 10:30 PM Richard Guo > wrote: > > Thanks for the review! > > Sure thing! I've updated the patch according to the reviews as attached. But I did not add ORDER BY clause in the test, as we don't need it for correctnes

There should be a way to use the force flag when restoring databases

2023-07-18 Thread Joan
Since posgres 13 there's the option to do a FORCE when dropping a database (so it disconnects current users) Documentation here: https://www.postgresql.org/docs/current/sql-dropdatabase.html I am currently using dir format for the output pg_dump -d "bdname" -F d -j 4 -v -f /tmp/dir And restori

Re: generic plans and "initial" pruning

2023-07-18 Thread Thom Brown
On Tue, 18 Jul 2023, 08:26 Amit Langote, wrote: > Hi Thom, > > On Tue, Jul 18, 2023 at 1:33 AM Thom Brown wrote: > > On Thu, 13 Jul 2023 at 13:59, Amit Langote > wrote: > > > In an absolutely brown-paper-bag moment, I realized that I had not > > > updated src/backend/executor/README to reflect

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-07-18 Thread Amit Kapila
On Mon, Jul 17, 2023 at 6:19 PM Amit Kapila wrote: > > On Fri, Jun 30, 2023 at 7:29 PM Hayato Kuroda (Fujitsu) > wrote: > > > > I have analyzed more, and concluded that there are no difference between > > manual > > and shutdown checkpoint. > > > > The difference was whether the CHECKPOINT recor

Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread mahendrakar s
Hi hackers, We have encountered an issue (invalid message length) when the password length is > 1000 in pg 11,12,13 versions. This is due to the limit(1000) on the max length of the password. In this case the password is an access token(JWT) which can have varied lengths > 1000. I see that this i

Re: doc: improve the restriction description of using indexes on REPLICA IDENTITY FULL table.

2023-07-18 Thread Amit Kapila
On Tue, Jul 18, 2023 at 12:10 PM Masahiko Sawada wrote: > > BTW, IsIndexOnlyExpression() is not necessary but the current code > still works fine. So do we need to backpatch it to PG16? I'm thinking > we can apply it to only HEAD. > Either way is fine but I think if we backpatch it then the code

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread Daniel Gustafsson
> On 18 Jul 2023, at 11:30, mahendrakar s wrote: > So we have two options: > 1. Backport patch[1] to 11,12,13 > 2. Change ONLY the limit on the max length of the password(my patch attached). We typically only backpatch bugfixes and not functional changes, and this seems to fall in the latter cat

Re: In Postgres 16 BETA, should the ParseNamespaceItem have the same index as it's RangeTableEntry?

2023-07-18 Thread Amit Langote
Hello, On Sat, Jul 15, 2023 at 4:43 AM Farias de Oliveira wrote: > I believe I have found something interesting that might be the root of the > problem with RTEPermissionInfo. But I do not know how to fix it exactly. In > AGE's code, the execution of it goes through a function called > analyze

Re: Add TOAST support for more system tables

2023-07-18 Thread Sofia Kopikova
On Mon, Jul 17, 2023 at 06:31:04PM -0400, Tom Lane wrote: Sofia Kopikova writes: This patch adds TOAST support for system tables pg_class, pg_attribute and pg_largeobject_metadata, as they include ACL columns, which may be potentially large in size. We have been around on this topic before, c

Re: Protect extension' internal tables - how?

2023-07-18 Thread Nikita Malakhov
Hi, Aleksander, thank you very much. Tables are already placed into special schema, but there are some dynamically created tables and the goal is to protect all these tables from direct insert, update and delete operations from users. I've read about the SECURITY DEFINER, it will do the trick. --

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread mahendrakar s
Access token length with bare minimal claims is more than 1000 in this case. Workarounds are not possible in production. On Tue, 18 Jul 2023 at 15:10, Daniel Gustafsson wrote: > > > On 18 Jul 2023, at 11:30, mahendrakar s wrote: > > > So we have two options: > > 1. Backport patch[1] to 11,12,13

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread Vik Fearing
On 7/18/23 11:30, mahendrakar s wrote: Hi hackers, We have encountered an issue (invalid message length) when the password length is > 1000 in pg 11,12,13 versions. This is due to the limit(1000) on the max length of the password. In this case the password is an access token(JWT) which can have

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread Amit Kapila
On Tue, Jul 18, 2023 at 2:33 PM Melih Mutlu wrote: > > Attached the fixed patchset. > Few comments on 0001 1. + logicalrep_worker_attach(worker_slot); + + /* Setup signal handling */ + pqsignal(SIGHUP, SignalHandlerForConfigReload); + pqsignal(SIGTERM, die); + BackgroundWorke

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2023-07-18 Thread stephane tachoires
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Only documentation patch applied on 4e465aac36ce9a9533c68dbdc83e67579880e628

Re: [RFC] Add jit deform_counter

2023-07-18 Thread Daniel Gustafsson
> On 15 Apr 2023, at 16:40, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On Fri, Mar 31, 2023 at 07:39:27PM +0200, Dmitry Dolgov wrote: >>> On Wed, Mar 29, 2023 at 01:50:37PM +1300, David Rowley wrote: I had a look at this patch today and I agree that it would be good to give the user an easier

Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Ashutosh Bapat
On Fri, Jul 14, 2023 at 7:33 PM Tomas Vondra wrote: > > Thanks for testing / confirming this! So, do we agree this behavior is > reasonable? > This behaviour doesn't need any on-disk changes or has nothing in it which prohibits us from changing it in future. So I think it's good as a v0. If requ

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread vignesh C
On Tue, 11 Jul 2023 at 08:30, Peter Smith wrote: > > On Tue, Jul 11, 2023 at 12:31 AM Melih Mutlu wrote: > > > > Hi, > > > > Hayato Kuroda (Fujitsu) , 6 Tem 2023 Per, > > 12:47 tarihinde şunu yazdı: > > > > > > Dear Melih, > > > > > > > Thanks for the 0003 patch. But it did not work for me. Can y

Re: POC, WIP: OR-clause support for indexes

2023-07-18 Thread Alena Rybakina
Hi, all! I sent a patch to commitfest and noticed that the authors and the reviewer were incorrectly marked. Sorry about that. I fixed it and sent the current version of the patch. -- Regards, Alena Rybakina Postgres Professional From 087125cc413429bda05f22ebbd51115c23819285 Mon Sep 17 00:00:

Re: Increase limit on max length of the password( pg versions < 14)

2023-07-18 Thread Tom Lane
Vik Fearing writes: > On 7/18/23 11:30, mahendrakar s wrote: >> We have encountered an issue (invalid message length) when the >> password length is > 1000 in pg 11,12,13 versions. > The third option is to upgrade. Yeah. I don't see any good reason to consider this behavior change as something

Giving more detail in pg_upgrade errormessage

2023-07-18 Thread Daniel Gustafsson
Looking at the upgrade question in [0] made me realize that we discard potentially useful information for troubleshooting. When we check if the cluster is properly shut down we might as well include the status from pg_controldata in the errormessage as per the trivial (but yet untested) proposed d

Re: Giving more detail in pg_upgrade errormessage

2023-07-18 Thread Mingli Zhang
Hi, > Is there a reason not to be verbose here as users might copy/paste this output > when asking for help? Seems better than nothing. > [0] CACoPQdbQTysF=ekckyfngtdpodxxmesf_2acno+bcnqqcb5...@mail.gmail.com Full link for convenience. [0]https://www.postgresql.org/message-id/CACoPQdbQTysF=ek

Re: remaining sql/json patches

2023-07-18 Thread Alvaro Herrera
On 2023-Jul-18, Amit Langote wrote: > Attached updated patches. In 0002, I removed the mention of the > RETURNING clause in the JSON(), JSON_SCALAR() documentation, which I > had forgotten to do in the last version which removed its support in > code. > I think 0001 looks ready to go. Alvaro?

Re: Giving more detail in pg_upgrade errormessage

2023-07-18 Thread Tom Lane
Daniel Gustafsson writes: > Looking at the upgrade question in [0] made me realize that we discard > potentially useful information for troubleshooting. When we check if the > cluster is properly shut down we might as well include the status from > pg_controldata in the errormessage as per the tr

Re: Inefficiency in parallel pg_restore with many tables

2023-07-18 Thread Alvaro Herrera
On 2023-Jul-17, Nathan Bossart wrote: > @@ -35,7 +42,11 @@ binaryheap_allocate(int capacity, binaryheap_comparator > compare, void *arg) > binaryheap *heap; > > sz = offsetof(binaryheap, bh_nodes) + sizeof(Datum) * capacity; > +#ifdef FRONTEND > + heap = (binaryheap *) pg_malloc

Re: Inefficiency in parallel pg_restore with many tables

2023-07-18 Thread Nathan Bossart
On Tue, Jul 18, 2023 at 06:05:11PM +0200, Alvaro Herrera wrote: > On 2023-Jul-17, Nathan Bossart wrote: > >> @@ -35,7 +42,11 @@ binaryheap_allocate(int capacity, binaryheap_comparator >> compare, void *arg) >> binaryheap *heap; >> >> sz = offsetof(binaryheap, bh_nodes) + sizeof(Datum)

Re: Regarding Installation of PostgreSQL

2023-07-18 Thread David G. Johnston
You are still in the wrong place - this is a developers list, which is only slightly less bad than sending it to a security list. We have a "general" list if you really cannot find a better place to send stuff. But in this case your complaint has to do with the pgAdmin program so its support list

Re: harmonize password reuse in vacuumdb, clusterdb, and reindexdb

2023-07-18 Thread Gurjeet Singh
On Mon, Jul 17, 2023 at 1:47 PM Nathan Bossart wrote: > > Here is a new version of the patch in which I've updated this comment as > proposed. Gurjeet, do you have any other concerns about this patch? With the updated comment, the patch looks good to me. Best regards, Gurjeet http://Gurje.et

Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Tomas Vondra
On 7/18/23 15:52, Ashutosh Bapat wrote: > On Fri, Jul 14, 2023 at 7:33 PM Tomas Vondra > wrote: > >> >> Thanks for testing / confirming this! So, do we agree this behavior is >> reasonable? >> > > This behaviour doesn't need any on-disk changes or has nothing in it > which prohibits us from c

Re: Use of additional index columns in rows filtering

2023-07-18 Thread Jeff Davis
Hi, On Sun, 2023-07-16 at 22:36 +0200, Tomas Vondra wrote: > This kept bothering me, so I looked at it today, and reworked it to > use > the IOS approach. Initial comments on patch 20230716: * check_index_filter() alredy looks at "canreturn", which should mean that you don't need to later check

Re: [feature]COPY FROM enable FORCE_NULL/FORCE_NOT_NULL on all columns

2023-07-18 Thread Zhang Mingli
Hi, On Jul 9, 2023 at 11:51 +0800, Zhang Mingli , wrote: HI, Regards, Zhang Mingli On Jul 7, 2023, 18:00 +0800, Damir Belyalov , wrote: The patch does not work for the current version of postgres, it needs to be updated. I tested your patch. Everything looks simple and works well. The

Re: Use of additional index columns in rows filtering

2023-07-18 Thread Tomas Vondra
On 7/18/23 22:21, Jeff Davis wrote: > Hi, > > > On Sun, 2023-07-16 at 22:36 +0200, Tomas Vondra wrote: >> This kept bothering me, so I looked at it today, and reworked it to >> use >> the IOS approach. > > Initial comments on patch 20230716: > > * check_index_filter() alredy looks at "canreturn

Re: [PoC] Federated Authn/z with OAUTHBEARER

2023-07-18 Thread Thomas Munro
On Tue, Jul 18, 2023 at 11:55 AM Jacob Champion wrote: > We're not setting EV_RECEIPT for these -- is that because none of the > filters we're using are EV_CLEAR, and so it doesn't matter if we > accidentally pull pending events off the queue during the kevent() call? +1 for EV_RECEIPT ("just tel

Re: Use of additional index columns in rows filtering

2023-07-18 Thread Jeff Davis
On Wed, 2023-07-19 at 00:36 +0200, Tomas Vondra wrote: > > * I'm confused about the relationship of an IOS to an index filter. > > It > > seems like the index filter only works for an ordinary index scan? > > Why > > is that? > > What would it do for IOS? The way it's presented is slightly confus

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-18 Thread Andres Freund
Hi, I wanted this feature a couple times before... On 2023-07-03 13:56:29 +0530, Palak Chaturvedi wrote: > +PG_FUNCTION_INFO_V1(pg_buffercache_invalidate); > +Datum > +pg_buffercache_invalidate(PG_FUNCTION_ARGS) I don't think "invalidating" is the right terminology. Note that we already have In

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-18 Thread Thomas Munro
On Wed, Jul 19, 2023 at 12:45 PM Andres Freund wrote: > I don't think "invalidating" is the right terminology. Note that we already > have InvalidateBuffer() - but it's something we can't allow users to do, as it > throws away dirty buffer contents (it's used for things like dropping a > table). >

Re: pg_recvlogical prints bogus error when interrupted

2023-07-18 Thread Michael Paquier
On Mon, Jul 10, 2023 at 01:44:45PM +0900, Michael Paquier wrote: > As StreamLogicalLog() states once it leaves its main loop because > time_to_abort has been switched to true, we want a clean exit. I > think that this patch is just a more complicated way to avoid doing > twice the operations done

FATAL: operator class "xxxx" does not exist for access method "btree"

2023-07-18 Thread mao zhang
Dear pgsql: When we adding a custom system table and defining an index for it at the same time, the code compilation is possible, but the following errors may occur when starting the database: -

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2023-07-18 Thread Michael Paquier
On Sun, Jul 16, 2023 at 09:37:28PM -0500, Justin Pryzby wrote: > I understand that it's possible for it to be conditional, but I don't > undertand why it's desirable/important ? Because it's cheaper on repeated commands, like no CCI necessary. > It's not conditional in the tablespace code that th

FATAL: operator class "xxxx" does not exist for access method "btree"

2023-07-18 Thread mao zhang
Dear pgsql: When we adding a custom system table and defining an index for it at the same time, the code compilation is possible, but the following errors may occur when starting the database: -

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread Peter Smith
Some review comments for v19-0001 == src/backend/replication/logical/tablesync.c 1. run_tablesync_worker +run_tablesync_worker(WalRcvStreamOptions *options, + char *slotname, + char *originname, + int originname_size, + XLogRecPtr *origin_startpos) +{ + /* Start table synchronization. */ + st

Re: FATAL: operator class "xxxx" does not exist for access method "btree"

2023-07-18 Thread Tom Lane
mao zhang writes: > running bootstrap script ... 2023-07-19 09:40:47.083 CST [2808392] FATAL: > operator class "key_ops" does not exist for access method "btree" I'm not sure what you find so mysterious about that error message. > Oid global_key_id;

Re: Generating code for query jumbling through gen_node_support.pl

2023-07-18 Thread Michael Paquier
On Tue, Jul 11, 2023 at 07:35:43AM +0900, Michael Paquier wrote: > I still don't think that we need both methods based on these numbers, > but there may be more opinions about that? Are people OK if this open > item is discarded? Hearing nothing about this point, removed from the open item list,

Re: logicalrep_message_type throws an error

2023-07-18 Thread Amit Kapila
On Tue, Jul 18, 2023 at 10:27 AM Masahiko Sawada wrote: > > On Tue, Jul 18, 2023 at 12:15 PM Amit Kapila wrote: > > > > On Mon, Jul 17, 2023 at 7:54 PM Alvaro Herrera > > wrote: > > > > > > > I have tried to check whether we have such usage in any other error > > callbacks. Though I haven't scr

Re: harmonize password reuse in vacuumdb, clusterdb, and reindexdb

2023-07-18 Thread Zhang Mingli
HI, On Jun 29, 2023 at 13:24 +0800, Nathan Bossart , wrote: > > Connecting to different databases with the same > host/port/user information seems okay. Have a look, yeah, cluster_all_databases/vacuum_all_databases/reindex_all_databases will get there. LGTM. Regards, Zhang Mingli

Re: Support to define custom wait events for extensions

2023-07-18 Thread Masahiro Ikeda
Hi, I updated the patches. * v6-0001-Support-custom-wait-events-for-extensions.patch The main diffs are * rebase it atop current HEAD * update docs to show users how to use the APIs * rename of functions and variables * fix typos * define a new spinlock in shared memory for this purpose * outpu

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-07-18 Thread Amit Kapila
On Wed, Jul 19, 2023 at 8:38 AM Peter Smith wrote: > > Some review comments for v19-0001 > ... > == > src/backend/replication/logical/worker.c > > 3. set_stream_options > > +/* > + * Sets streaming options including replication slot name and origin start > + * position. Workers need these opti

Re: logicalrep_message_type throws an error

2023-07-18 Thread Ashutosh Bapat
On Wed, Jul 19, 2023 at 9:01 AM Amit Kapila wrote: > > On Tue, Jul 18, 2023 at 10:27 AM Masahiko Sawada > wrote: > > > > On Tue, Jul 18, 2023 at 12:15 PM Amit Kapila > > wrote: > > > > > > On Mon, Jul 17, 2023 at 7:54 PM Alvaro Herrera > > > wrote: > > > > > > > > > > I have tried to check w

Re: Allow pg_archivecleanup to remove backup history files

2023-07-18 Thread Michael Paquier
On Fri, Jun 30, 2023 at 03:48:43PM +0900, Michael Paquier wrote: > I have begun cleaning up my board, and applied 0001 for the moment. And a few weeks later.. I have come around this thread and applied 0002 and 0003. The flow of 0002 was straight-forward. My main issue was in 0003, actually, wh

Re: doc: clarify the limitation for logical replication when REPILICA IDENTITY is FULL

2023-07-18 Thread Amit Kapila
On Mon, Jul 17, 2023 at 11:51 AM Önder Kalacı wrote: > >> > >> > The last line seems repetitive to me. So, I have removed it. Apart >> > from that patch looks good to me. Sergie, Peter, and others, any >> > thoughts? >> >> The v5 patch LGTM. >> > > Overall looks good to me as well. Please consider

Re: logical decoding and replication of sequences, take 2

2023-07-18 Thread Ashutosh Bapat
On Wed, Jul 19, 2023 at 1:20 AM Tomas Vondra wrote: > >> > > > > This behaviour doesn't need any on-disk changes or has nothing in it > > which prohibits us from changing it in future. So I think it's good as > > a v0. If required we can add the protocol option to provide more > > flexible behavio

Re: Support to define custom wait events for extensions

2023-07-18 Thread Masahiro Ikeda
On 2023-07-19 12:52, Masahiro Ikeda wrote: Hi, I updated the patches. * v6-0001-Support-custom-wait-events-for-extensions.patch I updated the patch since the cfbot found a bug. * v7-0001-Support-custom-wait-events-for-extensions.patch Regards, -- Masahiro Ikeda NTT DATA CORPORATIONFrom e5d639

Re: Support to define custom wait events for extensions

2023-07-18 Thread Michael Paquier
On Wed, Jul 19, 2023 at 12:52:10PM +0900, Masahiro Ikeda wrote: > I would like to change the wait event name of contrib modules for example > postgres_fdw. But, I think it's better to do so after the APIs are > committed. Agreed to do things one step at a time here. Let's focus on the core APIs a

Re: Flush SLRU counters in checkpointer process

2023-07-18 Thread Anthonin Bonnefoy
I think I've managed to reproduce the issue. The test I've added to check slru flush was the one failing in the regression suite. SELECT SUM(flushes) > :slru_flushes_before FROM pg_stat_slru; ?column? -- t The origin seems to be a race condition on have_slrustats ( https://github.com/po

Re: pg_recvlogical prints bogus error when interrupted

2023-07-18 Thread Bharath Rupireddy
On Wed, Jul 19, 2023 at 8:04 AM Michael Paquier wrote: > > It took me some time to come back to this one, but attached is what I > had in mind. This stuff has three reasons to stop: keepalive, end LSN > or signal. This makes the code easier to follow. > > Thoughts or comments? Thanks. I have so

Do we want to enable foreign key constraints on subscriber?

2023-07-18 Thread Kyotaro Horiguchi
Hello. There's an issue brought up in the -bugs list [1]. Since triggers are deactivated on a subscriber by default, foreign key constraints don't fire for replicated changes. The docs state this is done to prevent repetitive data propagation between tables on subscribers. But foreign key triggers