Re: Enhancing Memory Context Statistics Reporting

2024-10-21 Thread Michael Paquier
On Mon, Oct 21, 2024 at 11:54:21PM +0530, Rahila Syed wrote: > On the other hand, [2] provides the statistics for all backends but logs > them in a file, which may not be convenient for quick access. To be precise, pg_log_backend_memory_contexts() pushes the memory context stats to LOG_SERVER_ONLY

Re: Fix C23 compiler warning

2024-10-21 Thread Tom Lane
Peter Eisentraut writes: > Committed with that change. Thanks. Should we back-patch this? (And also a67a49648d9?) It's not hard to imagine people wanting to compile our stable branches with C23 compilers. I might leave out v12, which is just days away from EOL, but this seems like a reasonabl

Re: Fix C23 compiler warning

2024-10-21 Thread Peter Eisentraut
On 20.10.24 17:56, Tom Lane wrote: Peter Eisentraut writes: This no longer works because in C23, because an empty argument list is now equivalent to (void), rather than an indeterminate one as before. And so this results in an incompatible function pointer type and compiler warnings. (gcc and

Re: Using read_stream in index vacuum

2024-10-21 Thread Andrey M. Borodin
> On 22 Oct 2024, at 00:05, Melanie Plageman wrote: > > I was suggesting you call RelationGetNumberOfBlocks() once > current_block == last_exclusive in the callback itself. Consider following sequence of events: 0. We schedule some buffers for IO 1. We call RelationGetNumberOfBlocks() in cal

Row pattern recognition

2024-10-21 Thread David G. Johnston
On Monday, October 21, 2024, Tatsuo Ishii wrote: > > I wonder how "PREV(col + 1)" is different from "PREV(col) + 1". > Currently my RPR implementation does not allow PREV(col + 1). If > "PREV(col + 1)" is different from "PREV(col) + 1", it maybe worthwhile > to implement "PREV(col + 1)". > Intere

Re: Set query_id for query contained in utility statement

2024-10-21 Thread Michael Paquier
On Tue, Oct 22, 2024 at 02:06:16PM +0900, Michael Paquier wrote: > I've looked at 0001, and finished by splitting the case of all-level > tracking with the multi-statements as the resulting table was feeling > heavily bloated, particularly because of MERGE that spawned in > multiple lines even if t

Re: Set query_id for query contained in utility statement

2024-10-21 Thread Michael Paquier
On Mon, Oct 21, 2024 at 10:35:17AM +0200, Anthonin Bonnefoy wrote: > I've updated 0001 to only use ORDER BY query. The query strings are > not exact doublons, as the nested statement has the additional ending > ';' due to using the whole string instead of just the RawStmt. Thus, > the other sort ex

Re: Row pattern recognition

2024-10-21 Thread Tatsuo Ishii
Hi, I wonder how "PREV(col + 1)" is different from "PREV(col) + 1". Currently my RPR implementation does not allow PREV(col + 1). If "PREV(col + 1)" is different from "PREV(col) + 1", it maybe worthwhile to implement "PREV(col + 1)". Best reagards, -- Tatsuo Ishii SRA OSS K.K. English: http://www

Re: cost delay brainstorming

2024-10-21 Thread Jay
I had suggested something more that just cost limit, throttling which would be re-startable vacuum - https://www.postgresql.org/message-id/CAPdcCKpvZiRCoDxQoo9mXxXAK8w=bx5nqdttgzvhv2suxp0...@mail.gmail.com . It may not be difficult to predict patterns of idle periods with cloud infrastructure and

Re: race condition in pg_class

2024-10-21 Thread Noah Misch
On Mon, Oct 21, 2024 at 10:00:00PM +0300, Alexander Lakhin wrote: > Please look at an anomaly introduced with a07e03fd8. > With the attached modification for intra-grant-inplace.spec, running this > test triggers a Valgrind-detected error for me: > ==00:00:00:09.624 319033== Conditional jump or mov

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-10-21 Thread Tender Wang
Alvaro Herrera 于2024年10月22日周二 05:52写道: > On 2024-Oct-21, Tender Wang wrote: > > > I suspect that we don't need the below if > > statement anymore. > > /* > > * If the referenced side is partitioned (which we know because our > > * parent's constraint points to a different relation than ours) then

Re: Better error reporting from extension scripts (Was: Extend ALTER OPERATOR)

2024-10-21 Thread Pavel Stehule
Hi pá 11. 10. 2024 v 19:39 odesílatel Pavel Stehule napsal: > > > pá 11. 10. 2024 v 18:08 odesílatel Tom Lane napsal: > >> Pavel Stehule writes: >> > I tested it and it is working nicely. I tested it against Orafce and I >> > found an interesting point. The body of plpgsql functions is not >>

Re: EXPLAIN IndexOnlyScan shows disabled when enable_indexonlyscan=on

2024-10-21 Thread David G. Johnston
On Monday, October 21, 2024, David Rowley wrote: > On Tue, 22 Oct 2024 at 14:46, David G. Johnston > wrote: > > We should probably at least improve the documentation in 19.17.1; this > interaction is apparently not self-evident. > > > > enable_indexscan > > > > Enable or disable the planner’s us

Re: EXPLAIN IndexOnlyScan shows disabled when enable_indexonlyscan=on

2024-10-21 Thread David Rowley
On Tue, 22 Oct 2024 at 14:46, David G. Johnston wrote: > We should probably at least improve the documentation in 19.17.1; this > interaction is apparently not self-evident. > > enable_indexscan > > Enable or disable the planner’s use of both index-scan and index-only-scans > plan types. > > ena

EXPLAIN IndexOnlyScan shows disabled when enable_indexonlyscan=on

2024-10-21 Thread David G. Johnston
On Monday, October 21, 2024, David Rowley wrote: > On Tue, 22 Oct 2024 at 13:45, Melanie Plageman > wrote: > > I was surprised today when I saw that with > > enable_indexscan=off > > enable_indexonlyscan=on > > Robert did propose to change this behaviour while he was working on > the disabled_no

Re: EXPLAIN IndexOnlyScan shows disabled when enable_indexonlyscan=on

2024-10-21 Thread David Rowley
On Tue, 22 Oct 2024 at 13:45, Melanie Plageman wrote: > I was surprised today when I saw that with > enable_indexscan=off > enable_indexonlyscan=on > EXPLAIN prints that the index only scan is disabled: > > QUERY PLAN > --- > Index Onl

Re: Pgoutput not capturing the generated columns

2024-10-21 Thread Peter Smith
Hi SHubham, Here are my review comments for v40-0001 (code) Please don't post a blanket response of "I have fixed all the comments" response to this. Sometimes things get missed. Instead, please reply done/not-done/whatever individually, so I can track the changes properly. == src/backend/cat

More CppAsString2() in psql's describe.c

2024-10-21 Thread Michael Paquier
Hi all, This was on my stack of things for some time, but please find attached a patch to clean up some code with $subject because HEAD's describe.c is a mixed bag of relying on CppAsString2() and hardcoded values. Switching to CppAsString2() has the advantage to make the code more self-documented

EXPLAIN IndexOnlyScan shows disabled when enable_indexonlyscan=on

2024-10-21 Thread Melanie Plageman
I was surprised today when I saw that with enable_indexscan=off enable_indexonlyscan=on EXPLAIN prints that the index only scan is disabled: QUERY PLAN --- Index Only Scan using history_pkey on history Disabled: true I wasn't sure

RE: Statistics Import and Export

2024-10-21 Thread Shinoda, Noriyoshi (SXD Japan FSIP)
> 1. Allow relpages to be set to -1 (partitioned tables with partitions have > this value after ANALYZE). > 2. Turn off autovacuum on tables (where possible) if they are going to be the > target of pg_set_relation_stats(). > 3. Allow pg_set_relation_stats to continue past an out-of-range detectio

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-10-21 Thread Alvaro Herrera
On 2024-Oct-21, Tender Wang wrote: > I suspect that we don't need the below if > statement anymore. > /* > * If the referenced side is partitioned (which we know because our > * parent's constraint points to a different relation than ours) then > * we must, in addition to the above, create pg_cons

Re: Using read_stream in index vacuum

2024-10-21 Thread Melanie Plageman
On Mon, Oct 21, 2024 at 4:49 PM Andrei Borodin wrote: > > 21.10.2024, 22:34, "Melanie Plageman" : > > The whole point of the read stream callback provided by the caller is > that the logic to get the next block should be there > > We must get number of blocks after examining last block. But callba

Re: Using read_stream in index vacuum

2024-10-21 Thread Andrei Borodin
Melanie, thanks for your comments.21.10.2024, 22:34, "Melanie Plageman" :The whole point of the read stream callback provided by the caller isthat the logic to get the next block should be thereWe must get number of blocks after examining last block. But callback returning EOF might be called befor

RE: msvc directory missing in PostgreSQL 17.0

2024-10-21 Thread Mark Hill
Thanks Bill! Do you have a sample meson command for building that you could share? Thanks, Mark From: Bill Smith Sent: Friday, October 18, 2024 4:11 PM To: Mark Hill Cc: pgsql-hackers@lists.postgresql.org Subject: Re: msvc directory missing in PostgreSQL 17.0 EXTERNAL On Oct 18, 2024, a

Re: Using read_stream in index vacuum

2024-10-21 Thread Melanie Plageman
On Mon, Oct 21, 2024 at 3:34 PM Melanie Plageman wrote: > > For the outer loop, I feel like we have options. For example, maybe > the read stream callback can call RelationGetNumberOfBlocks(). I mean > maybe we don't want to have to take a relation extension lock in a > callback. Also, given thi

Re: Using read_stream in index vacuum

2024-10-21 Thread Melanie Plageman
On Sun, Oct 20, 2024 at 10:19 AM Andrey M. Borodin wrote: > > > > > On 20 Oct 2024, at 15:16, Junwang Zhao wrote: > > > > I'm not sure if I did not express myself correctly, I didn't mean to > > restart the stream, > > I mean we can create a new stream for each outer loop, I attached a > > refact

Re: race condition in pg_class

2024-10-21 Thread Alexander Lakhin
Hello Noah, 25.09.2024 01:43, Noah Misch wrote: Pushed, but the pushes contained at least one defect: Please look at an anomaly introduced with a07e03fd8. With the attached modification for intra-grant-inplace.spec, running this test triggers a Valgrind-detected error for me: ==00:00:00:09.62

Enhancing Memory Context Statistics Reporting

2024-10-21 Thread Rahila Syed
Hi, PostgreSQL provides following capabilities for reporting memory contexts statistics. 1. pg_get_backend_memory_contexts(); [1] 2. pg_log_backend_memory_contexts(pid); [2] [1] provides a view of memory context statistics for a local backend, while [2] prints the memory context statistics of an

Re: Using Expanded Objects other than Arrays from plpgsql

2024-10-21 Thread Michel Pelletier
On Sun, Oct 20, 2024 at 8:46 PM Tom Lane wrote: > Michel Pelletier writes: > > On Sun, Oct 20, 2024 at 10:13 AM Tom Lane wrote: > (from thread https://www.postgresql.org/message-id/CACxu%3DvJaKFNsYxooSnW1wEgsAO5u_v1XYBacfVJ14wgJV_PYeg%40mail.gmail.com ) > >> But it seems like we could get an

Re: [BUG?] check_exclusion_or_unique_constraint false negative

2024-10-21 Thread Michail Nikolaev
Hello, Hou! I have sent [0] reproducer within the context of conflict detection and resolution to the original thread. [0]: https://www.postgresql.org/message-id/flat/CANtu0ojMjAwMRJK%3DH8y0YBB0ZEcN%2BJbdZeoXQn8dWO5F67jgsA%40mail.gmail.com#f5d1baf4702685aedf23daa9addc012e >

Re: [PATCH] Add array_reverse() function

2024-10-21 Thread Tom Lane
Ashutosh Bapat writes: > On Mon, Oct 21, 2024 at 2:36 PM Aleksander Alekseev > wrote: > + /* > + * There is no point in reversing empty arrays or arrays with less than > + * two items. > + */ > + if (ARR_NDIM(array) < 1 || ARR_DIMS(array)[0] < 2) > + PG_RETURN_ARRAYTYPE_P(array); > But it re

Re: Make default subscription streaming option as Parallel

2024-10-21 Thread vignesh C
On Mon, 21 Oct 2024 at 14:36, Amit Kapila wrote: > > On Mon, Oct 7, 2024 at 11:05 AM vignesh C wrote: > > > > The tests demonstrate a significant performance improvement when using > > the parallel streaming option, insert shows 40-48 %improvement, delete > > shows 34-39 %improvement, update show

Re: New "raw" COPY format

2024-10-21 Thread Joel Jacobson
On Mon, Oct 21, 2024, at 16:35, jian he wrote: > make the ProcessCopyOptions process in following order: > 1. Extract options from the statement node tree > 2. checking each option, if not there set default value. > 3. checking for interdependent options > > I still think > making step2 aligned wit

Re: [PATCH] Add array_reverse() function

2024-10-21 Thread Joel Jacobson
On Mon, Oct 21, 2024, at 11:06, Aleksander Alekseev wrote: > Hi, > > Recently I wanted to call array_reverse() and discovered that we still > don't have it. I'm not the first one who encountered this limitation. > array_reverse() was requested at least since 2009 [1] and the > workaround on Postgre

Re: New "raw" COPY format

2024-10-21 Thread jian he
On Sat, Oct 19, 2024 at 11:33 PM Joel Jacobson wrote: > > > ProcessCopyOptions > > /* Extract options from the statement node tree */ > > foreach(option, options) > > { > > } > > /* --- DELIMITER option --- */ > > /* --- NULL option --- */ > > /* --- QUOTE option --- */ > > Currently the regress t

Re: ECPG Refactor: move sqlca variable in ecpg_log()

2024-10-21 Thread Fujii Masao
On 2024/10/19 2:43, Tom Lane wrote: Fujii Masao writes: I've attached the latest version of the patch, now including the commit log. Unless there are any objections, I'll proceed with committing it. LGTM. Maybe move down the sqlca variable declaration, so that the declarations still match

Re: [PATCH] Add array_reverse() function

2024-10-21 Thread Ashutosh Bapat
On Mon, Oct 21, 2024 at 2:36 PM Aleksander Alekseev wrote: > > Hi, > > Recently I wanted to call array_reverse() and discovered that we still > don't have it. I'm not the first one who encountered this limitation. > array_reverse() was requested at least since 2009 [1] and the > workaround on Post

Re: Question about VACUUM behavior with sub-transactions in stored procedures

2024-10-21 Thread David G. Johnston
On Monday, October 21, 2024, Кириллов Вячеслав wrote: > I have a question regarding the behavior of the auto VACUUM in PostgreSQL > in the context of using stored procedures with sub-transactions. > > This is a general usage inquiry not suited to discussion on -hackers. We have a -general mailin

Question about VACUUM behavior with sub-transactions in stored procedures

2024-10-21 Thread Кириллов Вячеслав
Hi hackers,, I have a question regarding the behavior of the auto VACUUM in PostgreSQL in the context of using stored procedures with sub-transactions. As I understand it, the parameters that control the initiation of VACUUM are set in the configuration file, such as autovacuum_vacuum_threshol

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2024-10-21 Thread Fujii Masao
On 2024/10/21 18:30, Kirill Reshke wrote: v4 no longer applies. It now conflicts with e7834a1a251d4a28245377f383ff20a657ba8262. Also, there were review comments. So, I decided to rebase. Thanks for the patch! Here are my review comments: I noticed that on_error=set_to_null does not trigger

Re: Misleading error "permission denied for table"

2024-10-21 Thread Ashutosh Bapat
On Wed, Oct 16, 2024 at 10:11 PM Tom Lane wrote: > > Nathan Bossart writes: > > On Wed, Oct 16, 2024 at 07:36:29PM +0530, Ashutosh Bapat wrote: > >> Shouldn't we report "permission defined for column atest5.three? > > > We do have "permission denied for column" messages in aclchk.c (e.g., > > acl

[PATCH] Add array_reverse() function

2024-10-21 Thread Aleksander Alekseev
Hi, Recently I wanted to call array_reverse() and discovered that we still don't have it. I'm not the first one who encountered this limitation. array_reverse() was requested at least since 2009 [1] and the workaround on PostgreSQL Wiki is dated 2010 [2]. The proposed patch adds this function. On

Re: Add pg_ownerships and pg_privileges system views

2024-10-21 Thread Joel Jacobson
On Mon, Oct 21, 2024, at 11:42, Alvaro Herrera wrote: > ... actually, the "AND pg_shdepend.dbid = ( SELECT pg_database_1.oid > ...)" part of this is useless, because you already had that in the ON > condition of the original join to pg_database. So, apologies for the > noise there. TBH I don't se

Re: type cache cleanup improvements

2024-10-21 Thread Alexander Korotkov
On Mon, Oct 21, 2024 at 1:16 PM Dagfinn Ilmari Mannsåker wrote: > Alexander Korotkov writes: > > > On Mon, Oct 21, 2024 at 8:40 AM Andrei Lepikhov wrote: > >> > >> On 21/10/2024 06:32, Dagfinn Ilmari Mannsåker wrote: > >> > Alexander Korotkov writes: > >> > > >> >> +static Oid *in_progress_list

Re: Pgoutput not capturing the generated columns

2024-10-21 Thread Amit Kapila
On Fri, Oct 18, 2024 at 5:42 PM Shubham Khanna wrote: > > > > > > I have fixed all the given comments. The attached patches contain the > > > required changes. Review comments: === 1. > B. when generated columns are not published * Publisher not-generated column => subscriber not-gen

Re: type cache cleanup improvements

2024-10-21 Thread Dagfinn Ilmari Mannsåker
Alexander Korotkov writes: > On Mon, Oct 21, 2024 at 8:40 AM Andrei Lepikhov wrote: >> >> On 21/10/2024 06:32, Dagfinn Ilmari Mannsåker wrote: >> > Alexander Korotkov writes: >> > >> >> +static Oid *in_progress_list; >> >> +static int in_progress_list_len; >> >> +static int in_progress_list_m

Re: Make default subscription streaming option as Parallel

2024-10-21 Thread Peter Smith
On Mon, Oct 21, 2024 at 5:09 PM Amit Kapila wrote: > > On Mon, Oct 7, 2024 at 4:03 PM vignesh C wrote: > > > > On Mon, 7 Oct 2024 at 12:26, Hayato Kuroda (Fujitsu) > > wrote: > > > > > One comment for your patch; > > > Shouldn't we add a streaming=off case for pg_dump test? You added lines > >

Re: commitfest.postgresql.org Specify thread msgid does not work for pgsql-bugs(at)lists(dot)postgresql(dot)org

2024-10-21 Thread Alvaro Herrera
On 2024-Oct-21, jian he wrote: > hi. > https://commitfest.postgresql.org/50/new/# > won't work for messages from pgsql-bugs > . > > recently, I try to register an entry for > https://www.postgresql.org/message-id/tencent_BAB00BBF45A08D941DF8290D9533941BB807%40qq.com > but cannot. > > see picture

Re: Make default subscription streaming option as Parallel

2024-10-21 Thread Amit Kapila
On Mon, Oct 7, 2024 at 11:05 AM vignesh C wrote: > > The tests demonstrate a significant performance improvement when using > the parallel streaming option, insert shows 40-48 %improvement, delete > shows 34-39 %improvement, update shows 26-30 %improvement. In the case > of rollback the improvemen

Re: Add pg_ownerships and pg_privileges system views

2024-10-21 Thread Alvaro Herrera
On 2024-Oct-20, Alvaro Herrera wrote: > SELECT > pg_shdepend.classid, > pg_shdepend.objid, > pg_shdepend.objsubid, > identify.*, > aclexplode.* > FROM pg_catalog.pg_shdepend > JOIN pg_catalog.pg_database ON pg_data

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2024-10-21 Thread Kirill Reshke
Hi! On Thu, 12 Sept 2024 at 15:13, jian he wrote: > > On Mon, Sep 9, 2024 at 10:34 PM Jim Jones wrote: > > > > > > Hi there > > > > On 26.08.24 02:00, jian he wrote: > > > hi all. > > > patch updated. > > > simplified the code a lot. > > > > > > idea is same: > > > COPY t_on_error_null FROM STDI

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-10-21 Thread Tender Wang
Alvaro Herrera 于2024年10月18日周五 22:52写道: > On 2024-Sep-26, Jehan-Guillaume de Rorthais wrote: > > > REL_14_STABLE backport doesn't seem trivial, so I'll wait for some > > feedback, review & decision before going further down in backpatching. > > Hi, thanks for these patches. I have made some edits

Re: type cache cleanup improvements

2024-10-21 Thread Alexander Korotkov
On Mon, Oct 21, 2024 at 10:51 AM jian he wrote: > > thanks for the > INJECTION_POINT("typecache-before-rel-type-cache-insert"); > Now I have better understanding of the whole changes. > > > +/* > + * Add possibly missing RelIdToTypeId entries related to TypeCacheHas > + * entries, marked as in-pro

Re: type cache cleanup improvements

2024-10-21 Thread Alexander Korotkov
On Mon, Oct 21, 2024 at 8:40 AM Andrei Lepikhov wrote: > > On 21/10/2024 06:32, Dagfinn Ilmari Mannsåker wrote: > > Alexander Korotkov writes: > > > >> +static Oid *in_progress_list; > >> +static int in_progress_list_len; > >> +static int in_progress_list_maxlen; > > > > Is there any particular

Re: type cache cleanup improvements

2024-10-21 Thread jian he
thanks for the INJECTION_POINT("typecache-before-rel-type-cache-insert"); Now I have better understanding of the whole changes. +/* + * Add possibly missing RelIdToTypeId entries related to TypeCacheHas + * entries, marked as in-progress by lookup_type_cache(). It may happen + * in case of an er

Re: Refactor GetLockStatusData() by skipping unused backends and groups

2024-10-21 Thread Bertrand Drouvot
Hi, On Mon, Oct 21, 2024 at 09:19:49AM +0900, Fujii Masao wrote: > Hi, > > While reading the fast-path lock code, I noticed that GetLockStatusData() > checks all slots for every backend to gather fast-path lock data. However, > backends with PID=0 don't hold fast-path locks, right? I think the s