Re: Regression tests fail on OpenBSD due to low semmns value

2024-12-24 Thread Alexander Lakhin
Hello Tom, 16.12.2024 07:23, Tom Lane wrote: Alexander Lakhin writes: ... So GetSafeSnapshot() waits indefinitely for possibleUnsafeConflicts to become empty (for other backend to remove itself from the list of possible conflicts inside ReleasePredicateLocks()), but it doesn't happen. This s

RFC: Allow EXPLAIN to Output Page Fault Information

2024-12-24 Thread torikoshia
Hi, When reading the output of EXPLAIN (ANALYZE) to diagnose slow queries for our customers, I often want to know how many page faults occurred, especially major page faults, which involve disk access. Currently, the BUFFERS option in EXPLAIN provides information on whether a page was found

Re: Object identifier types in logical replication binary mode

2024-12-24 Thread Shlok Kyal
Hi, On Thu, 14 Nov 2024 at 21:22, Emre Hasegeli wrote: > > I encountered a problem with logical replication. > > The object identifier types, regclass, regproc, regtype, etc. are > transferred as an oid in the binary mode. However, the subscriber > expects them as text which makes sense because

An improvement of ProcessTwoPhaseBuffer logic

2024-12-24 Thread Vitaly Davydov
Dear Hackers, I would like to discuss ProcessTwoPhaseBuffer function. It reads two-phase transaction states from disk or the WAL. It takes xid as well as some other input parameters and executes the following steps: Step #1: Check if xid is committed or aborted in clog (TransactionIdDidCommit,

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-24 Thread Nisha Moond
On Fri, Dec 20, 2024 at 3:12 PM Amit Kapila wrote: > On Mon, Dec 16, 2024 at 4:10 PM Nisha Moond > wrote: > > > > Here is the v56 patch set with the above comments incorporated. > > > > Review comments: > === > 1. > + { > + {"idle_replication_slot_timeout", PGC_SIGHUP, REPLICATION_SE

Re: transaction lost when delete clog file after normal shutdown

2024-12-24 Thread wenhui qiu
Hi Michael Paquier Thank you for the information you provided, Thanks On Tue, 24 Dec 2024 at 13:13, Michael Paquier wrote: > On Tue, Dec 24, 2024 at 09:55:09AM +0800, wenhui qiu wrote: > > However, on the other hand, oracle has many solutions to open the > database > > after the data fil

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-24 Thread Michail Nikolaev
Hello everyone! Yesterday I got a strange set of test errors, probably somehow related to that patch. It happened on changed master branch (based on d96d1d5152f30d15678e08e75b42756101b7cab6) but I don't think my changes were affecting it. My setup is a little bit tricky: Windows 11 run WSL2 with

Re: remove pgrminclude?

2024-12-24 Thread Peter Eisentraut
On 09.12.24 18:20, Tom Lane wrote: Peter Eisentraut writes: I propose to remove the pgrminclude scripts and annotations. AFAICT, per git log, the last time someone tried to do something with it was around 2011. Also, many (not all) of the "pgrminclude ignore" annotations are of a newer date b

Re: stored procedures vs pg_stat_statements

2024-12-24 Thread Merlin Moncure
On Mon, Dec 23, 2024 at 11:01 PM Michael Paquier wrote: > On Mon, Dec 23, 2024 at 10:06:58PM -0600, Merlin Moncure wrote: > > I'm aware of that and will set it -- it's the only option if I'm > following > > you. The way I've been doing things lately for bulk processing is a lot > > of orchestra

ERROR: corrupt MVNDistinct entry

2024-12-24 Thread Richard Guo
I ran into this error in estimate_multivariate_ndistinct, and it can be reproduced with the query below. create table t (a int, b int); insert into t select 1, 1; create statistics s (ndistinct) on a, b from t; analyze; explain select 1 from t t1 left join (select a c1, coalesce(a) c2 from t

Re: Exists pull-up application with JoinExpr

2024-12-24 Thread Ranier Vilela
Hi Alena. Em ter., 24 de dez. de 2024 às 01:44, Alena Rybakina < a.rybak...@postgrespro.ru> escreveu: > Hi, hackers! > > I found one pull-up that works if the inner join condition is written > through the where condition, > > create temp table ta (id int primary key, val int); > insert into ta va

Re: Fix logging for invalid recovery timeline

2024-12-24 Thread David Steele
On 12/20/24 23:28, Andrey M. Borodin wrote: On 20 Dec 2024, at 20:37, David Steele wrote: "Latest checkpoint is at %X/%X on timeline %u, but in the history of the requested timeline, the server forked off from that timeline at %X/%X." I think errdetai here is very hard to follow. I seem to

Re: Object identifier types in logical replication binary mode

2024-12-24 Thread Emre Hasegeli
> I tested with both 'binary = true' and 'binary = false' option while > creating a subscription. For me replication is working fine and I am > not getting any errors in both the cases. > I have also attached the test script. I modified your test script to demonstrate the problem. I created anoth

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2024-12-24 Thread Tom Lane
torikoshia writes: > I have attached a PoC patch that modifies EXPLAIN to include page fault > information during both the planning and execution phases of a query. Surely these numbers would be too unstable to be worth anything. regards, tom lane

Re: Listen for all channel notifications

2024-12-24 Thread trey
On 2024-12-24 13:09, Trey Boudreau wrote: Based on [1], please find attached an implementation for listening for notifications on all channels. The first two chunks lay some groundwork and do not change the current LISTEN behavior, except to improve performance of managing large numbers of cha

Re: Proposal to add a new URL data type.

2024-12-24 Thread Robert Treat
On Thu, Dec 19, 2024 at 7:52 AM Victor Yegorov wrote: > ср, 11 дек. 2024 г. в 19:04, Alexander Borisov : >> >> > I've created a commitfest entry for the patch: https:// >> > commitfest.postgresql.org/51/5432/ > > commitfest.postgresql.org/51/5432/> >> > I was not able to find you, please, register

Short-living Memory Context in the optimiser

2024-12-24 Thread Andrei Lepikhov
Hi, Here is a rebased and slightly commented Tom's patch separated from the discussion [1]. I still occasionally see that planner peak memory consumption is triggered by selectivity estimation of massive arrays and clauses generated for multiple partitions. I think that in case of a growing

Re: An improvement of ProcessTwoPhaseBuffer logic

2024-12-24 Thread Michael Paquier
On Tue, Dec 24, 2024 at 04:26:32PM +0300, Vitaly Davydov wrote: > In some, very rare scenarios, the postgres instance will newer > recover because of such logic. Imagine, that the two_phase directory > contains some files with two-phase states of transactions of distant > future. I assume, it can h

Re: Parametrization minimum password lenght

2024-12-24 Thread Japin Li
On Thu, 19 Dec 2024 at 09:57, Nathan Bossart wrote: > On Thu, Dec 19, 2024 at 09:36:17AM -0600, Nathan Bossart wrote: >> On Thu, Dec 19, 2024 at 07:25:30AM +, Bertrand Drouvot wrote: >>> -errmsg("password is too short"))); >>> +

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2024-12-24 Thread Michael Paquier
On Tue, Dec 24, 2024 at 02:06:26PM +0100, Michail Nikolaev wrote: > Now STIR used for validation (but without resetting of snapshot during > that phase for now). Perhaps I am the only one, but what you are doing here is confusing. There is a dependency between one patch and the follow-up ones, bu

Re: Proposal: add new API to stringinfo

2024-12-24 Thread Michael Paquier
On Wed, Dec 25, 2024 at 12:37:04PM +0900, Tatsuo Ishii wrote: > Attached is a patch to implement it. In the patch I add two new APIs. > > extern StringInfo makeStringInfoWithSize(int size); > extern void initStringInfoWithSize(StringInfo str, int size); > > Maybe I could re-invent the wheel by co

Re: Short-living Memory Context in the optimiser

2024-12-24 Thread Michael Paquier
On Wed, Dec 25, 2024 at 11:56:24AM +0700, Andrei Lepikhov wrote: > Here is a rebased and slightly commented Tom's patch separated from the > discussion [1]. Seems to me that there is no patch? -- Michael signature.asc Description: PGP signature

Re: stored procedures vs pg_stat_statements

2024-12-24 Thread Michael Paquier
On Tue, Dec 24, 2024 at 08:23:57AM -0600, Merlin Moncure wrote: > Actually, I hadn't gotten that far yet; I was just noticing that: > CALL foo(1,2,3); > CALL foo(2,3,4); > ...resolved to different queryids and if that was expected, and if not, if > some logic tune-ups in the extension improve behav

Re: ERROR: corrupt MVNDistinct entry

2024-12-24 Thread Andrei Lepikhov
On 12/24/24 15:00, Richard Guo wrote: Any thoughts? I have a couple of notes. 1. The nulling_relids provides us sensible information about possible nulls inside the input. We are not using it to estimate the number of such nulls for now. Does Your idea consist of obtaining 'clear' statistics

Re: Short-living Memory Context in the optimiser

2024-12-24 Thread Andrei Lepikhov
On 12/25/24 12:30, Michael Paquier wrote: On Wed, Dec 25, 2024 at 11:56:24AM +0700, Andrei Lepikhov wrote: Here is a rebased and slightly commented Tom's patch separated from the discussion [1]. Seems to me that there is no patch? -- Michael Pardon me, patch attached. -- regards, Andrei Lepi

RE: Object identifier types in logical replication binary mode

2024-12-24 Thread Hayato Kuroda (Fujitsu)
Dear Emre, > I modified your test script to demonstrate the problem. I could reproduce the error with your script, thanks. I also could reproduce the same error with the simplified version, see attached. I feel this is a normal behavior which can happen without the logical replication. I could

Re: ERROR: corrupt MVNDistinct entry

2024-12-24 Thread Richard Guo
On Tue, Dec 24, 2024 at 5:00 PM Richard Guo wrote: > It seems to me that when estimating the number of groups, we do not > need to concern ourselves with the outer joins that could null the > Vars/PHVs contained in the grouping expressions, and we should not > count the same Var more than once. F

Listen for all channel notifications

2024-12-24 Thread Trey Boudreau
Based on [1], please find attached an implementation for listeningfor notifications on all channels. The first two chunks lay somegroundwork and do not change the current LISTEN behavior, exceptto improve performance of managing large numbers of channels.v1-0001-Make-simplehash-more-flexible.patch

Re: Regression tests fail on OpenBSD due to low semmns value

2024-12-24 Thread Tom Lane
Alexander Lakhin writes: > I've reproduced this behavior with two reduced sqls. > prepared_xacts.sql: > BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE; >   CREATE TABLE pxtest4 (a int); > PREPARE TRANSACTION 'regress_sub2'; > \c - > COMMIT PREPARED 'regress_sub2'; > -- the script ends prematurely

Re: Enhancing Memory Context Statistics Reporting

2024-12-24 Thread Rahila Syed
Hi Tomas, > > I'd just remove that. I agree it might have been useful with the single > chunk of shared memory, but I think with separate chunks it's not very > useful. And if we can end up with multiple processed getting the same > pgprocno I guess we have way bigger problems, this won't fix tha

Proposal: add new API to stringinfo

2024-12-24 Thread Tatsuo Ishii
Currently the StringInfo package provides StringInfo object creation API with fixed length initial allocation size (1024 bytes). However, if we want to allocate much smaller size of initial allocation, this is waste of space. Background: While working on this: https://www.postgresql.org/message-id

Re: Logical Replication of sequences

2024-12-24 Thread vignesh C
On Fri, 20 Dec 2024 at 08:05, Peter Smith wrote: > > Hi Vignesh. > > Here are some review comments for the patch v20241211-0005. > > == > > Section "29.6.3. Examples" > > 2. > Should the Examples section also have an example of ALTER SUBSCRIPTION > ... REFRESH PUBLICATION to demonstrate (like

Re: Infinite loop in XLogPageRead() on standby

2024-12-24 Thread Michael Paquier
On Wed, Nov 13, 2024 at 02:18:06PM +0100, Alexander Kukushkin wrote: > Now that v17 is released and before v18 feature freeze we have a few > months, I hope you will find some time to look at it. My apologies for taking a couple of weeks before coming back to this thread. I have been informed a c

Autovacuum giving up on tables after crash because of lack of stats

2024-12-24 Thread Michael Paquier
Hi all, While digging again into the support for pgstats flushes across checkpoints, mainly to avoid autovacuum giving up on tables after a crash if stats entries cannot be found, I've been reminded about this point raised by Heikki: https://www.postgresql.org/message-id/54fdfc3e-74f6-4ea4-b844-05

stored short varlena in array

2024-12-24 Thread Quan Zongliang
Hi Now, the varlena type is stored directly in the array. Did not consider short varlena. If it's like fill_val(), using short varlena saves memory footprint and disk space. In TODO, there is a requirement to be implemented: Allow single-byte header storage for array elements This patch mo