RE: Parallel heap vacuum

2024-12-10 Thread Hayato Kuroda (Fujitsu)
Dear Tomas, > 1) I really like the idea of introducing "compute_workers" callback to > the heap AM interface. I faced a similar issue with calculating workers > for index builds, because right now plan_create_index_workers is doing > that the logic works for btree, but really not brin etc. It didn

Re: Wrong results with right-semi-joins

2024-12-10 Thread Richard Guo
On Mon, Dec 9, 2024 at 11:01 PM Melanie Plageman wrote: > Thanks for finding and fixing this. Just for my own benefit, could you > explain more about the minimal repro? Specifically, if you just need a > subplan in the hash side of a right semi-join, why do you also need > the outer part of the qu

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-10 Thread Peter Smith
Hi Nisha. Here are some review comments for patch v54-0002. (I had also checked patch v54-0001, but have no further review comments for that one). == doc/src/sgml/config.sgml 1. + +Slot invalidation due to idle timeout occurs during checkpoint. +If the checkpoint_time

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-10 Thread Dilip Kumar
On Wed, Dec 11, 2024 at 3:18 AM Masahiko Sawada wrote: > > On Mon, Dec 9, 2024 at 10:19 PM Dilip Kumar wrote: > > > > > > If the largest transaction is non-streamable, won't the transaction > > > returned by ReorderBufferLargestTXN() in the other case already > > > suffice the need? > > > > I se

Re: long-standing data loss bug in initial sync of logical replication

2024-12-10 Thread Shlok Kyal
On Tue, 8 Oct 2024 at 11:11, Shlok Kyal wrote: > > Hi Kuroda-san, > > > > I have also modified the tests in 0001 patch. These changes are only > > > related to syntax of writing tests. > > > > LGTM. I found small improvements, please find the attached. > > I have applied the changes and updated th

Re: [PoC] Reducing planning time when tables have many partitions

2024-12-10 Thread Yuya Watari
Hello Ashutosh and Alvaro, I appreciate you replying to the email. On Tue, Dec 3, 2024 at 7:38 PM Alvaro Herrera wrote: > > I don't think planning time in assert-enabled builds is something we > should worry about, at all. Planning time in production builds is the > important one. Thank you fo

Re: Conflict detection for update_deleted in logical replication

2024-12-10 Thread Nisha Moond
Here are the test steps and analysis for epoch-related handling (Tested with v15 Patch-Set). In the 'update_deleted' detection design, the launcher process compares XIDs to track minimum XIDs, and the apply workers maintain the oldest running XIDs. The launcher also requests publisher status at re

Memory leak in pg_logical_slot_{get,peek}_changes

2024-12-10 Thread vignesh C
Hi, I'm starting a new thread for one of the issues reported by Sawada-san at [1]. This is a memory leak on CacheMemoryContext when using pgoutput via SQL APIs: /* Map must live as long as the session does. */ oldctx = MemoryContextSwitchTo(CacheMemoryContext); entry->attrmap = build_attrmap_by_

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-10 Thread Michael Paquier
On Thu, Dec 05, 2024 at 06:34:31PM -0500, Tom Lane wrote: > You have a mighty optimistic view of what will happen. I predict > that if we do step (1), exactly nothing will happen in applications, > and step (2) will remain just as painful for them. (Assuming that > we remember to do step (2), whi

Re: Changing the state of data checksums in a running cluster

2024-12-10 Thread Michael Paquier
On Tue, Nov 26, 2024 at 11:07:12PM +0100, Tomas Vondra wrote: > I spent a bit more time doing some testing on the last version of the > patch from [1]. And I ran into this assert in PostmasterStateMachine() > when stopping the cluster: > > /* All types should be included in targetMask or remainM

Re: Subscription sometimes loses txns after initial table sync

2024-12-10 Thread Shlok Kyal
On Tue, 10 Dec 2024 at 07:24, Zhijie Hou (Fujitsu) wrote: > > On Monday, December 9, 2024 9:21 PM Pritam Baral > wrote: > > To: pgsql-hackers > > Subject: Subscription sometimes loses txns after initial table sync > > > > This was discovered when testing the plan for a major version upgrade via

Re: Conflict detection for update_deleted in logical replication

2024-12-10 Thread Amit Kapila
On Fri, Dec 6, 2024 at 1:28 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, December 5, 2024 6:00 PM Amit Kapila > wrote: > > > > > > A few more comments: > > 1. > > +static void > > +wait_for_local_flush(RetainConflictInfoData *data) > > { > > ... > > + > > + data->phase = RCI_GET_CANDIDATE_XID

Re: warn if GUC set to an invalid shared library

2024-12-10 Thread Michael Paquier
On Mon, Nov 11, 2024 at 12:09:15PM +0200, Heikki Linnakangas wrote: > I had a quick glance at this, and I agree with Robert's comment earlier that > this requires a lot of context to understand. All this feedback was 4 weeks ago, and is unanswered, so I've marked the CF entry as returned with feed

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-10 Thread Tom Lane
Michael Paquier writes: > FWIW, my first thought after reading this paragraph is that you sound > too dramatic here, especially after looking at codesearch to note that > the PHP core code stores attndims but does not actually use it. It appeared to me that it fetches it in order to return it in

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread vignesh C
On Tue, 10 Dec 2024 at 23:36, Masahiko Sawada wrote: > > On Mon, Dec 9, 2024 at 6:52 PM Amit Kapila wrote: > > > > On Tue, Dec 10, 2024 at 2:17 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Dec 9, 2024 at 2:06 AM Amit Kapila > > > wrote: > > > > > > > > On Thu, Dec 5, 2024 at 2:56 PM Masa

VACUUM cleaning of CF 2024-11

2024-12-10 Thread Michael Paquier
Hi all, I've done a pass over the CF app, and did some routine vacuuming of the whole. Here are the final scores: Committed: 78 Moved to next CF: 236. Withdrawn: 11 Rejected: 2 Returned with Feedback: 9 Total: 336 As usual, a lot of entries had an outdated status, more than the last time I've lo

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 7:24 PM, Andrey M. Borodin wrote: > > > > On 10 Dec 2024, at 16:26, Dilip Kumar wrote: > > > > IIUC, we do check that it should be in multiple of bank size (i.e.) > > which is multiple of 2, right? Am I missing something? > > Bank selection code assumes that number of buf

Re: XMLDocument (SQL/XML X030)

2024-12-10 Thread Andrew Dunstan
On 2024-12-10 Tu 2:48 AM, Jim Jones wrote: On 04.12.24 17:18, Jim Jones wrote: I'd like to propose the implementation of XMLDocument (SQL/XML X030). It basically returns an XML document from a given XML expression, e.g. SELECT   xmldocument(     xmlelement(NAME foo,   xmlattributes(42

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Robert Haas
On Tue, Dec 10, 2024 at 8:58 AM Dilip Kumar wrote: >> Bank selection code assumes that number of buffers is power of 2. >> If the number of buffers is not power of 2 - only subset of buffers will be >> used. In worst case, e.g. 65 buffers, everything will be buffered only in >> bank 64. > > But

timeline problem when recovery

2024-12-10 Thread Fabrice Chapuis
Hi, I got this message on a standby after a FO of PG14 cluster with 3 nodes. user=,db=,client=,application= LOG: new timeline 20 forked off current database system timeline 19 before current recovery point CC8/164E9350 this message come from xlog.c rescanLatestTimeLine function: if (currentTle-

Re: Pass ParseState as down to utility functions.

2024-12-10 Thread jian he
add parser_errposition to some places in transformTableConstraint, transformColumnDefinition where v8 didn't. From b49e1b74b5d479b854599c0f9d6b6df1e61aa67c Mon Sep 17 00:00:00 2001 From: jian he Date: Tue, 10 Dec 2024 22:36:45 +0800 Subject: [PATCH v9 2/2] Print out error position for number of DD

Re: SQL:2011 application time

2024-12-10 Thread Peter Eisentraut
On 07.12.24 20:29, Paul Jungwirth wrote: These five patches all look good to me. Note that my tests already include a section for REPLICA IDENTITY FULL, which passed. But the subscriber was using a SeqScan to look up tuples to update. Here are the steps (mostly just because it was confusing

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Andrey M. Borodin
> On 10 Dec 2024, at 16:26, Dilip Kumar wrote: > > IIUC, we do check that it should be in multiple of bank size (i.e.) > which is multiple of 2, right? Am I missing something? Bank selection code assumes that number of buffers is power of 2. If the number of buffers is not power of 2 - only

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread Pavel Borisov
On Tue, 10 Dec 2024 at 19:13, Tom Lane wrote: > Pavel Borisov writes: > > I see Aleksander worked on this patch (with many other hackers) and > marked > > it as rejected. And now Evgeniy, a new developer wants to continue and > > rebase this patch and also the patches in another 64-xid thread >

Re: Support for unsigned integer types

2024-12-10 Thread Peter Eisentraut
On 06.12.24 19:45, Jack Bay wrote: Unsigned integers are a widely used type and can be important for compact and well-aligned data structures, but are not currently available in PostgreSQL. In particular unsigned 64-bit integers and unsigned 32-bit integers are desirable as identifiers. They una

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Robert Haas
On Mon, Dec 9, 2024 at 7:31 PM Andres Freund wrote: > Pretty unexcited about all of these - XFS is fairly widely used for PG, but > this problem doesn't seem very common. It seems to me that we're missing > something that causes this to only happen in a small subset of cases. I wonder if this is

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
Hi, On 2024-12-10 12:36:40 -0500, Robert Haas wrote: > On Mon, Dec 9, 2024 at 7:31 PM Andres Freund wrote: > > Pretty unexcited about all of these - XFS is fairly widely used for PG, but > > this problem doesn't seem very common. It seems to me that we're missing > > something that causes this to

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
Robert Haas writes: > On Tue, Dec 10, 2024 at 2:53 PM Tom Lane wrote: >> I'm thinking about something like this: > That seems pretty good, although the last sentence seems like it might > be a little too alarming. Maybe add "although we know of no specific > problem" or something like that. OK,

Re: Vectored IO in XLogWrite()

2024-12-10 Thread Michael Paquier
On Thu, Aug 08, 2024 at 01:25:47PM -0400, Robert Haas wrote: > I wondered whether the regression tests actually hit the iovcnt == 2 > case, and it turns out that they do, rather frequently actually. > Making that case a FATAL causes ~260 regression test failure. However, > on larger systems, we'll

Re: Add ExprState hashing for GROUP BY and hashed SubPlans

2024-12-10 Thread David Rowley
On Wed, 11 Dec 2024 at 11:36, David Rowley wrote: > I've now pushed v7-0001. ... and, after a few comment tweaks, I've also pushed the v7-0002 patch. Thanks for your reviews, Andrei. David

Re: Fix early elog(FATAL)

2024-12-10 Thread Nathan Bossart
On Sat, Dec 07, 2024 at 07:46:14PM -0800, Noah Misch wrote: > main() says: > > /* >* Fire up essential subsystems: error and memory management >* >* Code after this point is allowed to use elog/ereport, though >* localization of messages may not work right awa

Re: [18] Fix a few issues with the collation cache

2024-12-10 Thread Jeff Davis
On Tue, 2024-12-10 at 15:44 +0900, Michael Paquier wrote: > On Fri, Sep 20, 2024 at 05:28:48PM -0700, Jeff Davis wrote: > > Updated and rebased. > > The patch has been failing to apply for a couple of weeks now.  Could > you rebase please? I committed some of the patches and fixed problem #1. Th

Re: [18] Fix a few issues with the collation cache

2024-12-10 Thread Michael Paquier
On Tue, Dec 10, 2024 at 03:34:50PM -0800, Jeff Davis wrote: > I committed some of the patches and fixed problem #1. > > The way I used ResourceOwners to fix problems #2 and #3 is a bit > awkward. I'm not sure if it's worth the ceremony to try to avoid leaks > during OOM. And other paths that leak

Re: fix deprecation mention for age() and mxid_age()

2024-12-10 Thread John Naylor
On Wed, Nov 20, 2024 at 12:23 PM Michael Paquier wrote: > > On Tue, Nov 19, 2024 at 06:52:40AM +, Bertrand Drouvot wrote: > > Thanks for the feedback! > > Done. The section mistake in REL_16_STABLE was.. Interesting. Hi, there is a CF entry for this -- is it ready to mark committed? -- Jo

Re: long-standing data loss bug in initial sync of logical replication

2024-12-10 Thread Michael Paquier
On Tue, Dec 10, 2024 at 01:50:16PM -0800, Masahiko Sawada wrote: > Sorry I lost track of this thread. I'll check the test results and > patch soon. Thanks. -- Michael signature.asc Description: PGP signature

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi Jakub On Tue, 10 Dec 2024 at 22:36, Jakub Wartak wrote: > Yay, reflink=0, that's pretty old fs ?! This particular filesystem was created on Centos 7, and retained when the system was upgraded to RL9. So yes probably pretty old! > Could you get us maybe those below commands too? (or from any

Re: Expand applicability of aggregate's sortop optimization

2024-12-10 Thread Michael Paquier
On Fri, Nov 08, 2024 at 01:05:04PM +0700, Andrei Lepikhov wrote: > Rebase onto current master and slight improvement. Your patch is failing in the CI, please rebase. I have it to the next CF for now, waiting on author. -- Michael signature.asc Description: PGP signature

Re: Separate memory contexts for relcache and catcache

2024-12-10 Thread Michael Paquier
On Tue, Nov 26, 2024 at 05:11:16PM +0300, Melih Mutlu wrote: > That said, these changes come with a cost, and it may not be worth it to > separate every single cache into its own context. IIUC, introducing > contexts for heavily used caches results in much less fragmentation. If > that’s the case,

Re: Serverside SNI support in libpq

2024-12-10 Thread Michael Paquier
On Wed, Dec 04, 2024 at 02:44:18PM +0100, Daniel Gustafsson wrote: > No worries, I know you have a big path on your plate right now. The attached > v3 fixes a small buglet in the tests and adds silly reload testing to try and > stress out any issues. Looks like this still fails quite heavily in t

Re: Document NULL

2024-12-10 Thread David G. Johnston
v5 Attached, v5-0001 is just v4-0001 rebased; v5-0002 is the rework over v4-0001. There is no formatting-only patch this round. Wiki tracker: https://wiki.postgresql.org/wiki/Documenting_NULL#ToDo_Note On Tue, Dec 10, 2024 at 11:52 AM Jeff Davis wrote: > On Mon, 2024-12-09 at 15:27 -0700, Davi

RE: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Zhijie Hou (Fujitsu)
On Wednesday, December 11, 2024 2:14 AM Masahiko Sawada wrote: > > On Tue, Dec 10, 2024 at 1:16 AM Amit Kapila > wrote: > > > > On Tue, Dec 10, 2024 at 11:24 AM Amit Kapila > wrote: > > > > > > On Tue, Dec 10, 2024 at 8:54 AM vignesh C > wrote: > > > > > > > > On Tue, 10 Dec 2024 at 04:56, Mi

advanced patch feedback session at FOSDEM, responses needed

2024-12-10 Thread Peter Eisentraut
Hi all, I'm trying to organize an advanced patch feedback session adjacent to FOSDEM. Right now, I'm looking to gauge interest so that concrete plans can be made. Specifically, if you would like to participate and have your patches reviewed, please respond to this email. The session would be s

Re: fix deprecation mention for age() and mxid_age()

2024-12-10 Thread Bertrand Drouvot
Hi, On Wed, Dec 11, 2024 at 09:01:24AM +0900, Michael Paquier wrote: > On Wed, Dec 11, 2024 at 06:54:23AM +0700, John Naylor wrote: > > Hi, there is a CF entry for this -- is it ready to mark committed? > > Oops. I've missed that there was an entry in CF 51. Updated that > now. Thanks for the

Re: Make use of pg_memory_is_all_zeros() in more places

2024-12-10 Thread Bertrand Drouvot
Hi, On Wed, Dec 11, 2024 at 03:03:34PM +0900, Michael Paquier wrote: > On Tue, Dec 10, 2024 at 02:18:33PM +, Bertrand Drouvot wrote: > > While searching for memcmp() calls in "*stat*.c" files (due to [1]), it > > appeared > > that we could $SUBJECT. Please find attached a patch doing so. > >

Re: Fix comments related to pending statistics

2024-12-10 Thread Bertrand Drouvot
Hi, On Wed, Dec 11, 2024 at 02:56:08PM +0900, Michael Paquier wrote: > On Tue, Dec 10, 2024 at 02:16:14PM +, Bertrand Drouvot wrote: > > > > 2. One linked to PgStat_FunctionCounts pending stats, mentioning the use of > > memcmp() against zeroes to detect whether there are any pending stats: I

Avoid unnecessary wrapping for more complex expressions

2024-12-10 Thread Richard Guo
In commit f64ec81a8 we introduced an optimization that avoids wrapping for Vars and PHVs if they are lateral references to something outside the subquery, and the referenced rel is under the same lowest nulling outer join. It could be beneficial to get rid of such PHVs because they imply lateral d

Re: long-standing data loss bug in initial sync of logical replication

2024-12-10 Thread Masahiko Sawada
On Tue, Oct 8, 2024 at 2:51 AM Shlok Kyal wrote: > > On Wed, 31 Jul 2024 at 03:27, Masahiko Sawada wrote: > > > > On Wed, Jul 24, 2024 at 9:53 PM Amit Kapila wrote: > > > > > > On Wed, Jul 17, 2024 at 5:25 PM vignesh C wrote: > > > > > > > > On Wed, 17 Jul 2024 at 11:54, Amit Kapila > > > > w

Re: CRC32C Parallel Computation Optimization on ARM

2024-12-10 Thread John Naylor
I wrote: > > 1. I looked at a couple implementations of this idea, and found that > the constants used in the carryless multiply are tied to the length of > the blocks. With a lookup table we can do the 3-way algorithm on any > portion of a full block length, rather than immediately fall to doing >

Re: Track the amount of time waiting due to cost_delay

2024-12-10 Thread Bertrand Drouvot
Hi, On Tue, Dec 10, 2024 at 11:55:41AM -0600, Nathan Bossart wrote: > On Mon, Dec 09, 2024 at 04:41:03PM +, Bertrand Drouvot wrote: > > + time_delayed bigint > > I think it's also worth considering names like total_delay and > cumulative_delay. That's fine by me. Then I think that tota

RE: Conflict detection for update_deleted in logical replication

2024-12-10 Thread Zhijie Hou (Fujitsu)
On Wednesday, December 11, 2024 1:06 PM Amit Kapila wrote: > On Fri, Dec 6, 2024 at 1:28 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Thursday, December 5, 2024 6:00 PM Amit Kapila > wrote: > > > > > > > > > A few more comments: > > > 1. > > > +static void > > > +wait_for_local_flush(RetainConfl

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi again On Wed, 11 Dec 2024 at 12:09, Michael Harris wrote: > But another system I can access has multiple databases with ongoing > imports, yet all the errors bar one relate to one directory. > I will collect some data from that system and post it shortly. I've attached the same set of data co

Re: Assert failure on running a completed portal again

2024-12-10 Thread Robert Haas
On Tue, Dec 10, 2024 at 12:14 PM Tom Lane wrote: > > Did you look at the commit message for > > 691b8d59281b5177f16fe80858df921f77a8e955? IMHO, that's pretty clear > > about what the goals of this were. > > Well, it's not very clear about what implementation limitations we > are trying to protect.

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
I wrote: > My recollection is that even before parallelism we had some plan node > types that didn't cope well with being called again after they'd once > returned EOF (ie a null tuple). So maybe a defense against trying to > do that would be wise. I spent a little time trying to run that recolle

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Masahiko Sawada
On Tue, Dec 10, 2024 at 1:16 AM Amit Kapila wrote: > > On Tue, Dec 10, 2024 at 11:24 AM Amit Kapila wrote: > > > > On Tue, Dec 10, 2024 at 8:54 AM vignesh C wrote: > > > > > > On Tue, 10 Dec 2024 at 04:56, Michael Paquier wrote: > > > > > > > > On Mon, Dec 09, 2024 at 03:36:15PM +0530, Amit Kap

Separate GUC for replication origins

2024-12-10 Thread Euler Taveira
Hi, We usually use max_replication_slots as the maximum number of replication origins. It predates the logical replication infrastructure (commit 5aa2350426c). However, it is confusing (if it is the first time you are dealing with logical replication. Why do I need to set replication slots on subs

Re: Assert failure on running a completed portal again

2024-12-10 Thread Robert Haas
On Tue, Dec 10, 2024 at 1:37 PM Tom Lane wrote: > And ExecutorRun skips ExecutePlan if direction is > NoMovementScanDirection. So unless there are gaps in pquery.c's EOF > checks, I think we're OK: the "re-execution" call will not reach > ExecutePlan. This could probably do with more commentary

Re: Document NULL

2024-12-10 Thread Jeff Davis
On Mon, 2024-12-09 at 15:27 -0700, David G. Johnston wrote: > I have not done this.  This is already a large patch and this kind of > example doesn't seem like our norm.  I'm not opposed to more content > like this but for now would leave considering it as something an > interested party can propos

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread Pavel Borisov
Hi, Wenhui! On Tue, 10 Dec 2024 at 17:32, wenhui qiu wrote: > Hi aleksander >Didn't you mark this patch as rejected last time? Do we still need to > continue this path? > > > Thanks > > On Sun, Dec 8, 2024 at 10:16 PM Evgeny Voropaev > wrote: > >> Hi hackers! >> >> Upgraded the "Int64 GUC"

Re: XMLDocument (SQL/XML X030)

2024-12-10 Thread Jim Jones
Hi Andrew On 10.12.24 14:59, Andrew Dunstan wrote: > LGTM at a first glance. > > > Please add this to the next CommitFest if you haven't done already. Thanks! This is the CF entry: https://commitfest.postgresql.org/51/5431/ Best, Jim

Fix comments related to pending statistics

2024-12-10 Thread Bertrand Drouvot
Hi hackers, while working on [1], I came across 2 comments in pgstat.h that I think are not correct. 1. One linked to PgStat_TableCounts pending stats, mentioning the use of memcmp() against zeroes to detect whether there are any stats updates to apply. This is not true anymore as of 07e9e28b56

Make use of pg_memory_is_all_zeros() in more places

2024-12-10 Thread Bertrand Drouvot
Hi hackers, While searching for memcmp() calls in "*stat*.c" files (due to [1]), it appeared that we could $SUBJECT. Please find attached a patch doing so. While at it, I did a quick check on all the memcmp() calls (even those not in "*stat*.c" files) and it looks to me that there is no others th

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
On 2024-12-10 11:34:15 -0500, Andres Freund wrote: > On 2024-12-10 12:36:33 +0100, Jakub Wartak wrote: > > On Tue, Dec 10, 2024 at 7:34 AM Michael Harris wrote: > > 2. > > > > > # xfs_info /dev/mapper/ippvg-ipplv > > > meta-data=/dev/mapper/ippvg-ipplv isize=512agcount=4, > > agsize=26247142

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
Hi, On 2024-12-10 12:36:33 +0100, Jakub Wartak wrote: > On Tue, Dec 10, 2024 at 7:34 AM Michael Harris wrote: > 1. Well it doesn't look like XFS AG fragmentation to me (we had a customer > with a huge number of AGs with small space in them) reporting such errors > after upgrading to 16, but not

Re: Assert failure on running a completed portal again

2024-12-10 Thread Robert Haas
On Thu, Dec 5, 2024 at 8:09 PM Tom Lane wrote: > After looking at this further, I think this whole "run_once" > business is badly designed, worse documented, and redundant > (as well as buggy). It can be replaced with three self-contained > lines in ExecutePlan, as per the attached. Did you look

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, Dec 10, 2024 at 6:32 PM Andrey M. Borodin wrote: > > > > > On 10 Dec 2024, at 15:39, Yura Sokolov wrote: > > > > It is not critical bug, since it doesn't hurt correctness just performance. > > In worst case only one bank will be used. > > Ugh... yeah. IMO the problem is that we do not ha

Re: Sample rate added to pg_stat_statements

2024-12-10 Thread Ilia Evdokimov
Hi everyone, I attached previous sampling patch for pg_stat_statements (v4). Suggestions like sampling based on execution time remain unfeasible, as pg_stat_statements can track query during query planning, not execution. To evaluate the implementation, I ran a benchmark creating 1,000 random

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Andrey M. Borodin
> On 10 Dec 2024, at 16:58, Dilip Kumar wrote: > > slru buffers are in multiple of 16(bank size) Yes, my example with 64 buffers is incorrect. The worst case scenario is when user configures 80, 144, 528 or 1040 buffers, but only two banks (32 buffers) will be used. Best regards, Andrey Bo

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
Robert Haas writes: > On Thu, Dec 5, 2024 at 8:09 PM Tom Lane wrote: >> After looking at this further, I think this whole "run_once" >> business is badly designed, worse documented, and redundant >> (as well as buggy). It can be replaced with three self-contained >> lines in ExecutePlan, as per

Re: Refactoring postmaster's code to cleanup after child exit

2024-12-10 Thread Heikki Linnakangas
On 09/12/2024 22:55, Heikki Linnakangas wrote: Not sure how to fix this. A small sleep in the test would work, but in principle there's no delay that's guaranteed to be enough. A more robust solution would be to run a "select count(*) from pg_stat_activity" and wait until the number of connecti

Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE

2024-12-10 Thread Guillaume Lelarge
Hello, Le mar. 10 déc. 2024 à 03:57, David Rowley a écrit : > On Thu, 21 Nov 2024 at 08:30, Guillaume Lelarge > wrote: > > OK, I'm fine with this. v4 patch attached with one plan showing read, > written, and dirtied buffers. > > Today I spent more time polishing this patch. There were a few cas

Fix bank selection logic in SLRU

2024-12-10 Thread Yura Sokolov
Good day, hackers. Due to long discussion about SLRU size configuration [1] and code evolution, non-serious bug were introduced: - intermediate versions assumed cache size is always power of 2, - therefore to determine bank simple binary-and with mask were used: bankno = pageno & ctl->ban

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Masahiko Sawada
On Mon, Dec 9, 2024 at 6:52 PM Amit Kapila wrote: > > On Tue, Dec 10, 2024 at 2:17 AM Masahiko Sawada wrote: > > > > On Mon, Dec 9, 2024 at 2:06 AM Amit Kapila wrote: > > > > > > On Thu, Dec 5, 2024 at 2:56 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > > > > I realized that this patch

Re: Track the amount of time waiting due to cost_delay

2024-12-10 Thread Nathan Bossart
On Mon, Dec 09, 2024 at 04:41:03PM +, Bertrand Drouvot wrote: > + time_delayed bigint I think it's also worth considering names like total_delay and cumulative_delay. > + Total amount of time spent in milliseconds waiting during linkend="guc-vacuum-cost-delay"/> > + or . In

Re: Add Pipelining support in psql

2024-12-10 Thread Anthonin Bonnefoy
An improved version with simplifications and refinements. num_queries (2nd element in the pipeline status prompt) is now used to track queued queries that were not flushed (with a flush request or sync) to the server. It used to count both unflushed queries and flushed queries. Code in ExecQueryA

Re: Proposal to add a new URL data type.

2024-12-10 Thread Victor Yegorov
чт, 5 дек. 2024 г. в 17:02, Alexander Borisov : > My name is Alexander Borisov, I want to propose/discuss > adding a new URL type as an extension in PostgreSQL contrib. > I think everyone here knows/seen what URL/URI and their basics. > If someone is interested in the basics, you can read the orig

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread Tom Lane
Pavel Borisov writes: > I see Aleksander worked on this patch (with many other hackers) and marked > it as rejected. And now Evgeniy, a new developer wants to continue and > rebase this patch and also the patches in another 64-xid thread > disregarding the fact that the patch is rejected. It's not

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 7:30 PM, Robert Haas wrote: > On Tue, Dec 10, 2024 at 8:58 AM Dilip Kumar wrote: > >> Bank selection code assumes that number of buffers is power of 2. > >> If the number of buffers is not power of 2 - only subset of buffers > will be used. In worst case, e.g. 65 buffers,

Re: Define STATS_MIN_ROWS for minimum rows of stats in ANALYZE

2024-12-10 Thread Ilia Evdokimov
On 09.12.2024 16:10, Ilia Evdokimov wrote: Hi hackers, The repeated use of the number 300 in the ANALYZE-related code creates redundancy and relies on scattered, sometimes unclear, comments to explain its purpose. This can make the code harder to understand, especially for new contributors w

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread wenhui qiu
Hi aleksander Didn't you mark this patch as rejected last time? Do we still need to continue this path? Thanks On Sun, Dec 8, 2024 at 10:16 PM Evgeny Voropaev wrote: > Hi hackers! > > Upgraded the "Int64 GUC" patch in order to conform to f3f06b13308e3 > updates. Rebased and tested upon the

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 6:32 PM, Andrey M. Borodin wrote: > > > > On 10 Dec 2024, at 15:39, Yura Sokolov wrote: > > > > It is not critical bug, since it doesn't hurt correctness just > performance. In worst case only one bank will be used. > > Ugh... yeah. IMO the problem is that we do not have p

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
Hi, On 2024-12-10 17:28:21 +1100, Michael Harris wrote: > On Tue, 10 Dec 2024 at 11:31, Andres Freund wrote: > > It'd be useful to get the xfs_info output that Jakub asked for. Perhaps also > > xfs_spaceman -c 'freesp -s' /mountpoint > > xfs_spaceman -c 'health' /mountpoint > > and df. > > I gath

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
Robert Haas writes: > Thanks for the research, and +1 if you feel like adding more commentary. I'm thinking about something like this: diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 13f3fcdaee..7ebb17fc2e 100644 --- a/src/backend/executor/execMain.c +++ b/sr

Re: [PATCH] Add sortsupport for range types and btree_gist

2024-12-10 Thread Bernd Helmle
Am Montag, dem 09.12.2024 um 23:26 +0500 schrieb Andrey M. Borodin: > Besides unnecessary indentation changes in > contrib/btree_gist/Makefile, the patch seems good to me. Oh yes, fixed. -- Thanks, Bernd From 2d2c3b5940a8ec15b1c6bd6b829c8d44c0e615e0 Mon Sep 17 00:00:00 2001 From: Bernd

Re: Extensible storage manager API - SMGR hook Redux

2024-12-10 Thread Xun Gong
Thank you for your detailed review and insights. I share your view that a registration-based approach for custom storage managers (smgr) is more versatile. This method allows for the implementation of custom table access methods, facilitating the use of various storage services (such as file servic

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread Shubham Khanna
On Mon, Dec 9, 2024 at 7:43 PM vignesh C wrote: > > On Mon, 9 Dec 2024 at 16:25, Shubham Khanna > wrote: > > > > Hi all, > > > > I am writing to propose the addition of the two_phase option in > > pg_createsubscriber. As discussed in [1], supporting this feature > > during the development of pg_

Re: Refactoring postmaster's code to cleanup after child exit

2024-12-10 Thread Tomas Vondra
On 12/10/24 11:00, Heikki Linnakangas wrote: > On 09/12/2024 22:55, Heikki Linnakangas wrote: >> Not sure how to fix this. A small sleep in the test would work, but in >> principle there's no delay that's guaranteed to be enough. A more >> robust solution would be to run a "select count(*) from

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread Shubham Khanna
On Tue, Dec 10, 2024 at 7:42 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for the proposal! > > > I am writing to propose the addition of the two_phase option in > > pg_createsubscriber. As discussed in [1], supporting this feature > > during the development of pg_createsubscribe

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-10 Thread Nisha Moond
On Fri, Dec 6, 2024 at 11:04 AM vignesh C wrote: > > > Determining the correct time may be challenging for users, as it > depends on when the active_since value is set, as well as when the > checkpoint_timeout occurs and the subsequent checkpoint is triggered. > Even if the user sets it to an appr

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread Shubham Khanna
On Tue, Dec 10, 2024 at 8:05 AM Peter Smith wrote: > > Hi Shubham, > > Here are some review comments for the patch v1-0001. > > Note -- I think Kuroda-san's idea to use a new switch like > '--enable-two-phase' would eliminate lots of my review comments below > about the inconsistencies with CREATE

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Jakub Wartak
On Tue, Dec 10, 2024 at 7:34 AM Michael Harris wrote: Hi Michael, 1. Well it doesn't look like XFS AG fragmentation to me (we had a customer with a huge number of AGs with small space in them) reporting such errors after upgrading to 16, but not for earlier versions (somehow posix_fallocate() h

Re: WARNING: missing lock on database "postgres" (OID 5) @ TID (0,4)

2024-12-10 Thread Kirill Reshke
On Tue, 10 Dec 2024 at 10:47, Kirill Reshke wrote: > > On Tue, 10 Dec 2024 at 10:45, Kirill Reshke wrote: > > > PFA v2. > Also CF entry https://commitfest.postgresql.org/51/5430/ to get CI feedback. CI fails due to bad naming in the regression test. The change is deptestdb1 -> regressdeptestdb1

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Amit Kapila
On Tue, Dec 10, 2024 at 11:24 AM Amit Kapila wrote: > > On Tue, Dec 10, 2024 at 8:54 AM vignesh C wrote: > > > > On Tue, 10 Dec 2024 at 04:56, Michael Paquier wrote: > > > > > > On Mon, Dec 09, 2024 at 03:36:15PM +0530, Amit Kapila wrote: > > > > It couldn't solve the problem completely even in

Re: Pass ParseState as down to utility functions.

2024-12-10 Thread Kirill Reshke
On Tue, 10 Dec 2024 at 08:28, Michael Paquier wrote: > I would suggest to split the patch into two pieces for clarity, based > on the fact that your v7 patch is doing more than one thing at the > same time: > - Introduce new tests for the new coverage (domain, CREATE TABLE OF, > ALTER TABLE flavo

Re: bt_index_parent_check and concurrently build indexes

2024-12-10 Thread Michail Nikolaev
Hello, everyone and Peter. I simplified the patch to reproduce the issue more effectively. Now, bt_index_parent_check fails on a valid index containing just two tuples. Peter, I included you in this message because you are the author of bt_index_parent_check, so I thought you might find this rele

Re: bt_index_parent_check and concurrently build indexes

2024-12-10 Thread Andrey M. Borodin
> On 9 Dec 2024, at 23:51, Michail Nikolaev wrote: > > * Modify bt_index_parent_check to use an MVCC snapshot for the heap scan Hi! Interesting bug. It's amazing how long it stand, giving that it would be triggered by almost any check after updating a table. From my POV correct fix directi

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Andrey M. Borodin
> On 10 Dec 2024, at 15:39, Yura Sokolov wrote: > > It is not critical bug, since it doesn't hurt correctness just performance. > In worst case only one bank will be used. Ugh... yeah. IMO the problem is that we do not have protection that rejects values that are not power of 2. If other va

Re: Unmark gen_random_uuid() function leakproof

2024-12-10 Thread Masahiko Sawada
On Mon, Dec 9, 2024 at 2:48 PM Masahiko Sawada wrote: > > On Mon, Dec 9, 2024 at 2:23 PM Andres Freund wrote: > > > > Hi, > > > > On 2024-12-09 14:10:30 -0800, Masahiko Sawada wrote: > > > While reviewing UUIDv7 patch[1], I realized gen_random_uuid() is > > > marked leakproof even though it doesn

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread Peter Smith
Hi Shubham, Here are some review comments for patch v2-0001. == GENERAL - the new option name 1. I am not sure if this new switch needed to be called '--enable-two-phase'; probably just calling it '--two-phase' would mean the same because specifying the switch already implies "enabling" it t

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi Andres On Wed, 11 Dec 2024 at 03:09, Andres Freund wrote: > I think it's implied, but I just want to be sure: This was one of the affected > systems? Yes, correct. > Any chance to get df output? I'm mainly curious about the number of used > inodes. Sorry, I could swear I had included that a

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-10 Thread Masahiko Sawada
On Mon, Dec 9, 2024 at 10:19 PM Dilip Kumar wrote: > > On Tue, Dec 10, 2024 at 11:09 AM Amit Kapila wrote: > > > > On Tue, Dec 10, 2024 at 10:59 AM Dilip Kumar wrote: > > > > > > On Tue, Nov 26, 2024 at 3:02 AM Masahiko Sawada > > > wrote: > > > > > > > > > > > I've attached a new version patc

  1   2   >