Re: Make use of pg_memory_is_all_zeros() in more places

2024-12-10 Thread Bertrand Drouvot
Hi, On Wed, Dec 11, 2024 at 03:03:34PM +0900, Michael Paquier wrote: > On Tue, Dec 10, 2024 at 02:18:33PM +, Bertrand Drouvot wrote: > > While searching for memcmp() calls in "*stat*.c" files (due to [1]), it > > appeared > > that we could $SUBJECT. Please find attached a patch doing so. > >

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi again On Wed, 11 Dec 2024 at 12:09, Michael Harris wrote: > But another system I can access has multiple databases with ongoing > imports, yet all the errors bar one relate to one directory. > I will collect some data from that system and post it shortly. I've attached the same set of data co

RE: Conflict detection for update_deleted in logical replication

2024-12-10 Thread Zhijie Hou (Fujitsu)
On Wednesday, December 11, 2024 1:06 PM Amit Kapila wrote: > On Fri, Dec 6, 2024 at 1:28 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Thursday, December 5, 2024 6:00 PM Amit Kapila > wrote: > > > > > > > > > A few more comments: > > > 1. > > > +static void > > > +wait_for_local_flush(RetainConfl

Re: Fix comments related to pending statistics

2024-12-10 Thread Bertrand Drouvot
Hi, On Wed, Dec 11, 2024 at 02:56:08PM +0900, Michael Paquier wrote: > On Tue, Dec 10, 2024 at 02:16:14PM +, Bertrand Drouvot wrote: > > > > 2. One linked to PgStat_FunctionCounts pending stats, mentioning the use of > > memcmp() against zeroes to detect whether there are any pending stats: I

Avoid unnecessary wrapping for more complex expressions

2024-12-10 Thread Richard Guo
In commit f64ec81a8 we introduced an optimization that avoids wrapping for Vars and PHVs if they are lateral references to something outside the subquery, and the referenced rel is under the same lowest nulling outer join. It could be beneficial to get rid of such PHVs because they imply lateral d

Re: CRC32C Parallel Computation Optimization on ARM

2024-12-10 Thread John Naylor
I wrote: > > 1. I looked at a couple implementations of this idea, and found that > the constants used in the carryless multiply are tied to the length of > the blocks. With a lookup table we can do the 3-way algorithm on any > portion of a full block length, rather than immediately fall to doing >

Re: long-standing data loss bug in initial sync of logical replication

2024-12-10 Thread Masahiko Sawada
On Tue, Oct 8, 2024 at 2:51 AM Shlok Kyal wrote: > > On Wed, 31 Jul 2024 at 03:27, Masahiko Sawada wrote: > > > > On Wed, Jul 24, 2024 at 9:53 PM Amit Kapila wrote: > > > > > > On Wed, Jul 17, 2024 at 5:25 PM vignesh C wrote: > > > > > > > > On Wed, 17 Jul 2024 at 11:54, Amit Kapila > > > > w

advanced patch feedback session at FOSDEM, responses needed

2024-12-10 Thread Peter Eisentraut
Hi all, I'm trying to organize an advanced patch feedback session adjacent to FOSDEM. Right now, I'm looking to gauge interest so that concrete plans can be made. Specifically, if you would like to participate and have your patches reviewed, please respond to this email. The session would be s

Re: fix deprecation mention for age() and mxid_age()

2024-12-10 Thread Bertrand Drouvot
Hi, On Wed, Dec 11, 2024 at 09:01:24AM +0900, Michael Paquier wrote: > On Wed, Dec 11, 2024 at 06:54:23AM +0700, John Naylor wrote: > > Hi, there is a CF entry for this -- is it ready to mark committed? > > Oops. I've missed that there was an entry in CF 51. Updated that > now. Thanks for the

Re: Track the amount of time waiting due to cost_delay

2024-12-10 Thread Bertrand Drouvot
Hi, On Tue, Dec 10, 2024 at 11:55:41AM -0600, Nathan Bossart wrote: > On Mon, Dec 09, 2024 at 04:41:03PM +, Bertrand Drouvot wrote: > > + time_delayed bigint > > I think it's also worth considering names like total_delay and > cumulative_delay. That's fine by me. Then I think that tota

RE: Replace IN VALUES with ANY in WHERE clauses during optimization

2024-12-10 Thread postgresql_contributors
Hi Ivan, I tested the patch using the input provided in commands.sql and observed improvements in both planning and execution time. These optimizations are especially noticeable when working with a mulitple tables. Even when querying just a single table, there is a small improvement in planning

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-10 Thread vignesh C
On Tue, 10 Dec 2024 at 17:21, Nisha Moond wrote: > > On Fri, Dec 6, 2024 at 11:04 AM vignesh C wrote: > > > > > > Determining the correct time may be challenging for users, as it > > depends on when the active_since value is set, as well as when the > > checkpoint_timeout occurs and the subsequen

Re: Virtual generated columns

2024-12-10 Thread jian he
On Fri, Nov 29, 2024 at 6:01 PM Peter Eisentraut wrote: > The purpose of check_modified_virtual_generated() for trigger functions > written in C. The prevent someone from inserting real values into the > trigger tuples, because they would then be processed by the rest of the > system, which would

Re: Track the amount of time waiting due to cost_delay

2024-12-10 Thread Dilip Kumar
On Tue, Dec 10, 2024 at 11:25 PM Nathan Bossart wrote: > > On Mon, Dec 09, 2024 at 04:41:03PM +, Bertrand Drouvot wrote: > > + time_delayed bigint > > I think it's also worth considering names like total_delay and > cumulative_delay. +1, I vote for total_delay > > + Total amount

Re: Sort functions with specialized comparators

2024-12-10 Thread John Naylor
On Mon, Dec 9, 2024 at 8:02 PM Andrey M. Borodin wrote: > > > On 6 Dec 2024, at 08:49, John Naylor wrote: > > That's a good thing to raise right now -- intarray currently doesn't > > have one, and we haven't gotten complaints from people trying to sort > > large arrays and cancel the query. This

Re: WARNING: missing lock on database "postgres" (OID 5) @ TID (0,4)

2024-12-10 Thread Kirill Reshke
On Tue, 10 Dec 2024 at 14:14, Kirill Reshke wrote: > CI fails due to bad naming in the regression test. > The change is deptestdb1 -> regressdeptestdb1 I'm very sorry for spam. PFAv4 regressdeptestdb1 -> regressiondeptestdb1 -- Best regards, Kirill Reshke v4-0001-When-making-dependency-cha

Suggestion to standardize comment format in pg_dump

2024-12-10 Thread Nohez Poonawala
Currently, the pg_dump tool outputs comments in different formats for primary/foreign keys and indexes. Below is the comment format. - for Primary key: -- Name: TABLENAME CONSTRAINTNAME; Type: CONSTRAINT; Schema: SCHEMA; Owner: OWNER - for Foreign key: -- Name: TABLENAME CONSTRAINTNAME; Type:

Re: CREATE SCHEMA ... CREATE DOMAIN support

2024-12-10 Thread Kirill Reshke
Looks like this thread does not move forward. So I'm posting my thoughts to bump it. On Wed, 4 Dec 2024 at 01:07, Tom Lane wrote: > I'm totally willing to throw that possibility overboard > permanently in order to expand the set of creatable object types > without introducing a ton of restrictio

RE: Memory leak in pg_logical_slot_{get,peek}_changes

2024-12-10 Thread Zhijie Hou (Fujitsu)
On Wednesday, December 11, 2024 12:28 PM vignesh C wrote: > Hi, > > I'm starting a new thread for one of the issues reported by Sawada-san at [1]. > > This is a memory leak on CacheMemoryContext when using pgoutput via SQL > APIs: > /* Map must live as long as the session does. */ > oldctx = Mem

Re: Make use of pg_memory_is_all_zeros() in more places

2024-12-10 Thread Michael Paquier
On Tue, Dec 10, 2024 at 02:18:33PM +, Bertrand Drouvot wrote: > While searching for memcmp() calls in "*stat*.c" files (due to [1]), it > appeared > that we could $SUBJECT. Please find attached a patch doing so. -SockAddrzero_clientaddr; -memset(&zero_clientaddr, 0

Re: Fix comments related to pending statistics

2024-12-10 Thread Michael Paquier
On Tue, Dec 10, 2024 at 02:16:14PM +, Bertrand Drouvot wrote: > 1. One linked to PgStat_TableCounts pending stats, mentioning the use of > memcmp() against zeroes to detect whether there are any stats updates to > apply. > > This is not true anymore as of 07e9e28b56. Oops, you're right. The

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread Shubham Khanna
On Wed, Dec 11, 2024 at 10:59 AM vignesh C wrote: > > On Tue, 10 Dec 2024 at 17:17, Shubham Khanna > wrote: > > > > On Mon, Dec 9, 2024 at 7:43 PM vignesh C wrote: > > > > The attached Patch contains the required changes. > > Few comments: > 1) This is not correct, currently enabling two_phase o

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread Shubham Khanna
On Wed, Dec 11, 2024 at 4:21 AM Peter Smith wrote: > > Hi Shubham, > > Here are some review comments for patch v2-0001. > > == > GENERAL - the new option name > > 1. > I am not sure if this new switch needed to be called > '--enable-two-phase'; probably just calling it '--two-phase' would > me

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Masahiko Sawada
On Tue, Dec 10, 2024 at 6:13 PM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, December 11, 2024 2:14 AM Masahiko Sawada > wrote: > > > > On Tue, Dec 10, 2024 at 1:16 AM Amit Kapila > > wrote: > > > > > > On Tue, Dec 10, 2024 at 11:24 AM Amit Kapila > > wrote: > > > > > > > > On Tue, Dec 10, 20

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread vignesh C
On Tue, 10 Dec 2024 at 17:17, Shubham Khanna wrote: > > On Mon, Dec 9, 2024 at 7:43 PM vignesh C wrote: > > The attached Patch contains the required changes. Few comments: 1) This is not correct, currently enabling two_phase option using alter subscription is supported: Notably, the replication

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-10 Thread Michael Paquier
On Wed, Dec 11, 2024 at 12:15:51AM -0500, Tom Lane wrote: > Michael Paquier writes: > > FWIW, my first thought after reading this paragraph is that you sound > > too dramatic here, especially after looking at codesearch to note that > > the PHP core code stores attndims but does not actually use i

VACUUM cleaning of CF 2024-11

2024-12-10 Thread Michael Paquier
Hi all, I've done a pass over the CF app, and did some routine vacuuming of the whole. Here are the final scores: Committed: 78 Moved to next CF: 236. Withdrawn: 11 Rejected: 2 Returned with Feedback: 9 Total: 336 As usual, a lot of entries had an outdated status, more than the last time I've lo

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread vignesh C
On Tue, 10 Dec 2024 at 23:36, Masahiko Sawada wrote: > > On Mon, Dec 9, 2024 at 6:52 PM Amit Kapila wrote: > > > > On Tue, Dec 10, 2024 at 2:17 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Dec 9, 2024 at 2:06 AM Amit Kapila > > > wrote: > > > > > > > > On Thu, Dec 5, 2024 at 2:56 PM Masa

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-10 Thread Tom Lane
Michael Paquier writes: > FWIW, my first thought after reading this paragraph is that you sound > too dramatic here, especially after looking at codesearch to note that > the PHP core code stores attndims but does not actually use it. It appeared to me that it fetches it in order to return it in

Re: warn if GUC set to an invalid shared library

2024-12-10 Thread Michael Paquier
On Mon, Nov 11, 2024 at 12:09:15PM +0200, Heikki Linnakangas wrote: > I had a quick glance at this, and I agree with Robert's comment earlier that > this requires a lot of context to understand. All this feedback was 4 weeks ago, and is unanswered, so I've marked the CF entry as returned with feed

Re: Conflict detection for update_deleted in logical replication

2024-12-10 Thread Amit Kapila
On Fri, Dec 6, 2024 at 1:28 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, December 5, 2024 6:00 PM Amit Kapila > wrote: > > > > > > A few more comments: > > 1. > > +static void > > +wait_for_local_flush(RetainConflictInfoData *data) > > { > > ... > > + > > + data->phase = RCI_GET_CANDIDATE_XID

Re: Subscription sometimes loses txns after initial table sync

2024-12-10 Thread Shlok Kyal
On Tue, 10 Dec 2024 at 07:24, Zhijie Hou (Fujitsu) wrote: > > On Monday, December 9, 2024 9:21 PM Pritam Baral > wrote: > > To: pgsql-hackers > > Subject: Subscription sometimes loses txns after initial table sync > > > > This was discovered when testing the plan for a major version upgrade via

Re: Changing the state of data checksums in a running cluster

2024-12-10 Thread Michael Paquier
On Tue, Nov 26, 2024 at 11:07:12PM +0100, Tomas Vondra wrote: > I spent a bit more time doing some testing on the last version of the > patch from [1]. And I ran into this assert in PostmasterStateMachine() > when stopping the cluster: > > /* All types should be included in targetMask or remainM

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-10 Thread Michael Paquier
On Thu, Dec 05, 2024 at 06:34:31PM -0500, Tom Lane wrote: > You have a mighty optimistic view of what will happen. I predict > that if we do step (1), exactly nothing will happen in applications, > and step (2) will remain just as painful for them. (Assuming that > we remember to do step (2), whi

Memory leak in pg_logical_slot_{get,peek}_changes

2024-12-10 Thread vignesh C
Hi, I'm starting a new thread for one of the issues reported by Sawada-san at [1]. This is a memory leak on CacheMemoryContext when using pgoutput via SQL APIs: /* Map must live as long as the session does. */ oldctx = MemoryContextSwitchTo(CacheMemoryContext); entry->attrmap = build_attrmap_by_

Re: long-standing data loss bug in initial sync of logical replication

2024-12-10 Thread Shlok Kyal
On Tue, 8 Oct 2024 at 11:11, Shlok Kyal wrote: > > Hi Kuroda-san, > > > > I have also modified the tests in 0001 patch. These changes are only > > > related to syntax of writing tests. > > > > LGTM. I found small improvements, please find the attached. > > I have applied the changes and updated th

Re: Conflict detection for update_deleted in logical replication

2024-12-10 Thread Nisha Moond
Here are the test steps and analysis for epoch-related handling (Tested with v15 Patch-Set). In the 'update_deleted' detection design, the launcher process compares XIDs to track minimum XIDs, and the apply workers maintain the oldest running XIDs. The launcher also requests publisher status at re

Re: [PoC] Reducing planning time when tables have many partitions

2024-12-10 Thread Yuya Watari
Hello Ashutosh and Alvaro, I appreciate you replying to the email. On Tue, Dec 3, 2024 at 7:38 PM Alvaro Herrera wrote: > > I don't think planning time in assert-enabled builds is something we > should worry about, at all. Planning time in production builds is the > important one. Thank you fo

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi Jakub On Tue, 10 Dec 2024 at 22:36, Jakub Wartak wrote: > Yay, reflink=0, that's pretty old fs ?! This particular filesystem was created on Centos 7, and retained when the system was upgraded to RL9. So yes probably pretty old! > Could you get us maybe those below commands too? (or from any

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-10 Thread Dilip Kumar
On Wed, Dec 11, 2024 at 3:18 AM Masahiko Sawada wrote: > > On Mon, Dec 9, 2024 at 10:19 PM Dilip Kumar wrote: > > > > > > If the largest transaction is non-streamable, won't the transaction > > > returned by ReorderBufferLargestTXN() in the other case already > > > suffice the need? > > > > I se

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-10 Thread Peter Smith
Hi Nisha. Here are some review comments for patch v54-0002. (I had also checked patch v54-0001, but have no further review comments for that one). == doc/src/sgml/config.sgml 1. + +Slot invalidation due to idle timeout occurs during checkpoint. +If the checkpoint_time

RE: Parallel heap vacuum

2024-12-10 Thread Hayato Kuroda (Fujitsu)
Dear Tomas, > 1) I really like the idea of introducing "compute_workers" callback to > the heap AM interface. I faced a similar issue with calculating workers > for index builds, because right now plan_create_index_workers is doing > that the logic works for btree, but really not brin etc. It didn

Re: Wrong results with right-semi-joins

2024-12-10 Thread Richard Guo
On Mon, Dec 9, 2024 at 11:01 PM Melanie Plageman wrote: > Thanks for finding and fixing this. Just for my own benefit, could you > explain more about the minimal repro? Specifically, if you just need a > subplan in the hash side of a right semi-join, why do you also need > the outer part of the qu

RE: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Zhijie Hou (Fujitsu)
On Wednesday, December 11, 2024 2:14 AM Masahiko Sawada wrote: > > On Tue, Dec 10, 2024 at 1:16 AM Amit Kapila > wrote: > > > > On Tue, Dec 10, 2024 at 11:24 AM Amit Kapila > wrote: > > > > > > On Tue, Dec 10, 2024 at 8:54 AM vignesh C > wrote: > > > > > > > > On Tue, 10 Dec 2024 at 04:56, Mi

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi Andres On Wed, 11 Dec 2024 at 03:09, Andres Freund wrote: > I think it's implied, but I just want to be sure: This was one of the affected > systems? Yes, correct. > Any chance to get df output? I'm mainly curious about the number of used > inodes. Sorry, I could swear I had included that a

Re: Add ExprState hashing for GROUP BY and hashed SubPlans

2024-12-10 Thread David Rowley
On Wed, 11 Dec 2024 at 11:36, David Rowley wrote: > I've now pushed v7-0001. ... and, after a few comment tweaks, I've also pushed the v7-0002 patch. Thanks for your reviews, Andrei. David

Re: Vectored IO in XLogWrite()

2024-12-10 Thread Michael Paquier
On Thu, Aug 08, 2024 at 01:25:47PM -0400, Robert Haas wrote: > I wondered whether the regression tests actually hit the iovcnt == 2 > case, and it turns out that they do, rather frequently actually. > Making that case a FATAL causes ~260 regression test failure. However, > on larger systems, we'll

Re: Serverside SNI support in libpq

2024-12-10 Thread Michael Paquier
On Wed, Dec 04, 2024 at 02:44:18PM +0100, Daniel Gustafsson wrote: > No worries, I know you have a big path on your plate right now. The attached > v3 fixes a small buglet in the tests and adds silly reload testing to try and > stress out any issues. Looks like this still fails quite heavily in t

Re: Separate memory contexts for relcache and catcache

2024-12-10 Thread Michael Paquier
On Tue, Nov 26, 2024 at 05:11:16PM +0300, Melih Mutlu wrote: > That said, these changes come with a cost, and it may not be worth it to > separate every single cache into its own context. IIUC, introducing > contexts for heavily used caches results in much less fragmentation. If > that’s the case,

Re: Expand applicability of aggregate's sortop optimization

2024-12-10 Thread Michael Paquier
On Fri, Nov 08, 2024 at 01:05:04PM +0700, Andrei Lepikhov wrote: > Rebase onto current master and slight improvement. Your patch is failing in the CI, please rebase. I have it to the next CF for now, waiting on author. -- Michael signature.asc Description: PGP signature

Re: fix deprecation mention for age() and mxid_age()

2024-12-10 Thread Michael Paquier
On Wed, Dec 11, 2024 at 06:54:23AM +0700, John Naylor wrote: > Hi, there is a CF entry for this -- is it ready to mark committed? Oops. I've missed that there was an entry in CF 51. Updated that now. Thanks for the poke. -- Michael signature.asc Description: PGP signature

Re: long-standing data loss bug in initial sync of logical replication

2024-12-10 Thread Michael Paquier
On Tue, Dec 10, 2024 at 01:50:16PM -0800, Masahiko Sawada wrote: > Sorry I lost track of this thread. I'll check the test results and > patch soon. Thanks. -- Michael signature.asc Description: PGP signature

Re: fix deprecation mention for age() and mxid_age()

2024-12-10 Thread John Naylor
On Wed, Nov 20, 2024 at 12:23 PM Michael Paquier wrote: > > On Tue, Nov 19, 2024 at 06:52:40AM +, Bertrand Drouvot wrote: > > Thanks for the feedback! > > Done. The section mistake in REL_16_STABLE was.. Interesting. Hi, there is a CF entry for this -- is it ready to mark committed? -- Jo

Re: [18] Fix a few issues with the collation cache

2024-12-10 Thread Michael Paquier
On Tue, Dec 10, 2024 at 03:34:50PM -0800, Jeff Davis wrote: > I committed some of the patches and fixed problem #1. > > The way I used ResourceOwners to fix problems #2 and #3 is a bit > awkward. I'm not sure if it's worth the ceremony to try to avoid leaks > during OOM. And other paths that leak

Re: [18] Fix a few issues with the collation cache

2024-12-10 Thread Jeff Davis
On Tue, 2024-12-10 at 15:44 +0900, Michael Paquier wrote: > On Fri, Sep 20, 2024 at 05:28:48PM -0700, Jeff Davis wrote: > > Updated and rebased. > > The patch has been failing to apply for a couple of weeks now.  Could > you rebase please? I committed some of the patches and fixed problem #1. Th

Re: Add a warning message when using unencrypted passwords

2024-12-10 Thread Nathan Bossart
On Sat, Dec 07, 2024 at 03:39:55PM +0100, Guillaume Lelarge wrote: > I thought about it, and tried to write a patch. I've mostly copied the > "Deprecate MD5 passwords" patch/commit from Nathan Bossart. My patch works > on current HEAD. Documentation and tests are dealt with. I've been thinking abo

Re: Document NULL

2024-12-10 Thread David G. Johnston
v5 Attached, v5-0001 is just v4-0001 rebased; v5-0002 is the rework over v4-0001. There is no formatting-only patch this round. Wiki tracker: https://wiki.postgresql.org/wiki/Documenting_NULL#ToDo_Note On Tue, Dec 10, 2024 at 11:52 AM Jeff Davis wrote: > On Mon, 2024-12-09 at 15:27 -0700, Davi

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2024-12-10 Thread Peter Smith
Hi Shubham, Here are some review comments for patch v2-0001. == GENERAL - the new option name 1. I am not sure if this new switch needed to be called '--enable-two-phase'; probably just calling it '--two-phase' would mean the same because specifying the switch already implies "enabling" it t

Re: Fix early elog(FATAL)

2024-12-10 Thread Nathan Bossart
On Sat, Dec 07, 2024 at 07:46:14PM -0800, Noah Misch wrote: > main() says: > > /* >* Fire up essential subsystems: error and memory management >* >* Code after this point is allowed to use elog/ereport, though >* localization of messages may not work right awa

Re: Unmark gen_random_uuid() function leakproof

2024-12-10 Thread Masahiko Sawada
On Mon, Dec 9, 2024 at 2:48 PM Masahiko Sawada wrote: > > On Mon, Dec 9, 2024 at 2:23 PM Andres Freund wrote: > > > > Hi, > > > > On 2024-12-09 14:10:30 -0800, Masahiko Sawada wrote: > > > While reviewing UUIDv7 patch[1], I realized gen_random_uuid() is > > > marked leakproof even though it doesn

Re: long-standing data loss bug in initial sync of logical replication

2024-12-10 Thread Masahiko Sawada
On Mon, Dec 9, 2024 at 10:27 PM Michael Paquier wrote: > > On Tue, Oct 08, 2024 at 03:21:38PM +0530, Shlok Kyal wrote: > > I have tested the scenario shared by you on the thread [1]. And I > > confirm that the latest patch [2] fixes this issue. > > > > [1] > > https://www.postgresql.org/message-i

Re: Skip collecting decoded changes of already-aborted transactions

2024-12-10 Thread Masahiko Sawada
On Mon, Dec 9, 2024 at 10:19 PM Dilip Kumar wrote: > > On Tue, Dec 10, 2024 at 11:09 AM Amit Kapila wrote: > > > > On Tue, Dec 10, 2024 at 10:59 AM Dilip Kumar wrote: > > > > > > On Tue, Nov 26, 2024 at 3:02 AM Masahiko Sawada > > > wrote: > > > > > > > > > > > I've attached a new version patc

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
Robert Haas writes: > On Tue, Dec 10, 2024 at 2:53 PM Tom Lane wrote: >> I'm thinking about something like this: > That seems pretty good, although the last sentence seems like it might > be a little too alarming. Maybe add "although we know of no specific > problem" or something like that. OK,

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
Hi, On 2024-12-10 12:36:40 -0500, Robert Haas wrote: > On Mon, Dec 9, 2024 at 7:31 PM Andres Freund wrote: > > Pretty unexcited about all of these - XFS is fairly widely used for PG, but > > this problem doesn't seem very common. It seems to me that we're missing > > something that causes this to

Re: Assert failure on running a completed portal again

2024-12-10 Thread Robert Haas
On Tue, Dec 10, 2024 at 2:53 PM Tom Lane wrote: > Robert Haas writes: > > Thanks for the research, and +1 if you feel like adding more commentary. > > I'm thinking about something like this: > > diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c > index 13f3fcdaee..7eb

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
Robert Haas writes: > Thanks for the research, and +1 if you feel like adding more commentary. I'm thinking about something like this: diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 13f3fcdaee..7ebb17fc2e 100644 --- a/src/backend/executor/execMain.c +++ b/sr

Re: Document NULL

2024-12-10 Thread Jeff Davis
On Mon, 2024-12-09 at 15:27 -0700, David G. Johnston wrote: > I have not done this.  This is already a large patch and this kind of > example doesn't seem like our norm.  I'm not opposed to more content > like this but for now would leave considering it as something an > interested party can propos

Re: Assert failure on running a completed portal again

2024-12-10 Thread Robert Haas
On Tue, Dec 10, 2024 at 1:37 PM Tom Lane wrote: > And ExecutorRun skips ExecutePlan if direction is > NoMovementScanDirection. So unless there are gaps in pquery.c's EOF > checks, I think we're OK: the "re-execution" call will not reach > ExecutePlan. This could probably do with more commentary

Separate GUC for replication origins

2024-12-10 Thread Euler Taveira
Hi, We usually use max_replication_slots as the maximum number of replication origins. It predates the logical replication infrastructure (commit 5aa2350426c). However, it is confusing (if it is the first time you are dealing with logical replication. Why do I need to set replication slots on subs

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
I wrote: > My recollection is that even before parallelism we had some plan node > types that didn't cope well with being called again after they'd once > returned EOF (ie a null tuple). So maybe a defense against trying to > do that would be wise. I spent a little time trying to run that recolle

Re: Assert failure on running a completed portal again

2024-12-10 Thread Robert Haas
On Tue, Dec 10, 2024 at 12:14 PM Tom Lane wrote: > > Did you look at the commit message for > > 691b8d59281b5177f16fe80858df921f77a8e955? IMHO, that's pretty clear > > about what the goals of this were. > > Well, it's not very clear about what implementation limitations we > are trying to protect.

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Masahiko Sawada
On Tue, Dec 10, 2024 at 1:16 AM Amit Kapila wrote: > > On Tue, Dec 10, 2024 at 11:24 AM Amit Kapila wrote: > > > > On Tue, Dec 10, 2024 at 8:54 AM vignesh C wrote: > > > > > > On Tue, 10 Dec 2024 at 04:56, Michael Paquier wrote: > > > > > > > > On Mon, Dec 09, 2024 at 03:36:15PM +0530, Amit Kap

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-10 Thread Masahiko Sawada
On Mon, Dec 9, 2024 at 6:52 PM Amit Kapila wrote: > > On Tue, Dec 10, 2024 at 2:17 AM Masahiko Sawada wrote: > > > > On Mon, Dec 9, 2024 at 2:06 AM Amit Kapila wrote: > > > > > > On Thu, Dec 5, 2024 at 2:56 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > > > > I realized that this patch

Re: Track the amount of time waiting due to cost_delay

2024-12-10 Thread Nathan Bossart
On Mon, Dec 09, 2024 at 04:41:03PM +, Bertrand Drouvot wrote: > + time_delayed bigint I think it's also worth considering names like total_delay and cumulative_delay. > + Total amount of time spent in milliseconds waiting during linkend="guc-vacuum-cost-delay"/> > + or . In

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Robert Haas
On Mon, Dec 9, 2024 at 7:31 PM Andres Freund wrote: > Pretty unexcited about all of these - XFS is fairly widely used for PG, but > this problem doesn't seem very common. It seems to me that we're missing > something that causes this to only happen in a small subset of cases. I wonder if this is

Re: Assert failure on running a completed portal again

2024-12-10 Thread Tom Lane
Robert Haas writes: > On Thu, Dec 5, 2024 at 8:09 PM Tom Lane wrote: >> After looking at this further, I think this whole "run_once" >> business is badly designed, worse documented, and redundant >> (as well as buggy). It can be replaced with three self-contained >> lines in ExecutePlan, as per

Re: Assert failure on running a completed portal again

2024-12-10 Thread Robert Haas
On Thu, Dec 5, 2024 at 8:09 PM Tom Lane wrote: > After looking at this further, I think this whole "run_once" > business is badly designed, worse documented, and redundant > (as well as buggy). It can be replaced with three self-contained > lines in ExecutePlan, as per the attached. Did you look

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
Hi, On 2024-12-10 12:36:33 +0100, Jakub Wartak wrote: > On Tue, Dec 10, 2024 at 7:34 AM Michael Harris wrote: > 1. Well it doesn't look like XFS AG fragmentation to me (we had a customer > with a huge number of AGs with small space in them) reporting such errors > after upgrading to 16, but not

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
On 2024-12-10 11:34:15 -0500, Andres Freund wrote: > On 2024-12-10 12:36:33 +0100, Jakub Wartak wrote: > > On Tue, Dec 10, 2024 at 7:34 AM Michael Harris wrote: > > 2. > > > > > # xfs_info /dev/mapper/ippvg-ipplv > > > meta-data=/dev/mapper/ippvg-ipplv isize=512agcount=4, > > agsize=26247142

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Andres Freund
Hi, On 2024-12-10 17:28:21 +1100, Michael Harris wrote: > On Tue, 10 Dec 2024 at 11:31, Andres Freund wrote: > > It'd be useful to get the xfs_info output that Jakub asked for. Perhaps also > > xfs_spaceman -c 'freesp -s' /mountpoint > > xfs_spaceman -c 'health' /mountpoint > > and df. > > I gath

Re: Support for unsigned integer types

2024-12-10 Thread Peter Eisentraut
On 06.12.24 19:45, Jack Bay wrote: Unsigned integers are a widely used type and can be important for compact and well-aligned data structures, but are not currently available in PostgreSQL. In particular unsigned 64-bit integers and unsigned 32-bit integers are desirable as identifiers. They una

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread Pavel Borisov
On Tue, 10 Dec 2024 at 19:13, Tom Lane wrote: > Pavel Borisov writes: > > I see Aleksander worked on this patch (with many other hackers) and > marked > > it as rejected. And now Evgeniy, a new developer wants to continue and > > rebase this patch and also the patches in another 64-xid thread >

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread Tom Lane
Pavel Borisov writes: > I see Aleksander worked on this patch (with many other hackers) and marked > it as rejected. And now Evgeniy, a new developer wants to continue and > rebase this patch and also the patches in another 64-xid thread > disregarding the fact that the patch is rejected. It's not

Re: SQL:2011 application time

2024-12-10 Thread Peter Eisentraut
On 07.12.24 20:29, Paul Jungwirth wrote: These five patches all look good to me. Note that my tests already include a section for REPLICA IDENTITY FULL, which passed. But the subscriber was using a SeqScan to look up tuples to update. Here are the steps (mostly just because it was confusing

Re: Pass ParseState as down to utility functions.

2024-12-10 Thread jian he
add parser_errposition to some places in transformTableConstraint, transformColumnDefinition where v8 didn't. From b49e1b74b5d479b854599c0f9d6b6df1e61aa67c Mon Sep 17 00:00:00 2001 From: jian he Date: Tue, 10 Dec 2024 22:36:45 +0800 Subject: [PATCH v9 2/2] Print out error position for number of DD

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Andrey M. Borodin
> On 10 Dec 2024, at 16:58, Dilip Kumar wrote: > > slru buffers are in multiple of 16(bank size) Yes, my example with 64 buffers is incorrect. The worst case scenario is when user configures 80, 144, 528 or 1040 buffers, but only two banks (32 buffers) will be used. Best regards, Andrey Bo

Re: Sample rate added to pg_stat_statements

2024-12-10 Thread Ilia Evdokimov
Hi everyone, I attached previous sampling patch for pg_stat_statements (v4). Suggestions like sampling based on execution time remain unfeasible, as pg_stat_statements can track query during query planning, not execution. To evaluate the implementation, I ran a benchmark creating 1,000 random

Make use of pg_memory_is_all_zeros() in more places

2024-12-10 Thread Bertrand Drouvot
Hi hackers, While searching for memcmp() calls in "*stat*.c" files (due to [1]), it appeared that we could $SUBJECT. Please find attached a patch doing so. While at it, I did a quick check on all the memcmp() calls (even those not in "*stat*.c" files) and it looks to me that there is no others th

Fix comments related to pending statistics

2024-12-10 Thread Bertrand Drouvot
Hi hackers, while working on [1], I came across 2 comments in pgstat.h that I think are not correct. 1. One linked to PgStat_TableCounts pending stats, mentioning the use of memcmp() against zeroes to detect whether there are any stats updates to apply. This is not true anymore as of 07e9e28b56

Re: XMLDocument (SQL/XML X030)

2024-12-10 Thread Jim Jones
Hi Andrew On 10.12.24 14:59, Andrew Dunstan wrote: > LGTM at a first glance. > > > Please add this to the next CommitFest if you haven't done already. Thanks! This is the CF entry: https://commitfest.postgresql.org/51/5431/ Best, Jim

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 6:32 PM, Andrey M. Borodin wrote: > > > > On 10 Dec 2024, at 15:39, Yura Sokolov wrote: > > > > It is not critical bug, since it doesn't hurt correctness just > performance. In worst case only one bank will be used. > > Ugh... yeah. IMO the problem is that we do not have p

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 7:30 PM, Robert Haas wrote: > On Tue, Dec 10, 2024 at 8:58 AM Dilip Kumar wrote: > >> Bank selection code assumes that number of buffers is power of 2. > >> If the number of buffers is not power of 2 - only subset of buffers > will be used. In worst case, e.g. 65 buffers,

timeline problem when recovery

2024-12-10 Thread Fabrice Chapuis
Hi, I got this message on a standby after a FO of PG14 cluster with 3 nodes. user=,db=,client=,application= LOG: new timeline 20 forked off current database system timeline 19 before current recovery point CC8/164E9350 this message come from xlog.c rescanLatestTimeLine function: if (currentTle-

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Robert Haas
On Tue, Dec 10, 2024 at 8:58 AM Dilip Kumar wrote: >> Bank selection code assumes that number of buffers is power of 2. >> If the number of buffers is not power of 2 - only subset of buffers will be >> used. In worst case, e.g. 65 buffers, everything will be buffered only in >> bank 64. > > But

Re: XMLDocument (SQL/XML X030)

2024-12-10 Thread Andrew Dunstan
On 2024-12-10 Tu 2:48 AM, Jim Jones wrote: On 04.12.24 17:18, Jim Jones wrote: I'd like to propose the implementation of XMLDocument (SQL/XML X030). It basically returns an XML document from a given XML expression, e.g. SELECT   xmldocument(     xmlelement(NAME foo,   xmlattributes(42

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, 10 Dec 2024 at 7:24 PM, Andrey M. Borodin wrote: > > > > On 10 Dec 2024, at 16:26, Dilip Kumar wrote: > > > > IIUC, we do check that it should be in multiple of bank size (i.e.) > > which is multiple of 2, right? Am I missing something? > > Bank selection code assumes that number of buf

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Andrey M. Borodin
> On 10 Dec 2024, at 16:26, Dilip Kumar wrote: > > IIUC, we do check that it should be in multiple of bank size (i.e.) > which is multiple of 2, right? Am I missing something? Bank selection code assumes that number of buffers is power of 2. If the number of buffers is not power of 2 - only

Re: Fix bank selection logic in SLRU

2024-12-10 Thread Dilip Kumar
On Tue, Dec 10, 2024 at 6:32 PM Andrey M. Borodin wrote: > > > > > On 10 Dec 2024, at 15:39, Yura Sokolov wrote: > > > > It is not critical bug, since it doesn't hurt correctness just performance. > > In worst case only one bank will be used. > > Ugh... yeah. IMO the problem is that we do not ha

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread Pavel Borisov
Hi, Wenhui! On Tue, 10 Dec 2024 at 17:32, wenhui qiu wrote: > Hi aleksander >Didn't you mark this patch as rejected last time? Do we still need to > continue this path? > > > Thanks > > On Sun, Dec 8, 2024 at 10:16 PM Evgeny Voropaev > wrote: > >> Hi hackers! >> >> Upgraded the "Int64 GUC"

Re: [PATCH] Support Int64 GUCs

2024-12-10 Thread wenhui qiu
Hi aleksander Didn't you mark this patch as rejected last time? Do we still need to continue this path? Thanks On Sun, Dec 8, 2024 at 10:16 PM Evgeny Voropaev wrote: > Hi hackers! > > Upgraded the "Int64 GUC" patch in order to conform to f3f06b13308e3 > updates. Rebased and tested upon the

  1   2   >