Re: Add memory_limit_hits to pg_stat_replication_slots

Hi, On Thu, Sep 11, 2025 at 03:24:54PM -0700, Masahiko Sawada wrote: > On Wed, Aug 27, 2025 at 12:26 AM Bertrand Drouvot > wrote: > > > > Looking forward to your feedback, > > Yes, Thanks for looking at it! > it's a quite different situation in two cases: sp

Get rid of pgstat_count_backend_io_op*() functions

r pending stats are populated while other stats kinds are flushing. [1]: https://www.postgresql.org/message-id/7fhpds4xqk6bnudzmzkqi33pinsxammpljwde5gfkjdygvejrj%40ojkzfr7dxkmm Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.a

Re: Per backend relation statistics tracking

Hi, On Tue, Aug 26, 2025 at 06:38:41AM +, Bertrand Drouvot wrote: > On Mon, Aug 25, 2025 at 08:28:04PM -0400, Andres Freund wrote: > > FWIW, I think this was done wrongly for the per-backend IO stats too. I've > > seen the increased overhead in profiles > > That

Re: Get rid of pgstat_count_backend_io_op*() functions

loop in pgstat_report_stat() That way the backend are flushed last and that solves your concern about custom stats kind. The backend would not be the "only" exception in pgstat_report_stat(), the db stats are already handled as exception with pgstat_update_dbstats(). Regards, -- Bertra

Re: Get rid of pgstat_count_backend_io_op*() functions

Hi, On Tue, Sep 02, 2025 at 07:41:59AM +, Bertrand Drouvot wrote: > On Tue, Sep 02, 2025 at 08:19:22AM +0900, Michael Paquier wrote: > > Removing both the function calls and the extra counter incrementations > > means to do the same thing as the WAL stats, where we have one &

Re: Get rid of pgstat_count_backend_io_op*() functions

Hi, On Tue, Sep 02, 2025 at 08:19:22AM +0900, Michael Paquier wrote: > On Mon, Sep 01, 2025 at 02:07:27PM +0000, Bertrand Drouvot wrote: > > Instead, it now copies the IO pending stats to the backend IO pending stats > > when > > the pending IO stats are flushed. This behave

Re: Improve LWLock tranche name visibility across backends

ptr) % LWLOCK_PADDED_SIZE; in CreateLWLocks(), and just make use of CACHELINEALIGN(). Attached, a patch doing so. It applies on top of your v20. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 4720ffe5c804c6a

Generate pgstat_count_slru*() functions for slru using macros

named to match their associated counter. This is the same idea as 850f4b4, 8018ffb or 83a1a1b. Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Improve LWLock tranche name visibility across backends

ngth. I think that is much safer. Same remark as above but in RequestNamedLWLockTranche(). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: make LWLockCounter a global variable

*LWLockCounter; to use the global variable. That way we preserve the current memory layout and there is no need to change LWLockShmemSize(). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Adding per backend commit and rollback counters

Hi, On Sun, Aug 10, 2025 at 07:47:09AM +, Bertrand Drouvot wrote: > To sum up, v3 contains: > > 0001 - > Adding per backend commit and rollback counters > 0002 - > Adding XID generation count per backend > 0003 - > Adding the pg_stat_backend view Following re

Re: Per backend relation statistics tracking

Hi, On Wed, Aug 27, 2025 at 01:57:13PM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Aug 26, 2025 at 04:55:09PM -0500, Sami Imseih wrote: > > I worry that a single view will grow very wide, and we will have to > > eventually > > split it. So we may as well

Re: Improve LWLock tranche name visibility across backends

amically created tranches in shared memory. + * Any unused entries in the array will contain NULL. This variable is + * non-static so that postmaster.c can copy it to child processes in + * EXEC_BACKEND builds. "Any unused entries in the array will contain NULL": this is not true anym

Re: Per backend relation statistics tracking

ll add a new pg_stat_backend_wal view if the current proposal goes in (for consistency). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Report bytes and transactions actually sent downtream

he > corresponding column has "plugin" prefix The way it is done makes sense to me. > 3. The names of new columns have the prefix "plugin_" but the internal > variables tracking those don't for the sake of brevity. If you prefer > to have the same

Re: Per backend relation statistics tracking

Hi, On Tue, Aug 26, 2025 at 07:18:10AM -0400, Andres Freund wrote: > Hi, > > On 2025-08-26 06:38:41 +, Bertrand Drouvot wrote: > > > and IO related counters aren't > > > incremented remotely as often as the scan related counters are. > > > > You m

Re: Improve LWLock tranche name visibility across backends

OK-ish. Using strcpy() might be OK too, as we already have validated the length, but maybe it would be safer to switch to strlcpy(), instead? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Report reorder buffer size

Hi, On Tue, Aug 26, 2025 at 09:48:04AM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Aug 26, 2025 at 02:56:14PM +0530, Ashutosh Bapat wrote: > > On Mon, Aug 25, 2025 at 6:28 PM Bertrand Drouvot > > wrote: > > > > I didn't know this is how it wo

Add memory_limit_hits to pg_stat_replication_slots

277 |247 (1 row) Not sure I like memory_limit_hits that much, maybe work_mem_exceeded is better? Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.a

Re: Per backend relation statistics tracking

for more "sophisticated" ones (like pg_stat_get_backend_io()) that need multiple rows per backend. Having said that, we could imagine adding pg_stat_get_backend_wal() output to pg_stat_backend too. Thoughts? -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Report reorder buffer size

Hi, On Tue, Aug 26, 2025 at 02:56:14PM +0530, Ashutosh Bapat wrote: > On Mon, Aug 25, 2025 at 6:28 PM Bertrand Drouvot > wrote: > > > I didn't know this is how it works. Thanks for correcting me. In that > > > case, I think we should > > > add a column in p

Re: Per backend relation statistics tracking

Hi, On Tue, Aug 26, 2025 at 06:38:41AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Aug 25, 2025 at 08:28:04PM -0400, Andres Freund wrote: > > I don't like that this basically doubles the overhead of keeping stats by > > tracking everythign twice. The proper solutio

Re: Per backend relation statistics tracking

Hi, On Mon, Aug 25, 2025 at 08:28:04PM -0400, Andres Freund wrote: > Hi, > > On 2025-08-12 07:48:10 +, Bertrand Drouvot wrote: > > From 9e2f8cb9a87f1d9be91f2f39ef25fbb254944968 Mon Sep 17 00:00:00 2001 > > From: Bertrand Drouvot > > Date: Mon, 4 Aug 2025 08:14:0

Re: Per backend relation statistics tracking

form more index scans than UserB?" Right. I think we can see the relations as the "destinations" of the activity and the backends as the "source" of it. Tracking both sides help to visualize the activity from different angles. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Per backend relation statistics tracking

with a > connection pooler with some specific configuration or when checking > transaction activity across multiple databases by joining with > pg_stat_activity? Yes exactly. And also to get XID (virtual transactions excluded) consumption rate per backend like in [1]. In [1], the number of X

Re: Report reorder buffer size

Hi, On Mon, Aug 25, 2025 at 01:48:42PM +0530, Ashutosh Bapat wrote: > On Fri, Aug 22, 2025 at 11:43 AM Bertrand Drouvot > wrote: > > > > Yes as long as subtrans are not involved. But if say I've one transaction > > made of > > 1500 subtrans, I'd

Re: Add os_page_num to pg_buffercache

See also the corresponding discussion [1]. > > Shall we add it to the function introduced in this patch as well? Yeah, I think so. Thanks for the ping, done in attached. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Servi

Re: Report reorder buffer size

Hi Ashutosh, On Thu, Aug 21, 2025 at 07:26:41PM +0530, Ashutosh Bapat wrote: > Hi Bertrand, > Thanks for your response. I am glad that you have found the proposal > to be generally useful. > > On Thu, Aug 14, 2025 at 3:50 PM Bertrand Drouvot > wrote: > > > > > U

Re: Improve LWLock tranche name visibility across backends

al need for dsarray for those). > I think it's a separate discussion at this point. OTOH, that would be a valid use case to introduce this new API but I'm not sure it's worth it given that the dshash looks good enough for our case (even if not ideal though). [1]: https://www.postg

Re: Improve LWLock tranche name visibility across backends

g_loc unless $ENABLE_LOG_WAIT; is ENABLE_LOG_WAIT needed as it does not change? [1]: https://www.postgresql.org/message-id/aKLsu2sdpnyeuSSc%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Remove useless pointer advance in StatsShmemInit()

Hi, On Mon, Aug 18, 2025 at 06:13:05PM +0900, Michael Paquier wrote: > On Mon, Aug 18, 2025 at 09:04:59AM +0000, Bertrand Drouvot wrote: > > As StatsShmemInit() is existing code, let's fix it: the pointer is not used > > after > > its last advance, so that advance

Remove useless pointer advance in StatsShmemInit()

https://www.postgresql.org/message-id/CAA5RZ0ukmNd%2BC1jH4V6BGEea-wmyLxDtDE5QoEtfXd2W5HNHfQ%40mail.gmail.com Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From b8f56fe7df72830b0054c3652170716341d9dee2 Mon Sep 17 00:00:00 2

Re: Report reorder buffer size

. Also, should we log some of information to the server log? > But before I work on those, I would like to know others' opinions, > usefulness and acceptance. Thanks for working on it. I like the idea and also think that is an area where more details/metrics should be provided. Regards,

Re: Improve LWLock tranche name visibility across backends

e get rid of max_used_index and I think that this part is simpler and easier to understand. Thoughts? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Adding locks statistics

Hi, On Tue, Aug 12, 2025 at 08:42:48AM -0700, Jeff Davis wrote: > On Tue, 2025-08-12 at 09:37 +0000, Bertrand Drouvot wrote: > > It can be used for example for: > > > > 1. checking if "waits" is close to "requests". Then it means you > > usually ha

Re: Adding locks statistics

Hi, On Tue, Aug 12, 2025 at 08:54:07AM -0400, Greg Sabino Mullane wrote: > On Tue, Aug 12, 2025 at 5:32 AM Bertrand Drouvot < > bertranddrouvot...@gmail.com> wrote: > > > Docs: seem good. Needs a section on how to reset via > > > SELECT pg_stat_reset_shared('

Re: Adding locks statistics

vious to me. Thanks for providing your thoughts! I thought it was more natural to aggregate by locktype because: - I think that matches how they are categorized in the doc (from a "wait event" point of view i.e "Wait Events of Type Lock"). - It provides a natural drill-do

Re: Adding locks statistics

n these are called in critical sections, which is I guess why > you've done it this way, right? Yes and there is no need to over complicate this stuff as the max size is known: LOCKTAG_LAST_TYPE + 1. [1]: https://www.postgresql.org/message-id/aIyNxBWFCybgBZBS%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Adding locks statistics

#x27;m not sure that's worth for this particular case and code paths. That said, if you (and/or others) have strong concerns about performance penalties, I could study this area more in depth. > Busier than I realized until I saw it in this useful > view! Thanks! Did you observe some noticeable performance penalties? [1]: https://www.postgresql.org/message-id/202501211750.xf5j6thdkppy%40alvherre.pgsql Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Per backend relation statistics tracking

the memory increase is relatively small. [1]: https://www.postgresql.org/message-id/aJhOfZ2c7XdHXOAU%40ip-10-97-1-34.eu-west-3.compute.internal Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https

Re: Adding per backend commit and rollback counters

Hi, On Thu, Aug 07, 2025 at 08:17:26AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Aug 04, 2025 at 02:20:48PM +, Bertrand Drouvot wrote: > > This patch is pretty straightforward as it relies on the existing per > > backend > > statistics machinery that has be

Re: Adding per backend commit and rollback counters

Hi, On Mon, Aug 04, 2025 at 02:20:48PM +, Bertrand Drouvot wrote: > This patch is pretty straightforward as it relies on the existing per backend > statistics machinery that has been added in 9aea73fc61d (so that there is not > that much design to discuss). Still, while working

Re: BF mamba failure

Hi, On Thu, Aug 07, 2025 at 04:17:29PM +0900, Michael Paquier wrote: > On Thu, Aug 07, 2025 at 05:28:17AM +0000, Bertrand Drouvot wrote: > > On Thu, Aug 07, 2025 at 08:18:26AM +0900, Michael Paquier wrote: > >> On Mon, Aug 04, 2025 at 01:00:35PM +0300, Kouber Saparev wrote: >

Re: BF mamba failure

for this relation. Is that vanilla > streaming replication with heap? I was wondering about adding the "generation" (introduced in 818119afccd3) to the above error log message, thoughts? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Improve LWLock tranche name visibility across backends

(tranche_name != NULL) + SetLocalTrancheName(trancheId, tranche_name); + } + else + tranche_name = LWLockTrancheNames.local[trancheId]; Then does not find the lock in the local cache and then returns "extension". I think that in that case we should fall back to querying the DSA (and populate the local cache), thoughts? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: don't include tableam.h in nbtree.h

Shouldn't that be the case? If so, PFA a patch to apply on top of your v1. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com diff --git a/contrib/btree_gist/btree_bool.c b/contrib/btree_gist/btree_bool

Re: Improve LWLock tranche name visibility across backends

CreateLWLocks() Maybe we should put this back? - /* This should only be called for user-defined tranches. */ - if (tranche_id < LWTRANCHE_FIRST_USER_DEFINED) - return; - /* Convert to array index. */ - tranche_id -= LWTRANCHE_FIRST_USER_DEFINED; and remove: + tranche_index = tranche_id - LWTRANCHE_FIRST_USER_DEFINED; from LWLockNewTrancheId()? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Adding per backend commit and rollback counters

ight time. Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 28747c241ac531fb54ca8cf3943e72fdbabb7d17 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot Date: Mon, 4 Aug 2025 08:

Re: Improve LWLock tranche name visibility across backends

all > the > above code paths. > > Perhaps there's a better approach? I was thinking to switch to the DSA (and update local copy) when a name is not found in the local copy. That way there is no need to maintain a counter and the DSA overhead should be rare enough. Regards, -- Ber

Re: Missing import in 035_standby_logical_decoding.pl

move these checks I'm afraid that could lead to false positives or to the test taking longer as waiting for a timeout to occur. > or > switch them to use/add a log message that's triggered with a state > that we'd expect to happen in reality? If no conflicts occur, we do not emit messages in the log, then having nothing to check. I think that's fine that way. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Missing import in 035_standby_logical_decoding.pl

is actually > "invalidating obsolete replication slot" > > It is correctly specified elsewhere in the same test. > so, that should be fixed too. Thanks for the report! Indeed, fixed in 0001 just shared up-thread. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Missing import in 035_standby_logical_decoding.pl

It's typo/bad wording fixed in the attached. hot_standby_feedback does not need to be enabled during the initial phases of the test because there is no catalog changes that could produce a conflict. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Serv

Re: Improve LWLock tranche name visibility across backends

#x27;m not sure that would be worth the extra complexity and using the old API would "keep" the issue we're trying to solve here. I don't think we should be worried that much by the number of extensions impacted but more about the change complexity and it looks pretty simple.

Re: Adding wait events statistics

es we had to > wait, and how many of those times we waited for longer than > deadlock_timeout". I did find this idea interesting, and gave it a stab in [1]. [1]: https://www.postgresql.org/message-id/aIyNxBWFCybgBZBS%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand

Adding locks statistics

is is merged, a next step could be to record those metrics per backend [1]: https://www.postgresql.org/message-id/CA%2BTgmobptuUWo7X5zcQrWKh22qeAn4eL%2B%3Dwtb8_ajCOR%2B7_tcw%40mail.gmail.com Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RD

Re: Adding wait events statistics

Hi, On Thu, Jul 24, 2025 at 10:38:49AM -0400, Robert Haas wrote: > On Thu, Jul 24, 2025 at 7:52 AM Bertrand Drouvot > wrote: > > Well, the idea was more: as we speak about "wait" events then it would make > > sense to add their duration. And then, to have meaningfu

Re: Add os_page_num to pg_buffercache

r to the above > (SET ROLE pg_database_owner/pg_monitor ... RESET role), with a couple of > different SELECT statements within. Should we rather add the above new > SELECTs there, instead of in the new pg_buffercache_os_pages.sql? Yeah, that probably makes more sense, done in the attached.

Re: Custom pgstat support performance regression for simple queries

Hi, On Fri, Jul 25, 2025 at 10:38:59AM +0900, Michael Paquier wrote: > On Thu, Jul 24, 2025 at 07:38:46AM +0000, Bertrand Drouvot wrote: > > What about? > > > > - create a global variable but that should be used only by extensions? Say, > > pgstat_report_custom_fixed &

Re: Adding wait events statistics

Hi, On Wed, Jul 23, 2025 at 11:38:07AM -0400, Robert Haas wrote: > On Tue, Jul 22, 2025 at 8:24 AM Bertrand Drouvot > wrote: > > So based on the cycles metric I think it looks pretty safe to implement for > > the > > whole majority of classes. > > I'm not

Re: Adding wait events statistics

Hi, On Tue, Jul 22, 2025 at 10:07:30AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-22 12:24:46 +, Bertrand Drouvot wrote: > > Anyway, let's forget about eBPF, I ran another experiment by counting the > > cycles > > with: > > > > static inlin

Re: Custom pgstat support performance regression for simple queries

stat_report_stat(): - do the check on have_iostats, have_slrustats and friends and on pgstat_report_custom_fixed - reset pgstat_report_custom_fixed That way I think it's less error prone for the builtin stats and more clear for the extensions (as at least "custom" is also part of the global variable name and the check in pgstat_report_stat() would make it clear that we rely on the custom global variable). Thoughts? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Custom pgstat support performance regression for simple queries

On Wed, Jul 23, 2025 at 05:09:54PM +0900, Michael Paquier wrote: > On Jul 23, 2025, at 16:33, Bertrand Drouvot > wrote: > > Maybe we could use a flag, say: > > > > #define PGSTAT_PENDING_IO (1 << 0) > > #define PGSTAT_PENDING_WAL(1 << 1)

Re: Custom pgstat support performance regression for simple queries

_pending_mask accordingly when they flush, have pending stats though. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

for example. I don't think that's a big deal though. Also, I think we don't need to read lwlocklist.h twice in generate-lwlocknames.pl. PFA v4 where the only change compared to v3 is that it reads lwlocklist.h once in generate-lwlocknames.pl. Regards, -- Bertrand Drouvot PostgreS

Re: Adding wait events statistics

Hi, On Fri, Jul 18, 2025 at 11:43:47AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-18 06:04:38 +, Bertrand Drouvot wrote: > > Here what I’ve done: > > > > 1. Added 2 probes: one in pgstat_report_wait_start() and one in > > pgstat_report_wait_end

Re: pgsql: Introduce pg_shmem_allocations_numa view

hub.com/torvalds/linux/commit/10d04c26ab2b7 > > Yay! I like how the first response is "you sent the patch wrong" ;-) > > I would have thought that coming from two major projects that use > email extensively (Debian, PostgreSQL), I would navigate that process > better.

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

Hi, On Mon, Jul 21, 2025 at 03:20:55PM -0500, Nathan Bossart wrote: > On Fri, Jul 18, 2025 at 01:39:15PM +0000, Bertrand Drouvot wrote: > > +#define PG_BUILTIN_LWTRANCHE(id, name) [id] = name, > > +#include "storage/lwlocktranchelist.h" > > +#undef PG_BUILT

Re: Verify predefined LWLocks tranches have entries in wait_event_names.txt

Hi, On Mon, Jul 21, 2025 at 03:28:14PM -0500, Nathan Bossart wrote: > On Mon, Jul 21, 2025 at 03:20:55PM -0500, Nathan Bossart wrote: > > On Fri, Jul 18, 2025 at 01:39:15PM +, Bertrand Drouvot wrote: > >> +#define PG_BUILTIN_LWTRANCHE(id, name) [id] = name, >

Re: Add os_page_num to pg_buffercache

he wording is almost the same as in pg_buffercache_numa_pages(). Also I think that it is not correct in pg_get_shmem_allocations_numa(), I think that it should be something like proposed in [1]. [1]: https://www.postgresql.org/message-id/aH4DDhdiG9Gi0rG7%40ip-10-97-1-34.eu-west-3.compute.internal Regards

Fix incorrect comment in pg_get_shmem_allocations_numa()

Hi hackers, One comment in pg_get_shmem_allocations_numa() incorrectly describes the function as dealing with database blocks when it actually processes shared memory allocations from the shmem index. PFA, a patch to $SUBJECT. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS

Re: Fix lwlock.c and wait_event_names.txt discrepancy

Hi, On Thu, Jul 17, 2025 at 02:25:46PM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Jul 17, 2025 at 09:50:12AM +0900, Michael Paquier wrote: > > On Wed, Jul 16, 2025 at 11:41:09AM +, Bertrand Drouvot wrote: > > > > > As a way to prevent this to occur we

Verify predefined LWLocks tranches have entries in wait_event_names.txt

; (in lwlock.h) from lwlocktranchelist.h with generate-lwlocknames.pl. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 830e9dce82d42eddbd7281f4267bb771162dbd47 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot Date: Fr

Re: Adding wait events statistics

Hi, On Tue, Jul 15, 2025 at 04:13:49PM +0200, Bertrand Drouvot wrote: > Hi, > > On Fri, Jul 11, 2025 at 11:59:39AM -0400, Andres Freund wrote: > > Hi, > > > > On 2025-07-10 08:21:56 +, Bertrand Drouvot wrote: > > > - Let's define a list of &quo

Re: Fix lwlock.c and wait_event_names.txt discrepancy

Hi, On Thu, Jul 17, 2025 at 09:50:12AM +0900, Michael Paquier wrote: > On Wed, Jul 16, 2025 at 11:41:09AM +0000, Bertrand Drouvot wrote: > > > As a way to prevent this to occur we might want to add extra input file(s) > > parameter to generate-wait_event_types.pl

Re: Fix lwlock.c and wait_event_names.txt discrepancy

Hi, On Wed, Jul 16, 2025 at 01:33:07PM +0200, Álvaro Herrera wrote: > On 2025-Jul-16, Michael Paquier wrote: > > > On Wed, Jul 16, 2025 at 09:30:31AM +0000, Bertrand Drouvot wrote: > > > > Indeed, the wait event names are MultixactOffsetSLRU and > > > Multix

Fix lwlock.c and wait_event_names.txt discrepancy

postgresql.org/message-id/aHZiHZ8sSQdHpyM6%40bdt-Laptop-13th-Gen-Intel-Core Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From ceef3dea35a28546bd4b6e4b5a78878640815c77 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot

Re: Improve LWLock tranche name visibility across backends

nd Sami have in mind. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Adding wait events statistics

Hi, On Fri, Jul 11, 2025 at 11:59:39AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-10 08:21:56 +, Bertrand Drouvot wrote: > > - Let's define a list of "cheap" and a list of "expensive" wait events > > - Change the code path to in

Re: Improve LWLock tranche name visibility across backends

t; wait_event_info = classId | eventId; > ``` > Do you think collision can still be possible? I meant to say collision between the trancheID and WaitEventCustomCounter->nextId Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Improve LWLock tranche name visibility across backends

use the passed in value, otherwise generate the next > eventId. > > I do like the latter approach more, what do you think? I think I do prefer it too, but in both cases we'll have to make sure there is no collision on the eventID (LWTRANCHE_FIRST_USER_DEFINED is currently 95). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Adding basic NUMA awareness

er;/* lock group leader, if I'm a member */ dlist_head lockGroupMembers; /* list of members, if I'm a leader */ dlist_node lockGroupLink; /* my member link, if I'm a member */ - - /* NUMA node */ - int numa_node; }; That could be back to 832 (the order does not make sense logically speaking though). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Improve LWLock tranche name visibility across backends

okup now proceeds in > the order of built-in names, the local list, and finally the shared memory. > The fallback name "extension" can still be returned if an extension does > not register a tranche. I did not look in details, but do you think we could make use of Wai

Re: Adding wait events statistics

Hi, On Wed, Jul 09, 2025 at 11:19:00AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-09 07:26:05 +, Bertrand Drouvot wrote: > > On Tue, Jul 08, 2025 at 10:19:07PM -0400, Andres Freund wrote: > > > On 2025-06-30 13:36:12 +, Bertrand Drouvot wrote: > > TB

Re: Adding basic NUMA awareness - Preliminary feedback and outline for an extensible approach

b's one)) could help to get performance gains depending on a "known" context (so less generic). So, probably having both could make sense but I think that they serve different purposes. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add os_page_num to pg_buffercache

ut a high-level "skeleton" > function, > with a couple of slots which would then be provided by the 3 variants. Yeah, I tried to avoid code duplication for the "os pages" related stuff in v1. I can check if more can be done (outside of the "os pages" related stuff).

Re: Adding wait events statistics

Hi, On Wed, Jul 09, 2025 at 09:56:10AM +0900, Michael Paquier wrote: > On Tue, Jul 08, 2025 at 03:25:26PM +0000, Bertrand Drouvot wrote: > > So, something like: > > > > ClassInfo LWLockClassInfo = { > > .numberOfEvents = NB_LWLOCK_WAIT_EVE

Re: Adding wait events statistics

Hi, On Tue, Jul 08, 2025 at 10:19:07PM -0400, Andres Freund wrote: > On 2025-06-30 13:36:12 +0000, Bertrand Drouvot wrote: > > * we don't want to allocate memory in some places where the counters are > > incremented (see 4feba03d8b9). We could still use the same implem

Re: Adding wait events statistics

Hi, On Tue, Jul 08, 2025 at 04:46:28AM +, Bertrand Drouvot wrote: > On Tue, Jul 08, 2025 at 09:36:53AM +0900, Michael Paquier wrote: > > Yes, you may need some level of meta-data generated for the wait > > classes generated when the perl script generating this data is run.

Re: Adding wait events statistics

Hi, On Tue, Jul 08, 2025 at 09:36:53AM +0900, Michael Paquier wrote: > On Mon, Jul 07, 2025 at 03:07:12PM +0000, Bertrand Drouvot wrote: > > > I think the question is: if the extension owner does not increment it, do > > we want > > to do it in core on their behalf? I&

Re: Adding wait events statistics

Hi, On Mon, Jul 07, 2025 at 02:56:29PM +0900, Michael Paquier wrote: > On Mon, Jun 30, 2025 at 01:36:12PM +0000, Bertrand Drouvot wrote: > > * Do we need to serialize the stats based on their names (as for > > PGSTAT_KIND_REPLSLOT)? This question is the same as "is the ord

Re: Remove unused wait_event_info parameter in FileStartReadV()

you're suggesting) but I think the closer to pg_preadv() the better. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Remove unused wait_event_info parameter in FileStartReadV()

t might need a deeper analysis though (to check if there are some outside of the callback cases). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 367acfe3608f4450e32ea306068d61d2c74389b9 Mon Sep 17 00:00:00

Re: Add os_page_num to pg_buffercache

Hi, On Wed, Jul 02, 2025 at 06:39:25AM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Jul 01, 2025 at 07:46:30PM +0200, Tomas Vondra wrote: > > On 7/1/25 19:20, Bertrand Drouvot wrote: > > > Now it's just a matter of extracting the necessary pieces from &g

Re: A assert failure when initdb with track_commit_timestamp=on

tstrapProcessingMode() is the right thing to do here. === 1 +* Don't bother to record commit_ts for Booststrap mode. typo: s/Booststrap/Bootstrap/ Also, grep on "Bootstrap mode" and "bootstrap mode" gives much more occurrences for the later, so maybe use "bo

Re: Fix inconsistency in the pg_buffercache documentation

Hi, On Wed, Jul 02, 2025 at 09:55:52AM +0200, Daniel Gustafsson wrote: > > On 2 Jul 2025, at 08:23, Bertrand Drouvot > > wrote: > > > > Hi hackers, > > > > While working on [1], I noticed an inconsistency in the pg_buffercache > > documentation: b

Re: Add os_page_num to pg_buffercache

Hi, On Tue, Jul 01, 2025 at 07:46:30PM +0200, Tomas Vondra wrote: > On 7/1/25 19:20, Bertrand Drouvot wrote: > > Now it's just a matter of extracting the necessary pieces from > > pg_buffercache_numa_pages() > > so that: > > > > * the new view could make

Fix inconsistency in the pg_buffercache documentation

cache_numa() has been introduced. [1]: https://www.postgresql.org/message-id/flat/Z/fFA2heH6lpSLlt%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon

Re: Add os_page_num to pg_buffercache

Hi, On Tue, Jul 01, 2025 at 06:45:37PM +0200, Tomas Vondra wrote: > On 7/1/25 18:34, Bertrand Drouvot wrote: > > But isn't the _numa view good enough for this? Sure, you need NUMA > support for it, and it may take a fair amount of time, but how often you > need to do suc

Re: Add os_page_num to pg_buffercache

Hi, On Tue, Jul 01, 2025 at 04:31:01PM +0200, Tomas Vondra wrote: > On 7/1/25 15:45, Bertrand Drouvot wrote: > > I took a quick look on this, Thanks for looking at it! > and I doubt we want to change the schema of > pg_buffercache like this. Adding columns is fine, but it seems

Re: Add os_page_num to pg_buffercache

Hi, On Thu, Apr 10, 2025 at 03:05:29PM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Apr 10, 2025 at 09:58:18AM -0500, Nathan Bossart wrote: > > On Thu, Apr 10, 2025 at 03:35:24PM +0200, Tomas Vondra wrote: > > > This seems like a good idea in principle, but at this po

  1   2   3   4   5   6   7   8   9   10   >