Re: Improve LWLock tranche name visibility across backends

e get rid of max_used_index and I think that this part is simpler and easier to understand. Thoughts? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Report reorder buffer size

. Also, should we log some of information to the server log? > But before I work on those, I would like to know others' opinions, > usefulness and acceptance. Thanks for working on it. I like the idea and also think that is an area where more details/metrics should be provided. Regards,

Re: BF mamba failure

for this relation. Is that vanilla > streaming replication with heap? I was wondering about adding the "generation" (introduced in 818119afccd3) to the above error log message, thoughts? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: BF mamba failure

Hi, On Thu, Aug 07, 2025 at 04:17:29PM +0900, Michael Paquier wrote: > On Thu, Aug 07, 2025 at 05:28:17AM +0000, Bertrand Drouvot wrote: > > On Thu, Aug 07, 2025 at 08:18:26AM +0900, Michael Paquier wrote: > >> On Mon, Aug 04, 2025 at 01:00:35PM +0300, Kouber Saparev wrote: >

Re: Adding per backend commit and rollback counters

Hi, On Mon, Aug 04, 2025 at 02:20:48PM +, Bertrand Drouvot wrote: > This patch is pretty straightforward as it relies on the existing per backend > statistics machinery that has been added in 9aea73fc61d (so that there is not > that much design to discuss). Still, while working

Re: Adding per backend commit and rollback counters

Hi, On Thu, Aug 07, 2025 at 08:17:26AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Aug 04, 2025 at 02:20:48PM +, Bertrand Drouvot wrote: > > This patch is pretty straightforward as it relies on the existing per > > backend > > statistics machinery that has be

Remove useless pointer advance in StatsShmemInit()

https://www.postgresql.org/message-id/CAA5RZ0ukmNd%2BC1jH4V6BGEea-wmyLxDtDE5QoEtfXd2W5HNHfQ%40mail.gmail.com Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From b8f56fe7df72830b0054c3652170716341d9dee2 Mon Sep 17 00:00:00 2

Re: Remove useless pointer advance in StatsShmemInit()

Hi, On Mon, Aug 18, 2025 at 06:13:05PM +0900, Michael Paquier wrote: > On Mon, Aug 18, 2025 at 09:04:59AM +0000, Bertrand Drouvot wrote: > > As StatsShmemInit() is existing code, let's fix it: the pointer is not used > > after > > its last advance, so that advance

Re: Improve LWLock tranche name visibility across backends

g_loc unless $ENABLE_LOG_WAIT; is ENABLE_LOG_WAIT needed as it does not change? [1]: https://www.postgresql.org/message-id/aKLsu2sdpnyeuSSc%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Improve LWLock tranche name visibility across backends

al need for dsarray for those). > I think it's a separate discussion at this point. OTOH, that would be a valid use case to introduce this new API but I'm not sure it's worth it given that the dshash looks good enough for our case (even if not ideal though). [1]: https://www.postg

Re: Per backend relation statistics tracking

Hi, On Tue, Aug 26, 2025 at 06:38:41AM +, Bertrand Drouvot wrote: > On Mon, Aug 25, 2025 at 08:28:04PM -0400, Andres Freund wrote: > > FWIW, I think this was done wrongly for the per-backend IO stats too. I've > > seen the increased overhead in profiles > > That

Re: Get rid of pgstat_count_backend_io_op*() functions

loop in pgstat_report_stat() That way the backend are flushed last and that solves your concern about custom stats kind. The backend would not be the "only" exception in pgstat_report_stat(), the db stats are already handled as exception with pgstat_update_dbstats(). Regards, -- Bertra

Re: Improve LWLock tranche name visibility across backends

ptr) % LWLOCK_PADDED_SIZE; in CreateLWLocks(), and just make use of CACHELINEALIGN(). Attached, a patch doing so. It applies on top of your v20. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 4720ffe5c804c6a

Re: Get rid of pgstat_count_backend_io_op*() functions

Hi, On Tue, Sep 02, 2025 at 08:19:22AM +0900, Michael Paquier wrote: > On Mon, Sep 01, 2025 at 02:07:27PM +0000, Bertrand Drouvot wrote: > > Instead, it now copies the IO pending stats to the backend IO pending stats > > when > > the pending IO stats are flushed. This behave

Re: Get rid of pgstat_count_backend_io_op*() functions

Hi, On Tue, Sep 02, 2025 at 07:41:59AM +, Bertrand Drouvot wrote: > On Tue, Sep 02, 2025 at 08:19:22AM +0900, Michael Paquier wrote: > > Removing both the function calls and the extra counter incrementations > > means to do the same thing as the WAL stats, where we have one &

Get rid of pgstat_count_backend_io_op*() functions

r pending stats are populated while other stats kinds are flushing. [1]: https://www.postgresql.org/message-id/7fhpds4xqk6bnudzmzkqi33pinsxammpljwde5gfkjdygvejrj%40ojkzfr7dxkmm Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.a

Re: Report reorder buffer size

Hi, On Mon, Aug 25, 2025 at 01:48:42PM +0530, Ashutosh Bapat wrote: > On Fri, Aug 22, 2025 at 11:43 AM Bertrand Drouvot > wrote: > > > > Yes as long as subtrans are not involved. But if say I've one transaction > > made of > > 1500 subtrans, I'd

Re: Per backend relation statistics tracking

with a > connection pooler with some specific configuration or when checking > transaction activity across multiple databases by joining with > pg_stat_activity? Yes exactly. And also to get XID (virtual transactions excluded) consumption rate per backend like in [1]. In [1], the number of X

Re: Per backend relation statistics tracking

form more index scans than UserB?" Right. I think we can see the relations as the "destinations" of the activity and the backends as the "source" of it. Tracking both sides help to visualize the activity from different angles. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Per backend relation statistics tracking

Hi, On Mon, Aug 25, 2025 at 08:28:04PM -0400, Andres Freund wrote: > Hi, > > On 2025-08-12 07:48:10 +, Bertrand Drouvot wrote: > > From 9e2f8cb9a87f1d9be91f2f39ef25fbb254944968 Mon Sep 17 00:00:00 2001 > > From: Bertrand Drouvot > > Date: Mon, 4 Aug 2025 08:14:0

Re: Per backend relation statistics tracking

Hi, On Tue, Aug 26, 2025 at 06:38:41AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Aug 25, 2025 at 08:28:04PM -0400, Andres Freund wrote: > > I don't like that this basically doubles the overhead of keeping stats by > > tracking everythign twice. The proper solutio

Re: Report reorder buffer size

Hi, On Tue, Aug 26, 2025 at 02:56:14PM +0530, Ashutosh Bapat wrote: > On Mon, Aug 25, 2025 at 6:28 PM Bertrand Drouvot > wrote: > > > I didn't know this is how it works. Thanks for correcting me. In that > > > case, I think we should > > > add a column in p

Re: Per backend relation statistics tracking

for more "sophisticated" ones (like pg_stat_get_backend_io()) that need multiple rows per backend. Having said that, we could imagine adding pg_stat_get_backend_wal() output to pg_stat_backend too. Thoughts? -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Add memory_limit_hits to pg_stat_replication_slots

277 |247 (1 row) Not sure I like memory_limit_hits that much, maybe work_mem_exceeded is better? Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.a

Re: Per backend relation statistics tracking

ll add a new pg_stat_backend_wal view if the current proposal goes in (for consistency). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Report bytes and transactions actually sent downtream

he > corresponding column has "plugin" prefix The way it is done makes sense to me. > 3. The names of new columns have the prefix "plugin_" but the internal > variables tracking those don't for the sake of brevity. If you prefer > to have the same

Re: Per backend relation statistics tracking

Hi, On Wed, Aug 27, 2025 at 01:57:13PM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Aug 26, 2025 at 04:55:09PM -0500, Sami Imseih wrote: > > I worry that a single view will grow very wide, and we will have to > > eventually > > split it. So we may as well

Re: Adding per backend commit and rollback counters

Hi, On Sun, Aug 10, 2025 at 07:47:09AM +, Bertrand Drouvot wrote: > To sum up, v3 contains: > > 0001 - > Adding per backend commit and rollback counters > 0002 - > Adding XID generation count per backend > 0003 - > Adding the pg_stat_backend view Following re

Re: Report reorder buffer size

Hi, On Tue, Aug 26, 2025 at 09:48:04AM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Aug 26, 2025 at 02:56:14PM +0530, Ashutosh Bapat wrote: > > On Mon, Aug 25, 2025 at 6:28 PM Bertrand Drouvot > > wrote: > > > > I didn't know this is how it wo

Re: Improve LWLock tranche name visibility across backends

ngth. I think that is much safer. Same remark as above but in RequestNamedLWLockTranche(). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: make LWLockCounter a global variable

*LWLockCounter; to use the global variable. That way we preserve the current memory layout and there is no need to change LWLockShmemSize(). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add os_page_num to pg_buffercache

See also the corresponding discussion [1]. > > Shall we add it to the function introduced in this patch as well? Yeah, I think so. Thanks for the ping, done in attached. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Servi

Re: Report reorder buffer size

Hi Ashutosh, On Thu, Aug 21, 2025 at 07:26:41PM +0530, Ashutosh Bapat wrote: > Hi Bertrand, > Thanks for your response. I am glad that you have found the proposal > to be generally useful. > > On Thu, Aug 14, 2025 at 3:50 PM Bertrand Drouvot > wrote: > > > > > U

Re: Improve LWLock tranche name visibility across backends

amically created tranches in shared memory. + * Any unused entries in the array will contain NULL. This variable is + * non-static so that postmaster.c can copy it to child processes in + * EXEC_BACKEND builds. "Any unused entries in the array will contain NULL": this is not true anym

Re: Improve LWLock tranche name visibility across backends

OK-ish. Using strcpy() might be OK too, as we already have validated the length, but maybe it would be safer to switch to strlcpy(), instead? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Per backend relation statistics tracking

Hi, On Tue, Aug 26, 2025 at 07:18:10AM -0400, Andres Freund wrote: > Hi, > > On 2025-08-26 06:38:41 +, Bertrand Drouvot wrote: > > > and IO related counters aren't > > > incremented remotely as often as the scan related counters are. > > > > You m

Generate pgstat_count_slru*() functions for slru using macros

named to match their associated counter. This is the same idea as 850f4b4, 8018ffb or 83a1a1b. Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add memory_limit_hits to pg_stat_replication_slots

Hi, On Thu, Sep 11, 2025 at 03:24:54PM -0700, Masahiko Sawada wrote: > On Wed, Aug 27, 2025 at 12:26 AM Bertrand Drouvot > wrote: > > > > Looking forward to your feedback, > > Yes, Thanks for looking at it! > it's a quite different situation in two cases: sp

Re: Add memory_limit_hits to pg_stat_replication_slots

On Mon, Sep 22, 2025 at 05:21:35PM +0530, shveta malik wrote: > On Mon, Sep 22, 2025 at 4:35 PM Amit Kapila wrote: > > > > On Mon, Sep 22, 2025 at 1:41 PM Bertrand Drouvot > > wrote: > > > > > > > > > > > Since other statistics counter names

Re: Report bytes and transactions actually sent downtream

Hi, On Thu, Sep 25, 2025 at 01:01:55PM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Sep 25, 2025 at 10:16:35AM +0530, Ashutosh Bapat wrote: > > Do you have any further review comments? > > Not right now. I'll give it another look by early next week the latest. &

Re: Report bytes and transactions actually sent downtream

Hi, On Thu, Sep 25, 2025 at 10:16:35AM +0530, Ashutosh Bapat wrote: > On Wed, Sep 24, 2025 at 8:11 PM Bertrand Drouvot > wrote: > > > > === 4 > > > > +extern Size ReorderBufferChangeSize(ReorderBufferChange *change); > > > > Another approach could be t

Re: Get rid of pgstat_count_backend_io_op*() functions

Hi, On Thu, Sep 25, 2025 at 03:42:33PM +0900, Michael Paquier wrote: > On Wed, Sep 24, 2025 at 07:48:32AM +0000, Bertrand Drouvot wrote: > > On Wed, Sep 03, 2025 at 07:33:37AM +, Bertrand Drouvot wrote: > >> As far the ordering concern for v1, what about: > >> &

Re: Report bytes and transactions actually sent downtream

Hi, On Wed, Sep 24, 2025 at 03:37:07PM +0530, Ashutosh Bapat wrote: > On Wed, Sep 24, 2025 at 1:55 PM Bertrand Drouvot > wrote: > > Right. But, in the example above, do you consider "skip-empty-xacts" as > > "core" > > or "plugin" filter

Re: Report bytes and transactions actually sent downtream

e shall skip counting msgtype as well. Agree. Maybe mention in the doc that metadata (including msgtype) bytes are not taken into account? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Report bytes and transactions actually sent downtream

Hi, On Fri, Sep 26, 2025 at 06:14:28PM +0530, Ashutosh Bapat wrote: > On Fri, Sep 26, 2025 at 4:43 PM Bertrand Drouvot > wrote: > > > > > > === 2 > > > > Issue 1 is that before any decoding happens, pg_stat_replication_slots is > > still > > sh

Re: Add memory_limit_hits to pg_stat_replication_slots

Hi, On Thu, Sep 25, 2025 at 12:14:04PM -0700, Masahiko Sawada wrote: > On Thu, Sep 25, 2025 at 3:17 AM Bertrand Drouvot > wrote: > > > > That probably means that mem_exceeded_count would need to be increased. > > > > What do you think? > > Right. But

Re: Add memory_limit_hits to pg_stat_replication_slots

Hi Evan, On Fri, Sep 26, 2025 at 02:34:58PM +0800, Chao Li wrote: > Hi Bertrand, > > Thanks for the patch. The patch overall goods look to me. Just a few small > comments: Thanks for looking at it! > > On Sep 25, 2025, at 18:17, Bertrand Drouvot > > wrote: > >

Re: Get rid of pgstat_count_backend_io_op*() functions

Hi, On Wed, Sep 03, 2025 at 07:33:37AM +, Bertrand Drouvot wrote: > Hi, > > On Wed, Sep 03, 2025 at 02:47:51PM +0900, Michael Paquier wrote: > > On Tue, Sep 02, 2025 at 12:42:54PM -0400, Andres Freund wrote: > > > I think that's the wrong direction to go. Di

Re: relfilenode statistics

Hi, On Wed, Oct 01, 2025 at 08:05:16AM +0900, Michael Paquier wrote: > On Tue, Sep 30, 2025 at 10:13:57AM +0000, Bertrand Drouvot wrote: > > As far Michael's concern about adding a new field in the hash key, as 8 > > bytes > > is allocated for the object ID, then we c

Re: relfilenode statistics

h key, as 8 bytes is allocated for the object ID, then we can go with: dboid (linked to RelFileLocator's dbOid) objoid (linked to RelFileLocator's spcOid and to the RelFileLocator's relNumber) and avoid adding a new field in the key. [1]: https://www.postgresql.org/message-id

Add stats_reset to pg_stat_all_tables|indexes and related views

the stats_reset field in the views that are concerned - Updates the documentation - Updates some tests Regards, [1]: https://postgr.es/m/ZlGYokUIlERemvpB%40ip-10-97-1-34.eu-west-3.compute.internal -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web

Re: Report bytes and transactions actually sent downtream

Hi, On Mon, Sep 29, 2025 at 12:54:24PM +0530, Ashutosh Bapat wrote: > On Fri, Sep 26, 2025 at 10:28 PM Bertrand Drouvot > wrote: > > > > > > > > > > > > I don't think this is an issue. There is no way for the core to tell > > > whether t

Re: Fix locking issue with fixed-size stats template in injection_points

gt; > Thoughts or comments? Patch LGTM. Remark: I like the "popo" prefix in your test ;-) Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add stats_reset to pg_stat_all_tables|indexes and related views

fail on repeated runs, may it not? It is a shared catalog. Yeah this one may need to be done differently. I just removed it as it does not provide extra value here. [1]: https://www.postgresql.org/message-id/flat/9837222c1001240837r5c103519lc6a74c37be5f1831%40mail.gmail.com Regards, -- Bertrand Drouvot PostgreSQL Contributors

Re: [BUG]: the walsender does not update its IO statistics until it exits

Hi, On Tue, Sep 30, 2025 at 04:37:25PM +0900, Michael Paquier wrote: > On Tue, Sep 30, 2025 at 07:14:14AM +0000, Bertrand Drouvot wrote: > > On Thu, Feb 27, 2025 at 12:09:46PM +0900, Michael Paquier wrote: > >> Agreed that something in the lines of non-transaction update of the

Re: Add stats_reset to pg_stat_all_tables|indexes and related views

Hi, On Fri, Oct 03, 2025 at 04:50:07PM +0900, Michael Paquier wrote: > On Fri, Oct 03, 2025 at 05:33:00AM +0000, Bertrand Drouvot wrote: > > On Fri, Oct 03, 2025 at 10:24:39AM +0900, Michael Paquier wrote: > >> Makes sense to me. This matters in terms of coverage for HEAD, >

Re: Add memory_limit_hits to pg_stat_replication_slots

se of background activity? I think that the question could be the same for spill_txns and spill_count. It seems to have been working fine (that way) since this test exists (added in 072ee847ad4c) but I think that you raised a good point. Sawada-San, what do you think about this particular test, is it

Re: Report bytes and transactions actually sent downtream

ated by version X. Attached patch fixes > issue 2 by zero'ing out the stats when the plugin does not report the > statistics. +#define REPLSLOT_SET_TO_ZERO(fld) statent->fld = 0 It looks like that the associated "undef" is missing. Regards, -- Bertrand Drouvot Postg

Re: Add memory_limit_hits to pg_stat_replication_slots

Hi, On Thu, Oct 02, 2025 at 04:39:40PM -0700, Masahiko Sawada wrote: > On Thu, Sep 25, 2025 at 10:26 PM Bertrand Drouvot > wrote: > > > Agreed. It would be better to show a raw statistic so that users can > use the number as they want. > > I've made a small comment

Re: Add memory_limit_hits to pg_stat_replication_slots

Hi, On Mon, Oct 06, 2025 at 10:50:52AM +0530, Ashutosh Bapat wrote: > On Fri, Oct 3, 2025 at 11:48 PM Masahiko Sawada wrote: > > > > On Fri, Oct 3, 2025 at 9:26 AM Ashutosh Bapat > > wrote: > > > > > > On Fri, Oct 3, 2025 at 6:45 PM Bertrand D

<    5   6   7   8   9   10