Re: per backend WAL statistics

2025-04-06 Thread Michael Paquier
On Mon, Mar 31, 2025 at 07:42:19AM +, Bertrand Drouvot wrote: > I think we can simply move the pgstat_fetch_stat_backend() call at the end > of pgstat_fetch_stat_backend_by_pid(), like in the attached. With this in > place > the issue is fixed on my side. > > Thoughts? Confirmed. I agree th

Re: per backend WAL statistics

2025-03-31 Thread Michael Paquier
> On Mar 31, 2025, at 16:42, Bertrand Drouvot > wrote: > I think we can simply move the pgstat_fetch_stat_backend() call at the end > of pgstat_fetch_stat_backend_by_pid(), like in the attached. With this in > place > the issue is fixed on my side. Thanks for the patch, I’ll check all that next

Re: per backend WAL statistics

2025-03-31 Thread Bertrand Drouvot
Hi, On Sat, Mar 29, 2025 at 07:14:16AM +0900, Michael Paquier wrote: > On Fri, Mar 28, 2025 at 09:00:00PM +0200, Alexander Lakhin wrote: > > Please try the following query: > > BEGIN; > > SET LOCAL stats_fetch_consistency = snapshot; > > SELECT * FROM pg_stat_get_backend_wal(pg_backend_pid()); Th

Re: per backend WAL statistics

2025-03-28 Thread Michael Paquier
On Fri, Mar 28, 2025 at 09:00:00PM +0200, Alexander Lakhin wrote: > Please try the following query: > BEGIN; > SET LOCAL stats_fetch_consistency = snapshot; > SELECT * FROM pg_stat_get_backend_wal(pg_backend_pid()); > > with sanitizers (or under Valgrind). When I run it, I get: > 2025-03-28 18:38:

Re: per backend WAL statistics

2025-03-28 Thread Alexander Lakhin
Hello Michael, 11.03.2025 02:06, Michael Paquier wrote: And I guess that we're OK here, so applied. That should be the last one. Please try the following query: BEGIN; SET LOCAL stats_fetch_consistency = snapshot; SELECT * FROM pg_stat_get_backend_wal(pg_backend_pid()); with sanitizers (or u

Re: per backend WAL statistics

2025-03-15 Thread Michael Paquier
On Wed, Mar 12, 2025 at 05:37:11AM +, Bertrand Drouvot wrote: > Thanks for the report! I think that it's better to remove the > PendingBackendStats > initializer (instead of adding extra braces). The reason is that I'm concerned > about padding bytes (that could be added to the struct in the f

Re: per backend WAL statistics

2025-03-13 Thread Nazir Bilal Yavuz
Hi, On Mon, 10 Mar 2025 at 17:43, Bertrand Drouvot wrote: > On Mon, Mar 10, 2025 at 03:08:49PM +0300, Nazir Bilal Yavuz wrote: > > > I think that every time we flush IO or WAL stats, we want(?) to flush > > backend stats as well, > > Yeah, I think that's happening anyway. > > > so would it make s

Re: per backend WAL statistics

2025-03-12 Thread Tom Lane
Michael Paquier writes: > And I guess that we're OK here, so applied. That should be the last > one. Quite a few buildfarm members are not happy about the initialization that 9a8dd2c5a added to PendingBackendStats. For instance [1]: gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith -W

Re: per backend WAL statistics

2025-03-11 Thread Bertrand Drouvot
Hi, On Tue, Mar 11, 2025 at 11:14:24PM -0400, Tom Lane wrote: > Michael Paquier writes: > > And I guess that we're OK here, so applied. That should be the last > > one. > > Quite a few buildfarm members are not happy about the initialization > that 9a8dd2c5a added to PendingBackendStats. For i

Re: per backend WAL statistics

2025-03-11 Thread Bertrand Drouvot
Hi, On Mon, Mar 10, 2025 at 03:08:49PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > Thank you for working on this! > > I just started reading the code and have a couple of questions. Thanks for looking at it! > I think that every time we flush IO or WAL stats, we want(?) to flush > backend stats

Re: per backend WAL statistics

2025-03-11 Thread Bertrand Drouvot
Hi, On Tue, Mar 11, 2025 at 09:06:27AM +0900, Michael Paquier wrote: > On Mon, Mar 10, 2025 at 11:52:26AM +, Bertrand Drouvot wrote: > > Hi, > > > > On Mon, Mar 10, 2025 at 04:46:53PM +0900, Michael Paquier wrote: > > > On Sat, Mar 08, 2025 at 07:53:04AM +, Bertrand Drouvot wrote: > > > >

Re: per backend WAL statistics

2025-03-10 Thread Michael Paquier
On Mon, Mar 10, 2025 at 11:52:26AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Mar 10, 2025 at 04:46:53PM +0900, Michael Paquier wrote: > > On Sat, Mar 08, 2025 at 07:53:04AM +, Bertrand Drouvot wrote: > > > That would not be an issue should we only access the struct > > > fields in the c

Re: per backend WAL statistics

2025-03-10 Thread Nazir Bilal Yavuz
Hi, Thank you for working on this! I just started reading the code and have a couple of questions. I think that every time we flush IO or WAL stats, we want(?) to flush backend stats as well, so would it make sense to move pgstat_flush_backend() calls to inside of pgstat_flush_io() and pgstat_wa

Re: per backend WAL statistics

2025-03-10 Thread Bertrand Drouvot
Hi, On Mon, Mar 10, 2025 at 04:46:53PM +0900, Michael Paquier wrote: > On Sat, Mar 08, 2025 at 07:53:04AM +, Bertrand Drouvot wrote: > > That would not be an issue should we only access the struct > > fields in the code, but that's not the case as we're making use of > > pg_memory_is_all_zeros

Re: per backend WAL statistics

2025-03-10 Thread Michael Paquier
On Sat, Mar 08, 2025 at 07:53:04AM +, Bertrand Drouvot wrote: > That would not be an issue should we only access the struct > fields in the code, but that's not the case as we're making use of > pg_memory_is_all_zeros() on it. It does not hurt to keep it as it is, honestly. I've reviewed the

Re: per backend WAL statistics

2025-03-07 Thread Bertrand Drouvot
Hi, On Sat, Mar 08, 2025 at 10:57:38AM +0900, Michael Paquier wrote: > On Fri, Mar 07, 2025 at 08:33:04AM +, Bertrand Drouvot wrote: > > But when it's time to flush, then pgstat_backend_have_pending_cb() checks > > for zeros in PendingBackendStats *without* any check on the backend type. > >

Re: per backend WAL statistics

2025-03-07 Thread Michael Paquier
On Thu, Mar 06, 2025 at 10:33:52AM +, Bertrand Drouvot wrote: > Indeed, there is no reason for pgstat_backend_have_pending_cb() to return > true if > pgstat_tracks_backend_bktype() is not satisfied. > > So it deserves a dedicated patch to fix this already existing issue: > 0001 attached. p

Re: per backend WAL statistics

2025-03-07 Thread Michael Paquier
On Fri, Mar 07, 2025 at 08:33:04AM +, Bertrand Drouvot wrote: > But when it's time to flush, then pgstat_backend_have_pending_cb() checks > for zeros in PendingBackendStats *without* any check on the backend type. > > I think the issue is "masked" on HEAD because PendingBackendStats is > proba

Re: per backend WAL statistics

2025-03-07 Thread Bertrand Drouvot
Hi, On Fri, Mar 07, 2025 at 02:42:13PM +0900, Michael Paquier wrote: > On Thu, Mar 06, 2025 at 10:33:52AM +, Bertrand Drouvot wrote: > > Indeed, there is no reason for pgstat_backend_have_pending_cb() to return > > true if > > pgstat_tracks_backend_bktype() is not satisfied. > > > > So it d

Re: per backend WAL statistics

2025-03-06 Thread Bertrand Drouvot
Hi, On Wed, Mar 05, 2025 at 05:26:40PM +, Bertrand Drouvot wrote: > So yeah, back to the issue, we have to pay more attention for the WAL stats > because pgWalUsage is "incremented" without any check with > pgstat_tracks_backend_bktype() > (that's not the case for the IO stats where the count

Re: per backend WAL statistics

2025-03-06 Thread Xuneng Zhou
Hi, Bertrand Drouvot 于2025年3月5日周三 21:03写道: > Hi, > > On Wed, Mar 05, 2025 at 05:45:57PM +0800, Xuneng Zhou wrote: > > Subject: Clarification Needed on WAL Pending Checks in Patchset > > > > Hi, > > > > Thank you for the patchset. I’ve spent some time learning and reviewing > it > > and have 2 co

Re: per backend WAL statistics

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 13:03:07 +, Bertrand Drouvot wrote: > But yeah, if 0002 in [1] does not go in, then your concern is valid, so adding > the extra check in the attached. This crashes in cfbot: https://cirrus-ci.com/task/5111872610893824 [13:42:37.315] src/tools/ci/cores_backtrace.sh freebsd

Re: per backend WAL statistics

2025-03-05 Thread Bertrand Drouvot
Hi, On Wed, Mar 05, 2025 at 09:18:16AM -0500, Andres Freund wrote: > Hi, > > On 2025-03-05 13:03:07 +, Bertrand Drouvot wrote: > > But yeah, if 0002 in [1] does not go in, then your concern is valid, so > > adding > > the extra check in the attached. > > This crashes in cfbot: > > https://

Re: per backend WAL statistics

2025-03-05 Thread Bertrand Drouvot
Hi, On Wed, Mar 05, 2025 at 05:45:57PM +0800, Xuneng Zhou wrote: > Subject: Clarification Needed on WAL Pending Checks in Patchset > > Hi, > > Thank you for the patchset. I’ve spent some time learning and reviewing it > and have 2 comments. Thanks for looking at it! > I noticed that in patches

Re: per backend WAL statistics

2025-03-05 Thread Xuneng Zhou
Subject: Clarification Needed on WAL Pending Checks in Patchset Hi, Thank you for the patchset. I’ve spent some time learning and reviewing it and have 2 comments. I'm new to PostgreSQL hacking, so please bear with me if I make mistakes or say something that seems trivial. I noticed that in pat

Re: per backend WAL statistics

2025-03-04 Thread Michael Paquier
On Tue, Mar 04, 2025 at 08:48:28AM +, Bertrand Drouvot wrote: > s/report/Report/ and s/system/system./? to be consistent with the other single > line comments around. Right. > Yeah, I also think that's the right location. We could be more optimal for the WAL receiver if we add more timestamp

Re: per backend WAL statistics

2025-03-04 Thread Bertrand Drouvot
Hi, On Tue, Mar 04, 2025 at 09:28:27AM +0900, Michael Paquier wrote: > On Mon, Mar 03, 2025 at 09:17:30AM +, Bertrand Drouvot wrote: > > On Mon, Mar 03, 2025 at 10:48:23AM +0900, Michael Paquier wrote: > >> Something that's still not quite right is that the WAL receiver and > >> the WAL summar

Re: per backend WAL statistics

2025-03-03 Thread Michael Paquier
On Mon, Mar 03, 2025 at 09:17:30AM +, Bertrand Drouvot wrote: > On Mon, Mar 03, 2025 at 10:48:23AM +0900, Michael Paquier wrote: >> Something that's still not quite right is that the WAL receiver and >> the WAL summarizer do not call pgstat_report_wal() at all, so we don't >> report much data a

Re: per backend WAL statistics

2025-03-03 Thread Bertrand Drouvot
Hi, On Mon, Mar 03, 2025 at 09:17:30AM +, Bertrand Drouvot wrote: > hmm, that would work as long as PGSTAT_BACKEND_FLUSH_ALL represents things > that need to be called from pgstat_report_wal(). But I think that's open > door for issue should be add a new PGSTAT_BACKEND_FLUSH_XXX where XXX is n

Re: per backend WAL statistics

2025-03-03 Thread Bertrand Drouvot
Hi, On Mon, Mar 03, 2025 at 10:48:23AM +0900, Michael Paquier wrote: > On Fri, Feb 28, 2025 at 09:26:08AM +, Bertrand Drouvot wrote: > > Also attaching the patch I mentioned up-thread to address some of Rahila's > > comments ([1]): It adds a AuxiliaryPidGetProc() call in > > pgstat_fetch_stat

Re: per backend WAL statistics

2025-03-02 Thread Michael Paquier
On Fri, Feb 28, 2025 at 09:26:08AM +, Bertrand Drouvot wrote: > Also attaching the patch I mentioned up-thread to address some of Rahila's > comments ([1]): It adds a AuxiliaryPidGetProc() call in > pgstat_fetch_stat_backend_by_pid() > and pg_stat_reset_backend_stats(). I think that fully make

Re: per backend WAL statistics

2025-02-28 Thread Michael Paquier
On Fri, Feb 28, 2025 at 09:26:08AM +, Bertrand Drouvot wrote: > Also attaching the patch I mentioned up-thread to address some of Rahila's > comments ([1]): It adds a AuxiliaryPidGetProc() call in > pgstat_fetch_stat_backend_by_pid() > and pg_stat_reset_backend_stats(). I think that fully make

Re: per backend WAL statistics

2025-02-28 Thread Bertrand Drouvot
Hi, On Fri, Feb 28, 2025 at 11:34:14AM +0900, Michael Paquier wrote: > On Thu, Feb 27, 2025 at 07:47:09AM +, Bertrand Drouvot wrote: > > That's how I did it initially but decided to move it to pgstat_backend.c. > > The > > reason was that it's fully linked to "per backend" stats and that ther

Re: per backend WAL statistics

2025-02-27 Thread Michael Paquier
On Thu, Feb 27, 2025 at 07:47:09AM +, Bertrand Drouvot wrote: > That's how I did it initially but decided to move it to pgstat_backend.c. The > reason was that it's fully linked to "per backend" stats and that there is > no SQL api on top of it (while I think that's the case for almost all the

Re: per backend WAL statistics

2025-02-26 Thread Bertrand Drouvot
Hi, On Thu, Feb 27, 2025 at 12:02:51PM +0900, Michael Paquier wrote: > 0001 was OK, so done. Thanks! > In 0002, couldn't it be better to have the pg_stat_get_backend_stats() > static in pgstatfuncs.c? In 0003, pg_stat_get_backend_wal() is also > in pgstatfuncs.c, meaning that all the callers of

Re: per backend WAL statistics

2025-02-26 Thread Michael Paquier
On Wed, Feb 26, 2025 at 10:59:11AM +, Bertrand Drouvot wrote: > Yup. That's what we've done in pg_stat_io_build_tuples() too (ff7c40d7fd6). > Without this we'd get "2000-01-01 00:00:00+00" in the stats_reset field of > pg_stat_get_backend_wal() and pg_stat_get_backend_io(). Right, forgot about

Re: per backend WAL statistics

2025-02-26 Thread Bertrand Drouvot
Hi, On Wed, Feb 26, 2025 at 04:52:13PM +0900, Michael Paquier wrote: > On Tue, Feb 25, 2025 at 03:00:35PM +, Bertrand Drouvot wrote: > > That makes fully sense. Done in 0004 attached. Somehow related to that, I've > > a patch in progress to address some of Rahila's comments ([1]) (the one > >

Re: per backend WAL statistics

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 03:00:35PM +, Bertrand Drouvot wrote: > That makes fully sense. Done in 0004 attached. Somehow related to that, I've > a patch in progress to address some of Rahila's comments ([1]) (the one > related > to the AuxiliaryPidGetProc() call is relevant specially since a051e

Re: per backend WAL statistics

2025-02-25 Thread Bertrand Drouvot
Hi, On Tue, Feb 25, 2025 at 03:50:38PM +0900, Michael Paquier wrote: > On Mon, Feb 24, 2025 at 09:07:39AM +, Bertrand Drouvot wrote: > > Now that 2421e9a51d2 is in, let's resume working in this thread. PFA a > > rebase to > > make the CF bot happy. Nothing has changed since V7, V8 only remove

Re: per backend WAL statistics

2025-02-24 Thread Michael Paquier
On Mon, Feb 24, 2025 at 09:07:39AM +, Bertrand Drouvot wrote: > Now that 2421e9a51d2 is in, let's resume working in this thread. PFA a rebase > to > make the CF bot happy. Nothing has changed since V7, V8 only removes > "v7-0001" ( > as part of 2421e9a51d2), so that v8-000N is nothing but v7-

Re: per backend WAL statistics

2025-02-24 Thread Bertrand Drouvot
Hi, On Wed, Feb 19, 2025 at 07:28:49AM +0900, Michael Paquier wrote: > On Tue, Feb 18, 2025 at 10:45:29AM +, Bertrand Drouvot wrote: > > Agree, done that way in the dedicated thread ([1]). > > > > [1]: > > https://www.postgresql.org/message-id/flat/Z7RkQ0EfYaqqjgz/%40ip-10-97-1-34.eu-west-3.

Re: per backend WAL statistics

2025-02-18 Thread Michael Paquier
On Tue, Feb 18, 2025 at 10:45:29AM +, Bertrand Drouvot wrote: > Agree, done that way in the dedicated thread ([1]). > > [1]: > https://www.postgresql.org/message-id/flat/Z7RkQ0EfYaqqjgz/%40ip-10-97-1-34.eu-west-3.compute.internal Thanks for splitting this part into its own thread. -- Michael

Re: per backend WAL statistics

2025-02-18 Thread Bertrand Drouvot
Hi, On Tue, Feb 18, 2025 at 08:34:32AM +0900, Michael Paquier wrote: > On Mon, Feb 17, 2025 at 03:14:59PM +, Bertrand Drouvot wrote: > > PFA the whole picture. 0001 is implementing the fields removal in > > pg_stat_wal > > (and also PendingWalStats). I think that's ok given the backend's type

Re: per backend WAL statistics

2025-02-17 Thread Michael Paquier
On Mon, Feb 17, 2025 at 03:14:59PM +, Bertrand Drouvot wrote: > PFA the whole picture. 0001 is implementing the fields removal in pg_stat_wal > (and also PendingWalStats). I think that's ok given the backend's type for > which > pgstat_tracks_io_bktype() returns false. But now you make me doub

Re: per backend WAL statistics

2025-02-17 Thread Bertrand Drouvot
Hi, On Mon, Feb 17, 2025 at 07:59:26AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Feb 17, 2025 at 04:25:46PM +0900, Michael Paquier wrote: > > On Mon, Feb 17, 2025 at 06:59:40AM +, Bertrand Drouvot wrote: > > > There is still something that would simplify what is done here: it's the > >

Re: per backend WAL statistics

2025-02-16 Thread Bertrand Drouvot
Hi, On Mon, Feb 17, 2025 at 04:25:46PM +0900, Michael Paquier wrote: > On Mon, Feb 17, 2025 at 06:59:40AM +, Bertrand Drouvot wrote: > > There is still something that would simplify what is done here: it's the > > "the elimination of the write & sync columns for pg_stat_wal" mentioned in > >

Re: per backend WAL statistics

2025-02-16 Thread Michael Paquier
On Mon, Feb 17, 2025 at 06:59:40AM +, Bertrand Drouvot wrote: > There is still something that would simplify what is done here: it's the > "the elimination of the write & sync columns for pg_stat_wal" mentioned in > [2]. Yeah, still you cannot just remove them because the data tracked in pg_s

Re: per backend WAL statistics

2025-02-16 Thread Nazir Bilal Yavuz
Hi, Thank you for working on this! On Mon, 17 Feb 2025 at 09:59, Bertrand Drouvot wrote: > > There is still something that would simplify what is done here: it's the > "the elimination of the write & sync columns for pg_stat_wal" mentioned in > [2]. > > I'll add it in the new patch serie for th

Re: per backend WAL statistics

2025-02-16 Thread Bertrand Drouvot
Hi, On Thu, Feb 06, 2025 at 10:28:48AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Feb 06, 2025 at 10:38:55AM +0900, Michael Paquier wrote: > > On Wed, Feb 05, 2025 at 02:28:08PM +, Bertrand Drouvot wrote: > > > Agree, I'll start a dedicated thread for that. > > > > Thanks. > > Done in

Re: per backend WAL statistics

2025-02-06 Thread Bertrand Drouvot
Hi, On Thu, Feb 06, 2025 at 10:38:55AM +0900, Michael Paquier wrote: > On Wed, Feb 05, 2025 at 02:28:08PM +, Bertrand Drouvot wrote: > > Agree, I'll start a dedicated thread for that. > > Thanks. Done in [1]. [1]: https://www.postgresql.org/message-id/flat/Z6SOha5YFFgvpwQY%40ip-10-97-1-34.

Re: per backend WAL statistics

2025-02-05 Thread Michael Paquier
On Wed, Feb 05, 2025 at 02:28:08PM +, Bertrand Drouvot wrote: > Agree, I'll start a dedicated thread for that. Thanks. -- Michael signature.asc Description: PGP signature

Re: per backend WAL statistics

2025-02-05 Thread Bertrand Drouvot
Hi, On Wed, Feb 05, 2025 at 07:31:13PM +0900, Michael Paquier wrote: > On Wed, Feb 05, 2025 at 10:22:55AM +, Bertrand Drouvot wrote: > > So, wal_buffers_full has been introduced after the WalUsage structure was > > there but I don't see any reason in the emails as to why it's not in the > > W

Re: per backend WAL statistics

2025-02-05 Thread Michael Paquier
On Wed, Feb 05, 2025 at 10:22:55AM +, Bertrand Drouvot wrote: > So, wal_buffers_full has been introduced after the WalUsage structure was > there but I don't see any reason in the emails as to why it's not in the > WalUsage > structure (I might have missed it though). > > I think that this pr

Re: per backend WAL statistics

2025-02-05 Thread Bertrand Drouvot
Hi, On Wed, Feb 05, 2025 at 11:16:15AM +0900, Michael Paquier wrote: > On Tue, Feb 04, 2025 at 08:49:41AM +, Bertrand Drouvot wrote: > > can be extracted from pg_stat_get_backend_io(), so there is no need to > > duplicate > > this information. The same comment could be done for pg_stat_wal an

Re: per backend WAL statistics

2025-02-04 Thread Michael Paquier
On Tue, Feb 04, 2025 at 08:49:41AM +, Bertrand Drouvot wrote: > I think that: > > wal_write (and wal_write_time) > wal_sync (and wal_sync_time) Right. We are not able to get this data from XLogWrite() and issue_xlog_fsync(), so there is no need to duplicate that anymore in your patch. > ca

Re: per backend WAL statistics

2025-02-04 Thread Bertrand Drouvot
Hi, On Thu, Jan 23, 2025 at 09:57:50AM +, Bertrand Drouvot wrote: > On Thu, Jan 23, 2025 at 05:05:30PM +0900, Michael Paquier wrote: > > As far I got it from a code > > read, prevWalUsage, prevBackendWalUsage and their local trackings in > > pgstat_backend.c and pgstat_wal.c rely on instrument

Re: per backend WAL statistics

2025-01-29 Thread Bertrand Drouvot
Hi, On Wed, Jan 29, 2025 at 01:14:09PM +0530, Rahila Syed wrote: > Hi, > > Thank you for the patchset. Having per-backend WAL statistics, > in addition to cluster-wide ones, is useful. Thanks for looking at it! > I had a few comments while looking at v6-0003-* patch. > > + /* > + * This could

Re: per backend WAL statistics

2025-01-28 Thread Rahila Syed
Hi, Thank you for the patchset. Having per-backend WAL statistics, in addition to cluster-wide ones, is useful. I had a few comments while looking at v6-0003-* patch. + /* + * This could be an auxiliary process but these do not report backend + * statistics due to pgstat_tracks_backend_bktype(),

Re: per backend WAL statistics

2025-01-23 Thread Bertrand Drouvot
Hi, On Thu, Jan 23, 2025 at 05:05:30PM +0900, Michael Paquier wrote: > On Tue, Jan 21, 2025 at 07:19:55AM +, Bertrand Drouvot wrote: > > PFA v6 that now relies on the new PendingBackendStats variable introduced in > > 4feba03d8b9. > > > > Remark: I moved PendingBackendStats back to pgstat.h b

Re: per backend WAL statistics

2025-01-23 Thread Michael Paquier
On Tue, Jan 21, 2025 at 07:19:55AM +, Bertrand Drouvot wrote: > PFA v6 that now relies on the new PendingBackendStats variable introduced in > 4feba03d8b9. > > Remark: I moved PendingBackendStats back to pgstat.h because I think that the > "simple" pending stats increment that we are adding in

Re: per backend WAL statistics

2025-01-20 Thread Bertrand Drouvot
Hi, On Fri, Jan 17, 2025 at 08:43:57AM +0900, Michael Paquier wrote: > On Thu, Jan 16, 2025 at 12:44:20PM -0500, Andres Freund wrote: > > On 2025-01-16 17:11:09 +, Bertrand Drouvot wrote: > >> So, do you think that the initial proposal that has been made here (See > >> R1. in > >> [2]) i.e ma

Re: per backend WAL statistics

2025-01-16 Thread Michael Paquier
On Thu, Jan 16, 2025 at 12:44:20PM -0500, Andres Freund wrote: > On 2025-01-16 17:11:09 +, Bertrand Drouvot wrote: >> So, do you think that the initial proposal that has been made here (See R1. >> in >> [2]) i.e make use of a new PendingBackendWalStats variable: > > Well, I think this first n

Re: per backend WAL statistics

2025-01-16 Thread Andres Freund
Hi, On 2025-01-16 17:11:09 +, Bertrand Drouvot wrote: > On Thu, Jan 16, 2025 at 11:38:47AM -0500, Andres Freund wrote: > > Hi, > > > > On 2025-01-16 15:59:31 +, Bertrand Drouvot wrote: > > > On Wed, Jan 15, 2025 at 03:11:32PM +0900, Michael Paquier wrote: > > > > On Fri, Jan 10, 2025 at 0

Re: per backend WAL statistics

2025-01-16 Thread Bertrand Drouvot
Hi, On Thu, Jan 16, 2025 at 11:38:47AM -0500, Andres Freund wrote: > Hi, > > On 2025-01-16 15:59:31 +, Bertrand Drouvot wrote: > > On Wed, Jan 15, 2025 at 03:11:32PM +0900, Michael Paquier wrote: > > > On Fri, Jan 10, 2025 at 09:40:38AM +, Bertrand Drouvot wrote: > > > + * WAL pending sta

Re: per backend WAL statistics

2025-01-16 Thread Andres Freund
Hi, On 2025-01-16 15:59:31 +, Bertrand Drouvot wrote: > On Wed, Jan 15, 2025 at 03:11:32PM +0900, Michael Paquier wrote: > > On Fri, Jan 10, 2025 at 09:40:38AM +, Bertrand Drouvot wrote: > > + * WAL pending statistics are incremented inside a critical section > > + * (see XLogWrite()), so

Re: per backend WAL statistics

2025-01-16 Thread Bertrand Drouvot
Hi, On Wed, Jan 15, 2025 at 03:11:32PM +0900, Michael Paquier wrote: > On Fri, Jan 10, 2025 at 09:40:38AM +, Bertrand Drouvot wrote: > > Please find attached v4 taking into account 2c14037bb5. > > +} PgStat_WalCounters; > + > +typedef struct PgStat_WalStats > +{ > + PgStat_WalCounters wal

Re: per backend WAL statistics

2025-01-14 Thread Michael Paquier
On Fri, Jan 10, 2025 at 09:40:38AM +, Bertrand Drouvot wrote: > Please find attached v4 taking into account 2c14037bb5. +} PgStat_WalCounters; + +typedef struct PgStat_WalStats +{ + PgStat_WalCounters wal_counters; I know that's a nit, but perhaps that could be a patch of its own, pushi

Re: per backend WAL statistics

2025-01-10 Thread Bertrand Drouvot
Hi, Michael Paquier wrote: > An extra thing I have finished by doing is removing > PgStat_BackendPendingIO, then applied the change. It was useful when > returned as a result of pgstat_prep_backend_pending(), but not so much > with the new PgStat_BackendPending that includes all the pending stats

Re: per backend WAL statistics

2025-01-09 Thread Michael Paquier
On Thu, Jan 09, 2025 at 07:05:54AM +, Bertrand Drouvot wrote: > PFA v3 which is v2 refactoring with your proposed above changes. An extra thing I have finished by doing is removing PgStat_BackendPendingIO, then applied the change. It was useful when returned as a result of pgstat_prep_backend

Re: per backend WAL statistics

2025-01-08 Thread Bertrand Drouvot
Hi, On Thu, Jan 09, 2025 at 01:03:15PM +0900, Michael Paquier wrote: > On Wed, Jan 08, 2025 at 11:11:59AM +, Bertrand Drouvot wrote: > > Yeah, that's more elegant as it also means that the main callback will not > > change > > (should we add even more stats in the future). Done that way in v2

Re: per backend WAL statistics

2025-01-08 Thread Michael Paquier
On Wed, Jan 08, 2025 at 11:11:59AM +, Bertrand Drouvot wrote: > Yeah, that's more elegant as it also means that the main callback will not > change > (should we add even more stats in the future). Done that way in v2 attached. I've put my hands on v2-0002 to begin with something. +/* flag bi

Re: per backend WAL statistics

2025-01-08 Thread Bertrand Drouvot
Hi, On Wed, Jan 08, 2025 at 03:21:26PM +0900, Michael Paquier wrote: > On Tue, Jan 07, 2025 at 08:48:51AM +, Bertrand Drouvot wrote: > > Now that commit 9aea73fc61 added backend-level statistics to pgstats (and > > per backend IO statistics), we can more easily add per backend statistics. > >

Re: per backend WAL statistics

2025-01-07 Thread Michael Paquier
On Tue, Jan 07, 2025 at 08:48:51AM +, Bertrand Drouvot wrote: > Now that commit 9aea73fc61 added backend-level statistics to pgstats (and > per backend IO statistics), we can more easily add per backend statistics. > > Please find attached a patch to implement $SUBJECT. I've looked at v1-0002