On Mon, Jan 20, 2025 at 01:26:55PM +, Bertrand Drouvot wrote:
> Right, that's another way to find it confusing ;-). But if we keep focus
> on where the pending stats are stored, it is not.
The name of the callbacks can be tuned infinitely. I have split the
renaming into its own patch, then th
Hi,
On Mon, Jan 20, 2025 at 09:11:16PM +0900, Michael Paquier wrote:
> On Mon, Jan 20, 2025 at 11:10:40AM +, Bertrand Drouvot wrote:
> > I think that it would be better to make the distinction based on
> > "local/static"
> > vs "dynamic memory" pending stats instead: I did so in v3 attached,
On Mon, Jan 20, 2025 at 11:10:40AM +, Bertrand Drouvot wrote:
> I think that it would be better to make the distinction based on
> "local/static"
> vs "dynamic memory" pending stats instead: I did so in v3 attached, using:
>
> .flush_dynamic_cb(): flushes pending entries tracked in dynamic me
Hi,
On Mon, Jan 20, 2025 at 03:34:41PM +0900, Michael Paquier wrote:
> On Sat, Jan 18, 2025 at 05:53:31PM +0900, Michael Paquier wrote:
> > Hmm. Such special complexities in pgstat.c are annoying. There is
> > a stupid thing I am wondering here. For the WAL stats, why couldn't
> > we place some
On Sat, Jan 18, 2025 at 05:53:31PM +0900, Michael Paquier wrote:
> Hmm. Such special complexities in pgstat.c are annoying. There is
> a stupid thing I am wondering here. For the WAL stats, why couldn't
> we place some calls of pgstat_prep_backend_pending() in strategic
> places like XLogBeginIn
On Fri, Jan 17, 2025 at 10:23:48AM +, Bertrand Drouvot wrote:
> Please find attached a patch implementing the ideas above, meaning:
>
> - It creates a new PendingBackendStats variable
> - It uses this variable to increment and flush per backend pending IO
> statistics
>
> That way we get rid
Hi,
On Fri, Jan 17, 2025 at 03:12:36PM +0900, Michael Paquier wrote:
> On Fri, Jan 17, 2025 at 06:06:35AM +, Bertrand Drouvot wrote:
> > On Fri, Jan 17, 2025 at 09:08:02AM +0900, Michael Paquier wrote:
> >> I could tweak that around the beginning of next week with a proposal
> >> of patch. Be
On Fri, Jan 17, 2025 at 06:06:35AM +, Bertrand Drouvot wrote:
> On Fri, Jan 17, 2025 at 09:08:02AM +0900, Michael Paquier wrote:
>> I could tweak that around the beginning of next week with a proposal
>> of patch. Bertrand, perhaps you'd prefer hack on this one?
>
> Yeah, I had in mind to wor
Hi,
On Fri, Jan 17, 2025 at 09:08:02AM +0900, Michael Paquier wrote:
> Anyway, let's just switch pgstat_backend.c so as we use a static
> PgStat_BackendPending (pending name) to store the pending IO stats
> that could be reused for also the WAL bits. It does not seem that
> complicated as far as
On Thu, Jan 16, 2025 at 11:28:43AM -0500, Andres Freund wrote:
> On 2025-01-15 18:27:22 +0900, Michael Paquier wrote:
>> My problem is that this is not only related to backend stats, but to
>> all variable-numbered stats kinds that require this behavior.
>
> The issue here imo is that recently IO
Hi,
On 2025-01-15 18:27:22 +0900, Michael Paquier wrote:
> On Wed, Jan 15, 2025 at 08:30:20AM +, Bertrand Drouvot wrote:
> > On Wed, Jan 15, 2025 at 11:03:54AM +0300, Nazir Bilal Yavuz wrote:
> >> With this commit it may not be possible to count IOs in the critical
> >> sections. I think the p
Hi,
On Thu, Jan 16, 2025 at 06:48:58AM +, Bertrand Drouvot wrote:
> Hi,
>
> On Thu, Jan 16, 2025 at 09:55:10AM +0900, Michael Paquier wrote:
> > On Wed, Jan 15, 2025 at 05:20:57PM +0300, Nazir Bilal Yavuz wrote:
> > > I think allowing only pgStatPendingContext to have
> > > MemoryContextAllow
Hi,
On Thu, Jan 16, 2025 at 09:55:10AM +0900, Michael Paquier wrote:
> On Wed, Jan 15, 2025 at 05:20:57PM +0300, Nazir Bilal Yavuz wrote:
> > I think allowing only pgStatPendingContext to have
> > MemoryContextAllowInCriticalSection() is not enough. We need to allow
> > at least pgStatSharedRefCon
On Wed, Jan 15, 2025 at 05:20:57PM +0300, Nazir Bilal Yavuz wrote:
> I think allowing only pgStatPendingContext to have
> MemoryContextAllowInCriticalSection() is not enough. We need to allow
> at least pgStatSharedRefContext as well to have
> MemoryContextAllowInCriticalSection() as it can be allo
Hi,
On Wed, Jan 15, 2025 at 05:20:57PM +0300, Nazir Bilal Yavuz wrote:
> While doing the initdb, we are restoring stats with the
> pgstat_restore_stats() and we do not expect any pending stats. The
> problem goes like that:
>
> I was a bit surprised that Bertrand did not encounter the same proble
Hi,
On Wed, 15 Jan 2025 at 12:27, Michael Paquier wrote:
>
> On Wed, Jan 15, 2025 at 08:30:20AM +, Bertrand Drouvot wrote:
> > On Wed, Jan 15, 2025 at 11:03:54AM +0300, Nazir Bilal Yavuz wrote:
> >> With this commit it may not be possible to count IOs in the critical
> >> sections. I think th
Hi,
On Wed, Jan 15, 2025 at 06:27:22PM +0900, Michael Paquier wrote:
> On Wed, Jan 15, 2025 at 08:30:20AM +, Bertrand Drouvot wrote:
> > On Wed, Jan 15, 2025 at 11:03:54AM +0300, Nazir Bilal Yavuz wrote:
> >> With this commit it may not be possible to count IOs in the critical
> >> sections. I
On Wed, Jan 15, 2025 at 08:30:20AM +, Bertrand Drouvot wrote:
> On Wed, Jan 15, 2025 at 11:03:54AM +0300, Nazir Bilal Yavuz wrote:
>> With this commit it may not be possible to count IOs in the critical
>> sections. I think the problem happens only if the local
>> PgStat_BackendPending entry is
Hi,
On Wed, Jan 15, 2025 at 11:03:54AM +0300, Nazir Bilal Yavuz wrote:
> Hi,
>
> On Thu, 19 Dec 2024 at 07:22, Michael Paquier wrote:
> >
> > Fixed that, bumped the two version counters, and done.
>
> I encountered a problem while trying to add WAL stats to pg_stat_io
> and I wanted to hear you
Hi,
On Thu, 19 Dec 2024 at 07:22, Michael Paquier wrote:
>
> Fixed that, bumped the two version counters, and done.
I encountered a problem while trying to add WAL stats to pg_stat_io
and I wanted to hear your thoughts.
Right now, pgstat_prep_backend_pending() is called in both
pgstat_count_io_
On Fri, Jan 10, 2025 at 01:56:01PM +, Bertrand Drouvot wrote:
> So now, I think that it is not a good idea for the per backend stats to
> behave differently than the other stats kinds. I think we should get rid of
> the "snapshot" for all of them or for none of them. Reading the above
> comme
Hi,
On Thu, Dec 19, 2024 at 06:12:04AM +, Bertrand Drouvot wrote:
> I think I'll start a dedicated thread to discuss the
> stats_fetch_consistency/'snapshot'
> point (will be easier to follow than resuming the discussion in this thread).
I gave more thoughts on it and did some research in th
On Tue, Jan 07, 2025 at 08:54:59AM +, Bertrand Drouvot wrote:
> A dedicated thread and a patch have been created, see [1].
>
> [1]:
> https://www.postgresql.org/message-id/flat/Z3zqc4o09dM/Ezyz%40ip-10-97-1-34.eu-west-3.compute.internal
Hmm. I can see that you have some refactoring pieces i
Hi,
On Fri, Jan 03, 2025 at 10:48:41AM +, Bertrand Drouvot wrote:
> I started to look at it and should be able to share a patch next week.
A dedicated thread and a patch have been created, see [1].
[1]:
https://www.postgresql.org/message-id/flat/Z3zqc4o09dM/Ezyz%40ip-10-97-1-34.eu-west-3.co
Hi,
On Tue, Dec 24, 2024 at 02:35:16PM +0900, Michael Paquier wrote:
> On Fri, Dec 20, 2024 at 09:57:19AM +, Bertrand Drouvot wrote:
> > BTW, now that the per backend I/O statistics is done, I'll start working on
> > per
> > backend wal statistics.
>
>
On Fri, Dec 20, 2024 at 09:57:19AM +, Bertrand Drouvot wrote:
> BTW, now that the per backend I/O statistics is done, I'll start working on
> per
> backend wal statistics.
I think that this is a good idea. It does not actually overlap the
proposal in [1] as the stats persiste
formation about other statistics related to
> >backends, depending on requirements or ideas.
BTW, now that the per backend I/O statistics is done, I'll start working on per
backend wal statistics.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Data
On Fri, Dec 20, 2024 at 09:09:00AM +, Bertrand Drouvot wrote:
> Thanks for the report! I was not able able to reproduce (even with
> asan-enabled)
> but I think the test is wrong. Indeed the backend could fsync too during the
> test
> (see register_dirty_segment() and the case where the reque
Hi,
On Fri, Dec 20, 2024 at 08:00:00AM +0200, Alexander Lakhin wrote:
> Hello Michael,
>
> 19.12.2024 06:21, Michael Paquier wrote:
> > Fixed that, bumped the two version counters, and done.
>
> Could you, please, look at recent failures produced by grassquit (which
> has fsync = on in it's conf
Hello Michael,
19.12.2024 06:21, Michael Paquier wrote:
Fixed that, bumped the two version counters, and done.
Could you, please, look at recent failures produced by grassquit (which
has fsync = on in it's config), on an added test case? For instance, [1]:
--- /home/bf/bf-build/grassquit/HEAD/
Hi,
On Thu, Dec 19, 2024 at 01:21:54PM +0900, Michael Paquier wrote:
>
> While doing more tests with backends exiting concurrently with their
> stats scanned, I have detected one path in pg_stat_get_backend_io()
> after calling pgstat_get_beentry_by_proc_number() where we should also
> check that
On Wed, Dec 18, 2024 at 08:11:55AM +, Bertrand Drouvot wrote:
> I think I prefer pg_stat_io_build_tuples() and used that name in v11
> attached.
I'll rely to your naming sense than mine, then :D
> I think that this comment is now confusing since the extra loop would be
> done in pg_stat_io_bu
On Wed, Dec 18, 2024 at 08:11:55AM +, Bertrand Drouvot wrote:
> Yeah, I also had something like this in mind (see R2 in [1]). So, +1 for it.
[1] being:
https://www.postgresql.org/message-id/Zy4bmvgHqGjcK1pI%40ip-10-97-1-34.eu-west-3.compute.internal
Regards,
--
Bertrand Drouvot
PostgreSQL
peIO */
+ pg_stat_io_build_tuples(rsinfo, bktype_stats, bktype,
+backends_io_stats->stat_reset_timestamp);
}
return (Datum) 0;
--
2.34.1
>From 6447dbe922e1db379dfd65f8041bc55cf3a3c573 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot
Date: Wed, 18 Dec 2024 06:39:02 +0000
Subject: [PATCH v11
Michael
From 75a2549543edf418b6e282acab3f8ce272c5c42a Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot
Date: Mon, 28 Oct 2024 12:50:32 +
Subject: [PATCH v10 1/2] per backend I/O statistics
While pg_stat_io provides cluster-wide I/O statistics, this commit adds the
ability to track and display pe
Hi,
On Tue, Dec 17, 2024 at 06:13:44PM +0900, Michael Paquier wrote:
> On Tue, Dec 17, 2024 at 08:13:59AM +, Bertrand Drouvot wrote:
> > Having said that, v9-0002 looks good to me (except the
> > pgstat_fetch_proc_stat_io
> > renaming).
>
> This routine returns a PgStat_Backend from a pgstat
On Tue, Dec 17, 2024 at 08:13:59AM +, Bertrand Drouvot wrote:
> Having said that, v9-0002 looks good to me (except the
> pgstat_fetch_proc_stat_io
> renaming).
This routine returns a PgStat_Backend from a pgstats entry of type
PGSTAT_KIND_BACKEND, so the name in v9-0001 would not be true once
Hi,
On Tue, Dec 17, 2024 at 03:26:02PM +0900, Michael Paquier wrote:
> On Mon, Dec 16, 2024 at 03:42:04PM +, Bertrand Drouvot wrote:
> >> Perhaps there's an argument for an entirely separate
> >> callback that would run before pgstat is plugged off, like a new
> >> before_shmem_exit() callback
0b7 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot
Date: Mon, 28 Oct 2024 12:50:32 +
Subject: [PATCH v9 1/2] per backend I/O statistics
While pg_stat_io provides cluster-wide I/O statistics, this commit adds the
ability to track and display per backend I/O statistics.
It adds a new statistics ki
the way we currently drop the backend stats entry is fine (unless
I miss something about your concern).
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
>From 5bd09f37d6fdb3db38eff8ce30677d3989fcf64c Mon Sep 17 0
On Fri, Dec 13, 2024 at 09:20:13AM +, Bertrand Drouvot wrote:
> Yeah makes sense, that's consistent with other kinds: done.
This looks to be taking shape. I don't have much more comments.
Not feeling so sure about the value brought by the backend_type
returned in pg_stat_get_backend_io(), bu
_tracks_per_backend_bktype. These could
> be removed, I guess, doing also a PGSTAT_KIND_PER_BACKEND =>
> PGSTAT_KIND_BACKEND?
Yeah makes sense, that's consistent with other kinds: done.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web
On Thu, Dec 12, 2024 at 02:02:38PM +, Bertrand Drouvot wrote:
> On Thu, Dec 12, 2024 at 01:52:03PM +0900, Michael Paquier wrote:
> Yeah, but it's needed in pg_stat_get_backend_io() for the stats filtering (to
> display only those linked to this backend type), later in the function here:
>
> +
w to use it
> with pg_stat_activity. An example with LATERAL, doing the same work:
> select a.pid, s.* from pg_stat_activity as a,
>lateral pg_stat_get_backend_io(a.pid) as s
> where pid = pg_backend_pid();
I'm not sure it's worth it. I think that's clear that to
On Mon, Nov 25, 2024 at 03:47:59PM +, Bertrand Drouvot wrote:
> === Remarks
>
> R1: as compared to v5, v6 removes the per-backend I/O stats reset from
> pg_stat_reset_shared(). I think it makes more sense that way, since we are
> adding pg_stat_reset_single_backend_io_counters(). The per-back
On Wed, Nov 27, 2024 at 04:00:14PM +, Bertrand Drouvot wrote:
> I'd vote to just keep the pgstat_create_backend_stat() call in
> pgstat_beinit().
>
> That said, we probably should document that init_backend_cb() should not call
> pgstat_get_entry_ref(), but that's probably worth another threa
Hi,
On Wed, Nov 27, 2024 at 03:33:38PM +0900, Michael Paquier wrote:
> On Mon, Nov 25, 2024 at 03:47:59PM +, Bertrand Drouvot wrote:
> > It also takes care of most of the comments that you have made in [1],
> > meaning
> > that it:
> >
> > - removes the backend type from PgStat_Backend and l
On Mon, Nov 25, 2024 at 03:47:59PM +, Bertrand Drouvot wrote:
> It also takes care of most of the comments that you have made in [1], meaning
> that it:
>
> - removes the backend type from PgStat_Backend and look for the backend type
> at "display" time.
> - creates PgStat_BackendPendingIO and
abases
Amazon Web Services: https://aws.amazon.com
>From 21d773ae2b64571b09a77d6de4c955338cf979ae Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot
Date: Mon, 28 Oct 2024 12:50:32 +
Subject: [PATCH v6] per backend I/O statistics
While pg_stat_io provides cluster-wide I/O statistics, this comm
On Mon, Nov 25, 2024 at 07:12:56AM +, Bertrand Drouvot wrote:
> Not sure here, could custom stats start incrementing before the database
> system
> is ready to accept connections?
In theory, that could be possible. Like pg_stat_io currently, I am
ready to assume that it likely won't matter m
Hi,
On Mon, Nov 25, 2024 at 10:06:44AM +0900, Michael Paquier wrote:
> On Fri, Nov 22, 2024 at 07:49:58AM +, Bertrand Drouvot wrote:
> > On Fri, Nov 22, 2024 at 10:36:29AM +0900, Michael Paquier wrote:
> >> Hmm. created_entry only matters for pgstat_init_function_usage().
> >> All the other c
On Fri, Nov 22, 2024 at 07:49:58AM +, Bertrand Drouvot wrote:
> On Fri, Nov 22, 2024 at 10:36:29AM +0900, Michael Paquier wrote:
>> Hmm. created_entry only matters for pgstat_init_function_usage().
>> All the other callers of pgstat_prep_pending_entry() pass a NULL
>> value.
>
> I meant to s
Hi,
On Fri, Nov 22, 2024 at 10:36:29AM +0900, Michael Paquier wrote:
> On Thu, Nov 21, 2024 at 05:23:42PM +, Bertrand Drouvot wrote:
> > So, given that:
> >
> > - the end result would be the same
> > - the code changes would be non negligible (unless we have a better idea
> > than
> > pgstat
On Thu, Nov 21, 2024 at 05:23:42PM +, Bertrand Drouvot wrote:
> So, given that:
>
> - the end result would be the same
> - the code changes would be non negligible (unless we have a better idea than
> pgstat_get_entry_ref() returning a NULL value).
Hmm. created_entry only matters for pgstat_
Hi,
On Thu, Nov 21, 2024 at 07:18:24AM +0900, Michael Paquier wrote:
> On Wed, Nov 20, 2024 at 02:20:18PM +, Bertrand Drouvot wrote:
> > Right. I did not had in mind to go that far here (for the per backend stats
> > needs). My idea was "just" to move the new pgstat_create_backend_stat()
> >
Hi,
On Tue, Nov 19, 2024 at 10:47:53AM +0900, Michael Paquier wrote:
> On Thu, Nov 14, 2024 at 01:30:11PM +, Bertrand Drouvot wrote:
> > - change the arguments to false in the pgstat_drop_entry_internal() call in
> > pgstat_drop_all_entries()
> > - start the engine
> > - kill -9 postgres
> >
On Wed, Nov 20, 2024 at 02:10:23PM +, Bertrand Drouvot wrote:
> Yeah, also this could useful for custom statistics. So I created a dedicated
> thread and a patch proposal (see [1]).
>
> [1]:
> https://www.postgresql.org/message-id/Zz3skBqzBncSFIuY%40ip-10-97-1-34.eu-west-3.compute.internal
T
On Wed, Nov 20, 2024 at 02:20:18PM +, Bertrand Drouvot wrote:
> Right. I did not had in mind to go that far here (for the per backend stats
> needs). My idea was "just" to move the new pgstat_create_backend_stat() (which
> is related to per backend stats only) call at the right place in Startup
Hi,
On Wed, Nov 20, 2024 at 09:01:26AM +0900, Michael Paquier wrote:
> On Tue, Nov 19, 2024 at 04:28:55PM +, Bertrand Drouvot wrote:
> > So, for the startup process only, what about?
> >
> > - don't call pgstat_create_backend_stat() in pgstat_beinit()...
> > - but call it in StartupXLOG() ins
Hi,
On Thu, Nov 14, 2024 at 03:31:51PM +0900, Michael Paquier wrote:
> + /*
> + * Do serialize or not this kind of stats.
> + */
> + boolto_serialize:1;
>
> Not sure that "serialize" is the best term that applies here. For
> pgstats entries, serialization refers to
On Tue, Nov 19, 2024 at 04:28:55PM +, Bertrand Drouvot wrote:
> So, for the startup process only, what about?
>
> - don't call pgstat_create_backend_stat() in pgstat_beinit()...
> - but call it in StartupXLOG() instead (after the stats are discarded or
> restored).
>
> That way we could get
On Thu, Nov 14, 2024 at 01:30:11PM +, Bertrand Drouvot wrote:
> - change the arguments to false in the pgstat_drop_entry_internal() call in
> pgstat_drop_all_entries()
> - start the engine
> - kill -9 postgres
> - restart the engine
>
> You'll see the assert failing due to the startup process
Hi,
On Thu, Nov 14, 2024 at 03:31:51PM +0900, Michael Paquier wrote:
> On Fri, Nov 08, 2024 at 02:09:30PM +, Bertrand Drouvot wrote:
> > 1. one assertion in pgstat_drop_entry_internal() is not necessary true
> > anymore
> > with this new stat kind. So, adding an extra bool as parameter to ta
On Fri, Nov 08, 2024 at 02:09:30PM +, Bertrand Drouvot wrote:
> 0001 (the largest one)
>
> Introduces a new statistics KIND. It is named PGSTAT_KIND_PER_BACKEND as it
> could
> be used in the future to store other statistics (than the I/O ones) per
> backend.
> The new KIND is a variabl
ther backend's stats in the snapshot). I choose to document it and to
not return any rows: I think that's better than returning some data (that would
not "ensure" the snapshot consistency) or an error.
Looking forward to your feedback,
Regards,
--
Bertrand Drouvot
Postgre
Hi,
On Thu, Nov 07, 2024 at 09:50:59AM +0900, Michael Paquier wrote:
> On Wed, Nov 06, 2024 at 01:51:02PM +, Bertrand Drouvot wrote:
> > That's not needed, the patch I'm working on stores the proc number in the
> > objid field of the key.
>
> Relying on the procnumber for the object ID gets a
On Wed, Nov 06, 2024 at 01:51:02PM +, Bertrand Drouvot wrote:
> That's not needed, the patch I'm working on stores the proc number in the
> objid field of the key.
Relying on the procnumber for the object ID gets a +1 here. That
provides an automatic cap on the maximum number of entries that
Hi,
On Wed, Nov 06, 2024 at 08:39:07AM +0900, Michael Paquier wrote:
> On Tue, Nov 05, 2024 at 05:37:15PM +, Bertrand Drouvot wrote:
> > I'm starting working on option 2, I think it will be easier to discuss with
> > a patch proposal to look at.
> >
> > If in the meantime, one strongly disagr
On Tue, Nov 05, 2024 at 05:37:15PM +, Bertrand Drouvot wrote:
> I'm starting working on option 2, I think it will be easier to discuss with
> a patch proposal to look at.
>
> If in the meantime, one strongly disagree with option 2 (means implement a
> brand
> new PGSTAT_KIND_BACKEND and keep
Hi,
On Mon, Nov 04, 2024 at 10:01:50AM +, Bertrand Drouvot wrote:
> And why not add more per-backend stats in the future? (once the I/O part is
> done).
>
> I think that's one more reason to go with option 2 (and implementing a brand
> new
> PGSTAT_KIND_BACKEND kind).
I'm starting working
Hi,
On Thu, Oct 31, 2024 at 05:09:56AM +, Bertrand Drouvot wrote:
> === OPTIONS ===
>
> So, based on this, I think that we could:
>
> Option 1: "move" the existing PGSTAT_KIND_IO to variable-numbered and let this
> KIND take care of the aggregated view (pg_stat_io) and the per-backend stats.
Hi,
On Tue, Oct 08, 2024 at 04:28:39PM +, Bertrand Drouvot wrote:
> > > On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote:
> >
> > Okay, per the above and the persistency of the stats.
>
> Great, I'll work on an updated patch version then.
>
I spend some time on this during t
Hi,
On Tue, Oct 08, 2024 at 01:46:23PM +0900, Michael Paquier wrote:
> On Mon, Oct 07, 2024 at 09:54:21AM +, Bertrand Drouvot wrote:
> > On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote:
> >> This would rely on the fact that we would use the ProcNumber for the
> >> dshash key, a
On Mon, Oct 07, 2024 at 09:54:21AM +, Bertrand Drouvot wrote:
> On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote:
>> This would rely on the fact that we would use the ProcNumber for the
>> dshash key, and this information is not provided in pg_stat_activity.
>> Perhaps we should
Hi,
On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote:
> On Tue, Sep 17, 2024 at 01:56:34PM +, Bertrand Drouvot wrote:
> > No problem at all! (I re-explained because I'm not always 100% sure that my
> > explanations are crystal clear ;-) )
>
> We've discussed a bit this patch of
Hi,
On Fri, Sep 20, 2024 at 12:53:43PM +0900, Michael Paquier wrote:
> On Wed, Sep 04, 2024 at 04:45:24AM +, Bertrand Drouvot wrote:
> > On Tue, Sep 03, 2024 at 04:07:58PM +0900, Kyotaro Horiguchi wrote:
> >> As an additional benefit of this approach, the client can set a
> >> connection varia
On Tue, Sep 17, 2024 at 01:56:34PM +, Bertrand Drouvot wrote:
> No problem at all! (I re-explained because I'm not always 100% sure that my
> explanations are crystal clear ;-) )
We've discussed a bit this patch offline, but after studying the patch
I doubt that this part is a good idea:
+
On Wed, Sep 04, 2024 at 04:45:24AM +, Bertrand Drouvot wrote:
> On Tue, Sep 03, 2024 at 04:07:58PM +0900, Kyotaro Horiguchi wrote:
>> As an additional benefit of this approach, the client can set a
>> connection variable, for example, no_backend_iostats to true, or set
>> its inverse variable t
Hi,
On Tue, Sep 17, 2024 at 04:47:51PM +0300, Nazir Bilal Yavuz wrote:
> Hi,
>
> On Tue, 17 Sept 2024 at 16:07, Bertrand Drouvot
> wrote:
> > So I think it's better to keep both functions as they behave differently.
> >
> > Thoughts?
>
> Yes, that is correct. Sorry, you already had explained it
Hi,
On Tue, 17 Sept 2024 at 16:07, Bertrand Drouvot
wrote:
> On Tue, Sep 17, 2024 at 02:52:01PM +0300, Nazir Bilal Yavuz wrote:
> > Could we remove pg_stat_get_my_io() completely and use
> > pg_stat_get_backend_io() with the current backend's pid to get the
> > current backend's stats?
>
> The re
Hi,
On Tue, Sep 17, 2024 at 02:52:01PM +0300, Nazir Bilal Yavuz wrote:
> Hi,
>
> On Fri, 13 Sept 2024 at 19:09, Bertrand Drouvot
> wrote:
> > On Fri, Sep 13, 2024 at 04:45:08PM +0300, Nazir Bilal Yavuz wrote:
> > > - The pgstat_reset_io_counter_internal() is called in the
> > > pgstat_shutdown_h
Hi,
On Fri, 13 Sept 2024 at 19:09, Bertrand Drouvot
wrote:
> On Fri, Sep 13, 2024 at 04:45:08PM +0300, Nazir Bilal Yavuz wrote:
> > - The pgstat_reset_io_counter_internal() is called in the
> > pgstat_shutdown_hook(). This causes the stats_reset column showing the
> > termination time of the old
just to be
right "conceptually" speaking).
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
>From 2bc490060ce8ed6557ca5a586f36cd43e7480d8e Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot
Date: Thu, 22
Hi,
Thanks for working on this!
Your patch applies and builds cleanly.
On Fri, 6 Sept 2024 at 18:03, Bertrand Drouvot
wrote:
>
> - As stated up-thread, the pg_stat_get_backend_io() behaves as if
> stats_fetch_consistency is set to none (each execution re-fetches counters
> from shared memory).
Hi,
On Fri, Sep 06, 2024 at 03:03:17PM +, Bertrand Drouvot wrote:
> The struct size is 1040 Bytes and that's much more reasonable than the size
> needed for per backend I/O stats in v1 (about 16KB).
One could think that's a high increase of shared memory usage with a high
number of connection
e: Thu, 22 Aug 2024 15:16:50 +
Subject: [PATCH v3 1/2] per backend I/O statistics
While pg_stat_io provides cluster-wide I/O statistics, this commit adds a new
pg_my_stat_io view to display "my" backend I/O statistics. The KIND_IO stats
are still "fixed amount" ones as th
pg_stat_get_my_io() and pg_stat_get_io() to avoid
duplicated code (it's not done yet to ease the review).
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
>From edc6da2601350edf8647b309ada4ecec4a86dd64 Mon Sep 17 00:0
Hi,
On Thu, Sep 05, 2024 at 03:03:32PM +0200, Alvaro Herrera wrote:
> On 2024-Sep-03, Bertrand Drouvot wrote:
>
> > Cons is probably allocating shared memory space that might not be used (
> > sizeof(PgStat_IO) is 16392 so that could be a concern for a high number of
> > unused connection). OTOH,
On 2024-Sep-03, Bertrand Drouvot wrote:
> Cons is probably allocating shared memory space that might not be used (
> sizeof(PgStat_IO) is 16392 so that could be a concern for a high number of
> unused connection). OTOH, if a high number of connections is not used that
> might
> be worth to reduce
Hi,
On Tue, Sep 03, 2024 at 04:07:58PM +0900, Kyotaro Horiguchi wrote:
> At Tue, 03 Sep 2024 15:37:49 +0900 (JST), Kyotaro Horiguchi
> wrote in
> > When I first looked at this patch, my initial thought was whether we
> > should let these stats stay "fixed." The reason why the current
> > PGSTAT
Hi,
On Tue, Sep 03, 2024 at 03:37:49PM +0900, Kyotaro Horiguchi wrote:
> At Mon, 2 Sep 2024 14:55:52 +, Bertrand Drouvot
> wrote in
> > Hi hackers,
> >
> > Please find attached a patch to implement $SUBJECT.
> >
> > While pg_stat_io provides cluster-wide I/O statistics, this patch adds a
At Tue, 03 Sep 2024 15:37:49 +0900 (JST), Kyotaro Horiguchi
wrote in
> When I first looked at this patch, my initial thought was whether we
> should let these stats stay "fixed." The reason why the current
> PGSTAT_KIND_IO is fixed is that there is only one global statistics
> storage for the en
_my_stat_io view and 0002 to add the pg_stat_get_backend_io() function.
> - the idea of having per backend I/O statistics has already been mentioned in
> [1] by Andres.
>
> Some implementation choices:
>
> - The KIND_IO stats are still "fixed amount" ones as the maximum numb
... in the current session.
Some remarks:
- it is split in 2 sub patches: 0001 introducing the necessary changes to
provide
the pg_my_stat_io view and 0002 to add the pg_stat_get_backend_io() function.
- the idea of having per backend I/O statistics has already been mentioned in
[1] by And
94 matches
Mail list logo
