Hi.
Em seg., 3 de fev. de 2025 às 01:07, Michael Paquier
escreveu:
> On Fri, Jan 31, 2025 at 11:29:31AM +0300, Nazir Bilal Yavuz wrote:
> > On Wed, 29 Jan 2025 at 18:16, Bertrand Drouvot
> > wrote:
> >> I think that's the main reason why ff99918c625 added this new GUC
> (looking at
> >> the com
Em ter., 18 de mar. de 2025 às 20:34, Michael Paquier
escreveu:
> On Tue, Mar 18, 2025 at 04:13:15PM +0300, Nazir Bilal Yavuz wrote:
> > Thanks for the report! Yes, it is an oversight on my part.
>
> No worries. It's most likely me while reviewing the whole as I've
> reordered these blocks a bit
On Tue, Mar 18, 2025 at 04:13:15PM +0300, Nazir Bilal Yavuz wrote:
> Thanks for the report! Yes, it is an oversight on my part.
No worries. It's most likely me while reviewing the whole as I've
reordered these blocks a bit while going through the patch.
This has little consequences for the end u
Hi,
On Tue, 18 Mar 2025 at 15:47, Ranier Vilela wrote:
>
> Sorry, I couldn't find the email in this thread, linked to the commit:
> a051e71
>
> I think it left an oversight.
> Copy and past thinko?
Thanks for the report! Yes, it is an oversight on my part.
> Attached a trivial patch.
LGTM.
--
On Thu, Feb 06, 2025 at 03:39:05PM +0300, Nazir Bilal Yavuz wrote:
> My vote goes to 029. It already has the '## check checkpoint and wal
> stats are incremented due to restart' part and what we are adding is
> similar to that.
Okay, sold, with a routine grabbing some stats from pg_stat_io based
o
Hi,
On Thu, 6 Feb 2025 at 11:39, Michael Paquier wrote:
>
> On Thu, Feb 06, 2025 at 07:32:01AM +, Bertrand Drouvot wrote:
> > That sounds ok, but I wonder if that's the best appropriate place. I could
> > think of the checkpointer test in 029_stats_restart.pl and the
> > startup/standby
> >
Hi,
On Wed, 5 Feb 2025 at 08:30, Michael Paquier wrote:
>
> Please note that I have marked the CF entry at [1] as committed as the
> main goal of this thread has been achieved. Do you want to pursue the
> topic about the potential refactorings in pgstat_tracks_io_op() and
> pgstat_tracks_io_obje
On Thu, Feb 06, 2025 at 07:32:01AM +, Bertrand Drouvot wrote:
> That sounds ok, but I wonder if that's the best appropriate place. I could
> think of the checkpointer test in 029_stats_restart.pl and the startup/standby
> one in one related to standby (030_stats_cleanup_replica.pl?). Though tha
Hi,
On Thu, Feb 06, 2025 at 12:35:28PM +0900, Michael Paquier wrote:
> On Wed, Feb 05, 2025 at 09:52:14PM -0500, Tom Lane wrote:
> > Michael Paquier writes:
> > Yeah, if we want to assume we can see stats counts left over from
> > initdb, we have to put this in a TAP test, though I dunno if that
On Wed, Feb 05, 2025 at 09:52:14PM -0500, Tom Lane wrote:
> Michael Paquier writes:
> Yeah, if we want to assume we can see stats counts left over from
> initdb, we have to put this in a TAP test, though I dunno if that is
> the most appropriate one.
A second option I can think of for the reads i
Michael Paquier writes:
> The queries for the normal context are not going to have this problem
> even if we have a pg_stat_reset_shared('io'), but the init context
> gets unstable, unfortunately. I don't see a way through here in the
> main regression test suite, so how about moving these into
>
On Wed, Feb 05, 2025 at 09:06:24PM -0500, Tom Lane wrote:
> Basically, this test is broken because it assumes that the server
> is freshly started. That's not a scenario the buildfarm runs
> the core tests under, unfortunately, but it is one commonly
> used in hand testing.
Right. You are pointi
Nazir Bilal Yavuz writes:
> On Wed, 5 Feb 2025 at 21:32, Tom Lane wrote:
>> Locally, the test added by this commit fails like so:
> My thoughts when adding this test was that startup process must do the
> WAL read I/O while server is starting, i.e.:
Oh! The problem is much simpler than I thoug
Hi,
On Wed, 5 Feb 2025 at 21:32, Tom Lane wrote:
>
> Michael Paquier writes:
> > At the end, we want this patch and this data, and my benchmarcking is
> > not showing much differences even if going through a workload with
> > many pages, so I've used the version relying entirely on
> > track_io_
Michael Paquier writes:
> At the end, we want this patch and this data, and my benchmarcking is
> not showing much differences even if going through a workload with
> many pages, so I've used the version relying entirely on
> track_io_timing and applied it.
Locally, the test added by this commit
On Tue, Feb 04, 2025 at 05:40:06PM +0300, Nazir Bilal Yavuz wrote:
> Hi,
>
> On Tue, 4 Feb 2025 at 17:29, Nazir Bilal Yavuz wrote:
> >
> > Thanks! My benchmark results are the same [1]. I could not see the
> > noticable difference between master and patched version. I run the
> > benchmark a coup
Hi,
On Tue, 4 Feb 2025 at 17:29, Nazir Bilal Yavuz wrote:
>
> Thanks! My benchmark results are the same [1]. I could not see the
> noticable difference between master and patched version. I run the
> benchmark a couple of times, sometimes the patched version, sometimes the
> master version is bet
Hi,
On Tue, 4 Feb 2025 at 10:55, Michael Paquier wrote:
>
> On Mon, Feb 03, 2025 at 02:34:29PM +0300, Nazir Bilal Yavuz wrote:
> > On Mon, 3 Feb 2025 at 11:50, Bertrand Drouvot <
bertranddrouvot...@gmail.com> wrote:
> >> === 1
> >>
> >> + pgstat_count_io_op_time(IOOBJECT_WAL, IOCONTEXT_INIT
On Mon, Feb 03, 2025 at 02:34:29PM +0300, Nazir Bilal Yavuz wrote:
> On Mon, 3 Feb 2025 at 11:50, Bertrand Drouvot
> wrote:
>> === 1
>>
>> + pgstat_count_io_op_time(IOOBJECT_WAL, IOCONTEXT_INIT, IOOP_WRITE,
>> + io_start, 1,
>> wal_segm
On Mon, Feb 03, 2025 at 10:37:54AM +, Bertrand Drouvot wrote:
> Based on those results the patch does not show a noticable impact when IO
> timing
> tracking is/are enabled.
I have spent a good portion of my day doing benchmarking with your
scenarios as much as Nazir's scenarios posted at [1]
Hi,
On Mon, 3 Feb 2025 at 11:50, Bertrand Drouvot
wrote:
>
> Hi,
>
> On Mon, Feb 03, 2025 at 01:07:26PM +0900, Michael Paquier wrote:
> > On Fri, Jan 31, 2025 at 11:29:31AM +0300, Nazir Bilal Yavuz wrote:
> > > On Wed, 29 Jan 2025 at 18:16, Bertrand Drouvot
> > > wrote:
> > >> I think that's the
On Mon, Feb 03, 2025 at 08:50:15AM +, Bertrand Drouvot wrote:
> === 6
>
> I'll also do some benchmark on my side.
So, I did some tests using:
c=1 && psql -c checkpoint -c 'select pg_switch_wal()' &&
pgbench -n -M prepared -c$c -j$c -f <(echo "SELECT
pg_logical_emit_message(true, 'test', rep
Hi,
On Mon, Feb 03, 2025 at 01:07:26PM +0900, Michael Paquier wrote:
> On Fri, Jan 31, 2025 at 11:29:31AM +0300, Nazir Bilal Yavuz wrote:
> > On Wed, 29 Jan 2025 at 18:16, Bertrand Drouvot
> > wrote:
> >> I think that's the main reason why ff99918c625 added this new GUC (looking
> >> at
> >> the
On Fri, Jan 31, 2025 at 11:29:31AM +0300, Nazir Bilal Yavuz wrote:
> On Wed, 29 Jan 2025 at 18:16, Bertrand Drouvot
> wrote:
>> I think that's the main reason why ff99918c625 added this new GUC (looking at
>> the commit message). I'd feel more comfortable if we keep it.
>
> As Michael suggested,
Hi,
On Wed, 29 Jan 2025 at 18:16, Bertrand Drouvot
wrote:
>
> Hi,
>
> On Wed, Jan 29, 2025 at 02:57:21PM +0300, Nazir Bilal Yavuz wrote:
> > I agree with you but it was discussed before in this thread [2]. It
> > was decided to use both track_wal_io_timing and track_io_timing
> > because of the o
On Wed, Jan 29, 2025 at 02:57:21PM +0300, Nazir Bilal Yavuz wrote:
> On Tue, 28 Jan 2025 at 07:23, Michael Paquier wrote:
>> What you doing in 0001 is a first good step towards this goal, as this
>> also plugs in a few things for backend statistics with the calls to
>> pgstat_count_io_op[_time]().
Hi,
On Wed, Jan 29, 2025 at 02:57:21PM +0300, Nazir Bilal Yavuz wrote:
> I agree with you but it was discussed before in this thread [2]. It
> was decided to use both track_wal_io_timing and track_io_timing
> because of the overhead that track_wal_io_timing creates but we can
> still re-discuss it
Hi,
On Wed, Jan 29, 2025 at 01:49:20PM +0300, Nazir Bilal Yavuz wrote:
> Sorry for replying late.
No problem at all!
> It turns out that the difference is expected.
> It is because pg_stat_wal.wal_bytes tracks the total amount of WAL
> generated, while pg_stat_io tracks the I/Os that are written
Hi,
On Tue, 28 Jan 2025 at 07:23, Michael Paquier wrote:
>
> On Mon, Jan 27, 2025 at 05:13:39PM +0300, Nazir Bilal Yavuz wrote:
> > On Mon, 27 Jan 2025 at 16:59, Nazir Bilal Yavuz wrote:
> >>
> >> On Mon, 27 Jan 2025 at 03:52, Michael Paquier wrote:
> >> We use PendingWalStats.wal_[ write | syn
Hi,
On Mon, 27 Jan 2025 at 10:45, Bertrand Drouvot
wrote:
>
> Hi,
>
> On Fri, Jan 24, 2025 at 06:29:46PM +0300, Nazir Bilal Yavuz wrote:
> > Hi,
> >
> > Thanks for looking into this!
> >
> > On Fri, 24 Jan 2025 at 17:20, Bertrand Drouvot
> > wrote:
> > >
> > > I did not look at the code yet but
On Mon, Jan 27, 2025 at 05:13:39PM +0300, Nazir Bilal Yavuz wrote:
> On Mon, 27 Jan 2025 at 16:59, Nazir Bilal Yavuz wrote:
>>
>> On Mon, 27 Jan 2025 at 03:52, Michael Paquier wrote:
>> We use PendingWalStats.wal_[ write | sync ]_time only to show timings
>> on the pg_stat_wal view, right? And no
Hi,
On Mon, 27 Jan 2025 at 16:59, Nazir Bilal Yavuz wrote:
>
> On Mon, 27 Jan 2025 at 03:52, Michael Paquier wrote:
> >
> > In 0002, you are removing PendingWalStats.wal_write_time, which does
> > not seem OK to me because we should still aggregate this data for
> > track_wal_io_timing, no?
>
>
Hi,
On Mon, 27 Jan 2025 at 03:52, Michael Paquier wrote:
>
> On Fri, Jan 24, 2025 at 06:29:46PM +0300, Nazir Bilal Yavuz wrote:
> > By saying that, this does not solve the problem; there is still a
> > difference although you omit WAL initialization stats from the
> > pg_stat_io.
>
> Reattaching
Hi,
On Fri, Jan 24, 2025 at 06:29:46PM +0300, Nazir Bilal Yavuz wrote:
> Hi,
>
> Thanks for looking into this!
>
> On Fri, 24 Jan 2025 at 17:20, Bertrand Drouvot
> wrote:
> >
> > I did not look at the code yet but did a few tests.
> > I can see diff between pg_stat_wal and pg_stat_io, for examp
On Fri, Jan 24, 2025 at 06:29:46PM +0300, Nazir Bilal Yavuz wrote:
> By saying that, this does not solve the problem; there is still a
> difference although you omit WAL initialization stats from the
> pg_stat_io.
Reattaching the remaining patches to make the cfbot happy, as 0001 has
been applied
Hi,
Thanks for looking into this!
On Fri, 24 Jan 2025 at 17:20, Bertrand Drouvot
wrote:
>
> I did not look at the code yet but did a few tests.
> I can see diff between pg_stat_wal and pg_stat_io, for example:
>
> "
> postgres=# select pg_stat_reset_shared();
> pg_stat_reset_shared
> --
Hi,
On Fri, Jan 24, 2025 at 11:31:02AM +0300, Nazir Bilal Yavuz wrote:
> v9 is rebased and attached as three patches. The first one is a
> squashed patch for the current version of Andres' proposed fix to pass
> the CI, the second one is for adding WAL stats to pg_stat_io and the
> third one is fo
Hi,
On Wed, 22 Jan 2025 at 03:14, Michael Paquier wrote:
>
> On Thu, Jan 16, 2025 at 11:40:51AM +0300, Nazir Bilal Yavuz wrote:
> > I encountered another problem while rebasing the patch. The problem is
> > basically we do not expect any pending stats while restoring the stats
> > at the initdb.
On Thu, Jan 16, 2025 at 11:40:51AM +0300, Nazir Bilal Yavuz wrote:
> I encountered another problem while rebasing the patch. The problem is
> basically we do not expect any pending stats while restoring the stats
> at the initdb. However, WAL IOs (WAL read and WAL init IOs for now)
> may happen bef
Hi,
On Fri, 19 Apr 2024 at 11:01, Nazir Bilal Yavuz wrote:
>
> If I am not missing any new changes, the only problem is reading
> variable bytes now. We have discussed a couple of solutions:
With the recent commit [1], pg_stat_io tracks IOs as bytes instead of
blocks. This solves the variable IO
> Perhaps Nitin was thinking of a scenario in which WAL hits are counted
> as hits on the same IOObject as shared buffer hits. Since this thread
> has been going on for awhile and we haven't recently had a schema
> overview, I could understand if there was some confusion
Yes. I was considering a s
On Thu, Jun 13, 2024 at 5:24 AM Nazir Bilal Yavuz wrote:
>
> On Sun, 9 Jun 2024 at 18:05, Nitin Jadhav
> wrote:
> >
> > > If possible, let's have all the I/O stats (even for WAL) in
> > > pg_stat_io. Can't we show the WAL data we get from buffers in the hits
> > > column and then have read_bytes
Hi,
Thank you for looking into this!
On Sun, 9 Jun 2024 at 18:05, Nitin Jadhav wrote:
>
> > If possible, let's have all the I/O stats (even for WAL) in
> > pg_stat_io. Can't we show the WAL data we get from buffers in the hits
> > column and then have read_bytes or something like that to know th
Hi,
Thank you for looking into this! And, sorry for the late answer.
On Mon, 13 May 2024 at 17:12, Bharath Rupireddy
wrote:
>
> On Fri, Apr 19, 2024 at 1:32 PM Nazir Bilal Yavuz wrote:
> >
> > > I wanted to inform you that the 73f0a13266 commit changed all WALRead
> > > calls to read variable b
> If possible, let's have all the I/O stats (even for WAL) in
> pg_stat_io. Can't we show the WAL data we get from buffers in the hits
> column and then have read_bytes or something like that to know the
> amount of data read?
The ‘hits’ column in ‘pg_stat_io’ is a vital indicator for adjusting a
On Mon, May 13, 2024 at 7:42 PM Bharath Rupireddy
wrote:
>
> On Fri, Apr 19, 2024 at 1:32 PM Nazir Bilal Yavuz wrote:
> >
> > > I wanted to inform you that the 73f0a13266 commit changed all WALRead
> > > calls to read variable bytes, only the WAL receiver was reading
> > > variable bytes before.
On Fri, Apr 19, 2024 at 1:32 PM Nazir Bilal Yavuz wrote:
>
> > I wanted to inform you that the 73f0a13266 commit changed all WALRead
> > calls to read variable bytes, only the WAL receiver was reading
> > variable bytes before.
>
> I want to start working on this again if possible. I will try to
>
Hi,
On Fri, 19 Apr 2024 at 11:01, Nazir Bilal Yavuz wrote:
> > On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote:
> > >
> > > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote:
> > > > I agree with your points. While the other I/O related work is
> > > > happening we can discus
Hi,
On Mon, 19 Feb 2024 at 10:28, Nazir Bilal Yavuz wrote:
>
> Hi,
>
> On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote:
> >
> > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote:
> > > I agree with your points. While the other I/O related work is
> > > happening we can discus
Hi,
On Thu, 18 Jan 2024 at 04:22, Michael Paquier wrote:
>
> On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote:
> > I agree with your points. While the other I/O related work is
> > happening we can discuss what we should do in the variable op_byte
> > cases. Also, this is happeni
On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote:
> I agree with your points. While the other I/O related work is
> happening we can discuss what we should do in the variable op_byte
> cases. Also, this is happening only for WAL right now but if we try to
> extend pg_stat_io in the
Hi,
On Mon, 15 Jan 2024 at 09:27, Michael Paquier wrote:
>
> On Fri, Jan 12, 2024 at 04:23:26PM +0300, Nazir Bilal Yavuz wrote:
> > On Thu, 11 Jan 2024 at 17:28, Melanie Plageman
> > wrote:
> >> Even if we made a separate view for WAL I/O stats, we would still have
> >> this issue of variable s
On Fri, Jan 12, 2024 at 04:23:26PM +0300, Nazir Bilal Yavuz wrote:
> On Thu, 11 Jan 2024 at 17:28, Melanie Plageman
> wrote:
>> Even if we made a separate view for WAL I/O stats, we would still have
>> this issue of variable sized I/O vs block sized I/O and would probably
>> end up solving it wit
Hi,
On Thu, 11 Jan 2024 at 17:28, Melanie Plageman
wrote:
>
> On Thu, Jan 11, 2024 at 6:19 AM Nazir Bilal Yavuz wrote:
> >
> > On Thu, 11 Jan 2024 at 08:01, Michael Paquier wrote:
> > >
> > > On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote:
> > > > On Wed, Jan 3, 2024 at 8:11 A
On Thu, Jan 11, 2024 at 6:19 AM Nazir Bilal Yavuz wrote:
>
> On Thu, 11 Jan 2024 at 08:01, Michael Paquier wrote:
> >
> > On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote:
> > > On Wed, Jan 3, 2024 at 8:11 AM Nazir Bilal Yavuz
> > > wrote:
> > >> On Sun, 31 Dec 2023 at 03:58, Mi
Hi,
On Thu, 11 Jan 2024 at 08:01, Michael Paquier wrote:
>
> On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote:
> > I have code review feedback as well, but I've saved that for my next email.
>
> Ah, cool.
>
> > On Wed, Jan 3, 2024 at 8:11 AM Nazir Bilal Yavuz wrote:
> >> On Sun,
On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote:
> I have code review feedback as well, but I've saved that for my next email.
Ah, cool.
> On Wed, Jan 3, 2024 at 8:11 AM Nazir Bilal Yavuz wrote:
>> On Sun, 31 Dec 2023 at 03:58, Michael Paquier wrote:
>> Oh, I understand it now.
I have code review feedback as well, but I've saved that for my next email.
On Wed, Jan 3, 2024 at 8:11 AM Nazir Bilal Yavuz wrote:
>
> On Sun, 31 Dec 2023 at 03:58, Michael Paquier wrote:
> >
> > On Tue, Dec 26, 2023 at 03:35:52PM +0300, Nazir Bilal Yavuz wrote:
> > > On Tue, 26 Dec 2023 at 13:
Hi,
On Wed, 10 Jan 2024 at 08:25, Michael Paquier wrote:
>
> On Wed, Jan 03, 2024 at 04:10:58PM +0300, Nazir Bilal Yavuz wrote:
> >
> > I thought removing op_bytes completely ( as you said "This patch
> > extends it with two more operation sizes, and there are even cases
> > where it may be a var
On Wed, Jan 03, 2024 at 04:10:58PM +0300, Nazir Bilal Yavuz wrote:
> On Sun, 31 Dec 2023 at 03:58, Michael Paquier wrote:
>> Apologies if my previous wording sounded confusing. The idea I had in
>> mind was to keep op_bytes in pg_stat_io, and extend it so as a value
>> of NULL (or 0, or -1) is a
Hi,
On Sun, 31 Dec 2023 at 03:58, Michael Paquier wrote:
>
> On Tue, Dec 26, 2023 at 03:35:52PM +0300, Nazir Bilal Yavuz wrote:
> > On Tue, 26 Dec 2023 at 13:10, Michael Paquier wrote:
> >> I am not sure while the whole point of the exercise is to have all the
> >> I/O related data in a single v
On Tue, Dec 26, 2023 at 03:35:52PM +0300, Nazir Bilal Yavuz wrote:
> On Tue, 26 Dec 2023 at 13:10, Michael Paquier wrote:
>> I am not sure while the whole point of the exercise is to have all the
>> I/O related data in a single view. Something that I've also found a
>> bit disturbing yesterday wh
Hi,
On Tue, 26 Dec 2023 at 13:10, Michael Paquier wrote:
>
> On Tue, Dec 26, 2023 at 11:27:16AM +0300, Nazir Bilal Yavuz wrote:
> > Maybe it is better to create a pg_stat_io_wal view like you said
> > before. We could remove unused columns and add op_bytes for each
> > writes and reads. Also, we
On Tue, Dec 26, 2023 at 11:27:16AM +0300, Nazir Bilal Yavuz wrote:
> Maybe it is better to create a pg_stat_io_wal view like you said
> before. We could remove unused columns and add op_bytes for each
> writes and reads. Also, we can track both the number of bytes and the
> number of the operations
Hi,
On Tue, 26 Dec 2023 at 03:06, Michael Paquier wrote:
>
> On Mon, Dec 25, 2023 at 04:09:34PM +0300, Nazir Bilal Yavuz wrote:
> > On Wed, 9 Aug 2023 at 21:52, Melanie Plageman
> > wrote:
> >> If there is any combination of BackendType and IOContext which will
> >> always read XLOG_BLCKSZ byte
On Mon, Dec 25, 2023 at 04:09:34PM +0300, Nazir Bilal Yavuz wrote:
> On Wed, 9 Aug 2023 at 21:52, Melanie Plageman
> wrote:
>> If there is any combination of BackendType and IOContext which will
>> always read XLOG_BLCKSZ bytes, we could use XLOG_BLCKSZ for that row's
>> op_bytes. For other cases
Hi,
Thanks for the review and feedback on your previous reply!
On Mon, 25 Dec 2023 at 09:40, Michael Paquier wrote:
>
> On Mon, Dec 25, 2023 at 03:20:58PM +0900, Michael Paquier wrote:
> > pgstat_tracks_io_bktype() does not look correct to me. Why is the WAL
> > receiver considered as something
On Mon, Dec 25, 2023 at 03:20:58PM +0900, Michael Paquier wrote:
> pgstat_tracks_io_bktype() does not look correct to me. Why is the WAL
> receiver considered as something correct in the list of backend types,
> while the intention is to *not* add it to pg_stat_io? I have tried to
> switche to th
On Sat, Dec 16, 2023 at 08:20:57PM +0100, Michael Paquier wrote:
> One thing that 0001 missed is an update of the header where the
> function is declared. I've edited a few things, and applied it to
> start on this stuff. The rest will have to wait a bit more..
I have been reviewing the whole, a
On Tue, Dec 12, 2023 at 02:29:03PM +0300, Nazir Bilal Yavuz wrote:
> On Tue, 5 Dec 2023 at 09:16, Michael Paquier wrote:
>> This interface from pgstat_prepare_io_time() is not really good,
>> because we could finish by setting io_start in the existing code paths
>> calling this routine even if tra
Hi,
Thanks for the feedback! The new version of the patch is attached.
On Tue, 5 Dec 2023 at 09:16, Michael Paquier wrote:
>
> - if (io_op == IOOP_WRITE || io_op == IOOP_EXTEND)
> + if (io_op == IOOP_EXTEND || io_op == IOOP_WRITE)
>
> Unrelated diff.
Done.
>
> + if (io_object ==
On Fri, Dec 01, 2023 at 12:02:05PM +0300, Nazir Bilal Yavuz wrote:
> Thanks for all the feedback. I am sharing the new version of the patchset.
>
> - I didn't move 'PendingWalStats.wal_sync' out from the
> 'pgstat_count_io_op_n' function because they count the same thing
> (block vs system calls)
Hi,
Thanks for all the feedback. I am sharing the new version of the patchset.
Current status of the patchset is:
- IOOBJECT_WAL / IOCONTEXT_NORMAL / read, write, fsync stats and their
tests are added.
- IOOBJECT_WAL / IOCONTEXT_INIT stats and their tests are added.
- Documentation is updated.
-
Hi,
On Wed, 8 Nov 2023 at 10:34, Bharath Rupireddy
wrote:
>
> Is there any plan to account for WAL read stats in the WALRead()
> function which will cover walsenders i.e. WAL read by logical and
> streaming replication, WAL read by pg_walinspect and so on? I see the
> patch already covers WAL rea
On Mon, Nov 20, 2023 at 05:43:17PM +0300, Nazir Bilal Yavuz wrote:
> Yes, the timings for the writes and the syncs should work. Another
> question I have in mind is the pg_stat_reset_shared() function. When
> we call it with 'io' it will reset pg_stat_wal's timings and when we
> call it with 'wal'
Hi,
Thanks for the feedback.
On Mon, 20 Nov 2023 at 10:47, Michael Paquier wrote:
>
> On Thu, Nov 09, 2023 at 02:39:26PM +0300, Nazir Bilal Yavuz wrote:
> > There are some differences between pg_stat_wal and pg_stat_io while
> > collecting WAL stats. For example in the XLogWrite() function in th
On Thu, Nov 09, 2023 at 02:39:26PM +0300, Nazir Bilal Yavuz wrote:
> There are some differences between pg_stat_wal and pg_stat_io while
> collecting WAL stats. For example in the XLogWrite() function in the
> xlog.c file, pg_stat_wal counts wal_writes as write system calls. This
> is not something
Hi,
On Wed, 8 Nov 2023 at 04:19, Andres Freund wrote:
>
> Hi,
>
> On 2023-11-08 09:52:16 +0900, Michael Paquier wrote:
> > By the way, if the write/sync quantities and times begin to be tracked
> > by pg_stat_io, I'd see a pretty good argument in removing the
> > equivalent columns in pg_stat_wal
Hi,
Thanks for all the feedback!
On Wed, 8 Nov 2023 at 08:59, Michael Paquier wrote:
>
> By the way, note that the patch is failing to apply, and that I've
> switched it as waiting on author on 10/26.
Here is an updated patchset in attachment. Rebased on the latest HEAD
and changed 'pgstat_repo
On Wed, Sep 20, 2023 at 1:28 PM Nazir Bilal Yavuz wrote:
>
> Hi,
>
> Thanks for the review!
>
> Current status of the patch is:
> - IOOBJECT_WAL / IOCONTEXT_NORMAL read, write and fsync stats are added.
> - IOOBJECT_WAL / IOCONTEXT_NORMAL write and fsync tests are added.
> - IOOBJECT_WAL / IOCONTE
On Wed, Nov 08, 2023 at 10:27:44AM +0900, Michael Paquier wrote:
> Yep, I'd be OK with that as well to maintain compatibility.
By the way, note that the patch is failing to apply, and that I've
switched it as waiting on author on 10/26.
--
Michael
signature.asc
Description: PGP signature
On Tue, Nov 07, 2023 at 05:19:28PM -0800, Andres Freund wrote:
> Another approach would be to fetch the relevant columns from pg_stat_io in the
> pg_stat_wal view. That'd avoid double accounting and breaking existing
> monitoring.
Yep, I'd be OK with that as well to maintain compatibility.
--
Mich
Hi,
On 2023-11-08 09:52:16 +0900, Michael Paquier wrote:
> By the way, if the write/sync quantities and times begin to be tracked
> by pg_stat_io, I'd see a pretty good argument in removing the
> equivalent columns in pg_stat_wal. It looks like this would reduce
> the confusion related to the han
On Tue, Nov 07, 2023 at 03:30:48PM -0800, Andres Freund wrote:
> I strongly disagree. A significant part of the design of pg_stat_io was to
> make it possible to collect multiple sources of IO in a single view, so that
> sysadmins don't have to look in dozens of places to figure out what is causing
Hi,
On 2023-10-26 15:28:32 +0900, Michael Paquier wrote:
> On top of that pg_stat_io is now for block-based I/O operations, so
> that does not fit entirely in the picture, though I guess that Melanie
> has thought more on the matter than me. That may be also a matter of
> taste.
I strongly disag
On Mon, Nov 06, 2023 at 03:35:01PM +0300, Nazir Bilal Yavuz wrote:
> Results are:
>
> ╔═╦═══╦╗
> ║ ║ track_wal_io_timing ║║
> ╠═╬═══╦═══╬╣
> ║ clock ║ on ║ off ║ cha
Hi,
On Tue, 31 Oct 2023 at 16:57, Nazir Bilal Yavuz wrote:
> On Thu, 26 Oct 2023 at 09:28, Michael Paquier wrote:
> >
> > And perhaps just putting that everything that calls
> > pgstat_count_io_op_time() under track_io_timing is just natural?
> > What's the performance regression you would expec
Hi,
Thank you for the feedback!
On Thu, 26 Oct 2023 at 09:28, Michael Paquier wrote:
>
> On Wed, Sep 20, 2023 at 10:57:48AM +0300, Nazir Bilal Yavuz wrote:
> > Any kind of feedback would be appreciated.
>
> This was registered in the CF, so I have given it a look. Note that
> 0001 has a conflic
On Wed, Sep 20, 2023 at 10:57:48AM +0300, Nazir Bilal Yavuz wrote:
> Any kind of feedback would be appreciated.
This was registered in the CF, so I have given it a look. Note that
0001 has a conflict with pgstat_count_io_op_time(), so it cannot be
applied.
+pgstat_should_track_io_time(IOObject i
Hi,
Thanks for the review!
Current status of the patch is:
- IOOBJECT_WAL / IOCONTEXT_NORMAL read, write and fsync stats are added.
- IOOBJECT_WAL / IOCONTEXT_NORMAL write and fsync tests are added.
- IOOBJECT_WAL / IOCONTEXT_INIT stats are added.
- pg_stat_io shows different op_bytes for the IOO
On Thu, Aug 03, 2023 at 04:38:41PM +0300, Nazir Bilal Yavuz wrote:
> Current status of the patch is:
> - 'WAL read' stats in xlogrecovery.c are added to pg_stat_io.
> - IOCONTEXT_INIT is added to count 'WAL init'. 'WAL init' stats are
> added to pg_stat_io.
> - pg_stat_io shows different op_bytes f
Hi,
Thanks for the review!
Current status of the patch is:
- 'WAL read' stats in xlogrecovery.c are added to pg_stat_io.
- IOCONTEXT_INIT is added to count 'WAL init'. 'WAL init' stats are
added to pg_stat_io.
- pg_stat_io shows different op_bytes for the IOOBJECT_WAL operations.
- Working on whi
On Wed, Jun 28, 2023 at 6:09 AM Nazir Bilal Yavuz wrote:
> This is a WIP patch to add WAL write and fsync stats to pg_stat_io
> view.
Thanks for working on this! I have some feedback on the content of the
patch as well as some items that I feel are missing.
I think it would be good to count WAL
Hi,
This is a WIP patch to add WAL write and fsync stats to pg_stat_io
view. There is a track_io_timing variable to control pg_stat_io
timings and a track_wal_io_timing variable to control WAL timings. I
couldn't decide on which logic to enable WAL timings on pg_stat_io.
For now, both pg_stat_io a
94 matches
Mail list logo
