Re: wal stats questions

2021-05-19 Thread Masahiro Ikeda
On 2021/05/19 11:40, Fujii Masao wrote: > Thanks for updating the patch! I modified some comments slightly and > pushed that version of the patch. Thanks a lot! Regards, -- Masahiro Ikeda NTT DATA CORPORATION

Re: wal stats questions

2021-05-18 Thread Fujii Masao
On 2021/05/18 9:57, Masahiro Ikeda wrote: On 2021/05/17 22:34, Fujii Masao wrote: On 2021/05/17 16:39, Masahiro Ikeda wrote: Thanks for your comments! + * is executed, wal records aren't nomally generated (although HOT makes nomally -> normally? Yes, fixed. + * It's not

Re: wal stats questions

2021-05-17 Thread Masahiro Ikeda
On 2021/05/17 22:34, Fujii Masao wrote: > > > On 2021/05/17 16:39, Masahiro Ikeda wrote: >> >> Thanks for your comments! >> + * is executed, wal records aren't nomally generated (although HOT makes >>> >>> nomally -> normally? >> >> Yes, fixed. >> + * It's not enough to

Re: wal stats questions

2021-05-17 Thread Fujii Masao
On 2021/05/17 16:39, Masahiro Ikeda wrote: Thanks for your comments! + * is executed, wal records aren't nomally generated (although HOT makes nomally -> normally? Yes, fixed. + * It's not enough to check the number of generated wal records, for + * example the walwriter m

Re: wal stats questions

2021-05-17 Thread Masahiro Ikeda
Thanks for your comments! >> + * is executed, wal records aren't nomally generated (although HOT makes > > nomally -> normally? Yes, fixed. >> + * It's not enough to check the number of generated wal records, for >> + * example the walwriter may write/sync the WAL although it doesn

Re: wal stats questions

2021-05-17 Thread torikoshia
On 2021-05-13 09:05, Masahiro Ikeda wrote: On 2021/05/12 19:19, Fujii Masao wrote: On 2021/05/11 18:46, Masahiro Ikeda wrote: On 2021/05/11 16:44, Fujii Masao wrote: On 2021/04/28 9:10, Masahiro Ikeda wrote: On 2021/04/27 21:56, Fujii Masao wrote: On 2021/04/26 10:11, Masahiro Iked

Re: wal stats questions

2021-05-12 Thread Masahiro Ikeda
On 2021/05/12 19:19, Fujii Masao wrote: > > > On 2021/05/11 18:46, Masahiro Ikeda wrote: >> >> >> On 2021/05/11 16:44, Fujii Masao wrote: >>> >>> >>> On 2021/04/28 9:10, Masahiro Ikeda wrote: On 2021/04/27 21:56, Fujii Masao wrote: > > > On 2021/04/26 10:11, Masahiro

Re: wal stats questions

2021-05-12 Thread Fujii Masao
On 2021/05/11 18:46, Masahiro Ikeda wrote: On 2021/05/11 16:44, Fujii Masao wrote: On 2021/04/28 9:10, Masahiro Ikeda wrote: On 2021/04/27 21:56, Fujii Masao wrote: On 2021/04/26 10:11, Masahiro Ikeda wrote: First patch has only the changes for pg_stat_wal view. ("v6-0001-performa

Re: wal stats questions

2021-05-11 Thread Masahiro Ikeda
On 2021/05/11 16:44, Fujii Masao wrote: > > > On 2021/04/28 9:10, Masahiro Ikeda wrote: >> >> >> On 2021/04/27 21:56, Fujii Masao wrote: >>> >>> >>> On 2021/04/26 10:11, Masahiro Ikeda wrote: First patch has only the changes for pg_stat_wal view. ("v6-0001-performance-improvement

Re: wal stats questions

2021-05-11 Thread Fujii Masao
On 2021/04/28 9:10, Masahiro Ikeda wrote: Second one has the changes for the type of the BufferUsage's and WalUsage's members. I change the type from long to int64. Is it better to make new thread? ("v6-0002-change-the-data-type-of-XXXUsage-from-long-to-int64.patch") Will review the patch la

Re: wal stats questions

2021-05-11 Thread Fujii Masao
On 2021/04/28 9:10, Masahiro Ikeda wrote: On 2021/04/27 21:56, Fujii Masao wrote: On 2021/04/26 10:11, Masahiro Ikeda wrote: First patch has only the changes for pg_stat_wal view. ("v6-0001-performance-improvements-of-reporting-wal-stats-without-introducing-a-new-variable.patch") +  

Re: wal stats questions

2021-04-27 Thread Masahiro Ikeda
On 2021/04/27 21:56, Fujii Masao wrote: > > > On 2021/04/26 10:11, Masahiro Ikeda wrote: >> >> First patch has only the changes for pg_stat_wal view. >> ("v6-0001-performance-improvements-of-reporting-wal-stats-without-introducing-a-new-variable.patch") >> > > +    pgWalUsage.wal_records

Re: wal stats questions

2021-04-27 Thread Fujii Masao
On 2021/04/26 10:11, Masahiro Ikeda wrote: On 2021/04/23 16:30, Fujii Masao wrote: On 2021/04/23 10:25, Andres Freund wrote: Hi, On 2021-04-23 09:26:17 +0900, Masahiro Ikeda wrote: On 2021/04/23 0:36, Andres Freund wrote: On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: On 2021/04/

Re: wal stats questions

2021-04-25 Thread Masahiro Ikeda
On 2021/04/23 16:30, Fujii Masao wrote: > > > On 2021/04/23 10:25, Andres Freund wrote: >> Hi, >> >> On 2021-04-23 09:26:17 +0900, Masahiro Ikeda wrote: >>> On 2021/04/23 0:36, Andres Freund wrote: On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: > On 2021/04/21 18:31, Masahiro Ikeda

Re: wal stats questions

2021-04-23 Thread Fujii Masao
On 2021/04/23 10:25, Andres Freund wrote: Hi, On 2021-04-23 09:26:17 +0900, Masahiro Ikeda wrote: On 2021/04/23 0:36, Andres Freund wrote: On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: On 2021/04/21 18:31, Masahiro Ikeda wrote: BTW, is it better to change PgStat_Counter from int64 to

Re: wal stats questions

2021-04-22 Thread Andres Freund
Hi, On 2021-04-23 09:26:17 +0900, Masahiro Ikeda wrote: > On 2021/04/23 0:36, Andres Freund wrote: > > On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: > >> On 2021/04/21 18:31, Masahiro Ikeda wrote: > BTW, is it better to change PgStat_Counter from int64 to uint64 because> > there ar

Re: wal stats questions

2021-04-22 Thread Fujii Masao
On 2021/04/23 9:26, Masahiro Ikeda wrote:  On 2021/04/23 0:36, Andres Freund wrote: Hi On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: On 2021/04/21 18:31, Masahiro Ikeda wrote: BTW, is it better to change PgStat_Counter from int64 to uint64 because> there aren't any counters with ne

Re: wal stats questions

2021-04-22 Thread Masahiro Ikeda
 On 2021/04/23 0:36, Andres Freund wrote: > Hi > > On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: >> On 2021/04/21 18:31, Masahiro Ikeda wrote: BTW, is it better to change PgStat_Counter from int64 to uint64 because> there aren't any counters with negative number? >> On second tho

Re: wal stats questions

2021-04-22 Thread Andres Freund
Hi On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: > > > On 2021/04/21 18:31, Masahiro Ikeda wrote: > >> BTW, is it better to change PgStat_Counter from int64 to uint64 because> > >> there aren't any counters with negative number? > > On second thought, it's ok even if the counters like wal

Re: wal stats questions

2021-04-22 Thread Fujii Masao
On 2021/04/21 18:31, Masahiro Ikeda wrote: BTW, is it better to change PgStat_Counter from int64 to uint64 because> there aren't any counters with negative number? On second thought, it's ok even if the counters like wal_records get overflowed. Because they are always used to calculate the

Re: wal stats questions

2021-04-21 Thread Masahiro Ikeda
On 2021/04/21 15:08, torikoshia wrote: > On 2021-04-16 10:27, Masahiro Ikeda wrote: >> On 2021/04/13 9:33, Fujii Masao wrote: >>> >>> >>> On 2021/03/30 20:37, Masahiro Ikeda wrote: OK, I added the condition to the fast-return check. I noticed that I misunderstood that the purpose is

Re: wal stats questions

2021-04-20 Thread torikoshia
On 2021-04-16 10:27, Masahiro Ikeda wrote: On 2021/04/13 9:33, Fujii Masao wrote: On 2021/03/30 20:37, Masahiro Ikeda wrote: OK, I added the condition to the fast-return check. I noticed that I misunderstood that the purpose is to avoid expanding a clock check using WAL stats counters. But,

Re: wal stats questions

2021-04-15 Thread Masahiro Ikeda
On 2021/04/13 9:33, Fujii Masao wrote: > > > On 2021/03/30 20:37, Masahiro Ikeda wrote: >> OK, I added the condition to the fast-return check. I noticed that I >> misunderstood that the purpose is to avoid expanding a clock check using WAL >> stats counters. But, the purpose is to make the cond

Re: wal stats questions

2021-04-12 Thread Fujii Masao
On 2021/03/30 20:37, Masahiro Ikeda wrote: OK, I added the condition to the fast-return check. I noticed that I misunderstood that the purpose is to avoid expanding a clock check using WAL stats counters. But, the purpose is to make the conditions stricter, right? Yes. Currently if the follo

Re: wal stats questions

2021-03-30 Thread Masahiro Ikeda
On 2021/03/30 17:28, Kyotaro Horiguchi wrote: > At Tue, 30 Mar 2021 09:41:24 +0900, Masahiro Ikeda > wrote in >> I update the patch since there were my misunderstanding points. >> >> On 2021/03/26 16:20, Masahiro Ikeda wrote: >>> Thanks for many your suggestions! >>> I made the patch to handle

Re: wal stats questions

2021-03-30 Thread Kyotaro Horiguchi
At Tue, 30 Mar 2021 09:41:24 +0900, Masahiro Ikeda wrote in > I update the patch since there were my misunderstanding points. > > On 2021/03/26 16:20, Masahiro Ikeda wrote: > > Thanks for many your suggestions! > > I made the patch to handle the issues. > > > >> 1) What is the motivation to ha

Re: wal stats questions

2021-03-29 Thread Masahiro Ikeda
I update the patch since there were my misunderstanding points. On 2021/03/26 16:20, Masahiro Ikeda wrote: > Thanks for many your suggestions! > I made the patch to handle the issues. > >> 1) What is the motivation to have both prevWalUsage and pgWalUsage, >>instead of just accumulating the s

Re: wal stats questions

2021-03-28 Thread Kyotaro Horiguchi
At Mon, 29 Mar 2021 11:09:00 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 26 Mar 2021 10:07:45 -0700, Andres Freund wrote > in > > Hi, > > > > On 2021-03-25 16:37:10 +0900, Kyotaro Horiguchi wrote: > > > On the other hand, the counters are incremented in XLogInsertRecord() > > > and I t

Re: wal stats questions

2021-03-28 Thread Kyotaro Horiguchi
At Fri, 26 Mar 2021 10:07:45 -0700, Andres Freund wrote in > Hi, > > On 2021-03-25 16:37:10 +0900, Kyotaro Horiguchi wrote: > > On the other hand, the counters are incremented in XLogInsertRecord() > > and I think we don't want add instructions there. > > I don't really buy this. Setting a bool

Re: wal stats questions

2021-03-26 Thread Andres Freund
Hi, On 2021-03-25 16:37:10 +0900, Kyotaro Horiguchi wrote: > On the other hand, the counters are incremented in XLogInsertRecord() > and I think we don't want add instructions there. I don't really buy this. Setting a boolean to true, in a cacheline you're already touching, isn't that much compar

Re: wal stats questions

2021-03-26 Thread Masahiro Ikeda
Thanks for many your suggestions! I made the patch to handle the issues. > 1) What is the motivation to have both prevWalUsage and pgWalUsage, >instead of just accumulating the stats since the last submission? >There doesn't seem to be any comment explaining it? Computing >differences

Re: wal stats questions

2021-03-25 Thread Kyotaro Horiguchi
At Fri, 26 Mar 2021 10:32:23 +0900, Fujii Masao wrote in > > I may be misunderstanding or missing something, but the only place > > where pgWalUsage counters are increased is XLogInsertRecrod. That is, > > pgWalUsage counts wal insertions, not writes nor flushes. AFAICS > > Yes. And WalStats

Re: wal stats questions

2021-03-25 Thread Fujii Masao
On 2021/03/26 10:08, Kyotaro Horiguchi wrote: At Thu, 25 Mar 2021 19:01:23 +0900, Fujii Masao wrote in On 2021/03/25 16:37, Kyotaro Horiguchi wrote: pgWalUsage was used without resetting and we (I) thought that that behavior should be preserved. On second thought, as Andres suggested, we

Re: wal stats questions

2021-03-25 Thread Kyotaro Horiguchi
At Thu, 25 Mar 2021 19:01:23 +0900, Fujii Masao wrote in > On 2021/03/25 16:37, Kyotaro Horiguchi wrote: > > pgWalUsage was used without resetting and we (I) thought that that > > behavior should be preserved. On second thought, as Andres suggested, > > we can just reset pgWalUsage at sending s

Re: wal stats questions

2021-03-25 Thread Fujii Masao
On 2021/03/25 16:37, Kyotaro Horiguchi wrote: At Wed, 24 Mar 2021 21:07:26 -0700, Andres Freund wrote in Hi, On 2021-03-25 10:51:56 +0900, Masahiro Ikeda wrote: On 2021/03/25 8:22, Andres Freund wrote: 1) What is the motivation to have both prevWalUsage and pgWalUsage, instead of just

Re: wal stats questions

2021-03-25 Thread Kyotaro Horiguchi
At Wed, 24 Mar 2021 21:07:26 -0700, Andres Freund wrote in > Hi, > > On 2021-03-25 10:51:56 +0900, Masahiro Ikeda wrote: > > On 2021/03/25 8:22, Andres Freund wrote: > > > 1) What is the motivation to have both prevWalUsage and pgWalUsage, > > >instead of just accumulating the stats since th

Re: wal stats questions

2021-03-24 Thread Andres Freund
Hi, On 2021-03-25 10:51:56 +0900, Masahiro Ikeda wrote: > On 2021/03/25 8:22, Andres Freund wrote: > > 1) What is the motivation to have both prevWalUsage and pgWalUsage, > >instead of just accumulating the stats since the last submission? > >There doesn't seem to be any comment explaining

Re: wal stats questions

2021-03-24 Thread Masahiro Ikeda
On 2021/03/25 8:22, Andres Freund wrote: > Hi, > > I got a few questions about the wal stats while working on the shmem > stats patch: Thanks for your reviews. > 1) What is the motivation to have both prevWalUsage and pgWalUsage, >instead of just accumulating the stats since the last sub

wal stats questions

2021-03-24 Thread Andres Freund
Hi, I got a few questions about the wal stats while working on the shmem stats patch: 1) What is the motivation to have both prevWalUsage and pgWalUsage, instead of just accumulating the stats since the last submission? There doesn't seem to be any comment explaining it? Computing differ