Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-10-10 Thread Anton A. Melnikov
On 10.10.2024 18:14, Fujii Masao wrote: Thanks for the review! Pushed. Thanks a lot! With the best regards, -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-10-10 Thread Fujii Masao
On 2024/10/08 23:16, Anton A. Melnikov wrote: On 08.10.2024 15:42, Fujii Masao wrote: On 2024/09/30 12:26, Fujii Masao wrote: In 0002.patch, I also modified the description of num_requested from "Number of backend requested checkpoints" to remove "backend," as it can be confusing since num_

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-10-08 Thread Anton A. Melnikov
On 08.10.2024 15:42, Fujii Masao wrote: On 2024/09/30 12:26, Fujii Masao wrote: In 0002.patch, I also modified the description of num_requested from "Number of backend requested checkpoints" to remove "backend," as it can be confusing since num_requested includes requests from sources other th

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-10-08 Thread Fujii Masao
On 2024/09/30 12:26, Fujii Masao wrote: In 0002.patch, I also modified the description of num_requested from "Number of backend requested checkpoints" to remove "backend," as it can be confusing since num_requested includes requests from sources other than the backend. Thought? Agreed. E.g. f

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-30 Thread Fujii Masao
On 2024/09/30 16:00, Anton A. Melnikov wrote: On 30.09.2024 06:26, Fujii Masao wrote: Thanks for the review! I've pushed the 0001 patch. Thanks a lot! As for switching in the pg_proc.dat entries the idea was to put them in order so that the pg_stat_get_checkpointer* functions were groupe

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-30 Thread Anton A. Melnikov
On 30.09.2024 06:26, Fujii Masao wrote: Thanks for the review! I've pushed the 0001 patch. Thanks a lot! As for switching in the pg_proc.dat entries the idea was to put them in order so that the pg_stat_get_checkpointer* functions were grouped together. I don't know if this is the common an

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-29 Thread Fujii Masao
On 2024/09/22 13:55, Anton A. Melnikov wrote: On 20.09.2024 19:19, Fujii Masao wrote: I've attached the updated version (0001.patch). I made some cosmetic changes, including reverting the switch in the entries for pg_stat_get_checkpointer_write_time and pg_stat_get_checkpointer_sync_time in

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-21 Thread Anton A. Melnikov
On 20.09.2024 19:19, Fujii Masao wrote: I've attached the updated version (0001.patch). I made some cosmetic changes, including reverting the switch in the entries for pg_stat_get_checkpointer_write_time and pg_stat_get_checkpointer_sync_time in pg_proc.dat, as I didn’t think that change was nec

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-20 Thread Fujii Masao
On 2024/09/19 19:16, Anton A. Melnikov wrote: On 18.09.2024 21:04, Fujii Masao wrote: -    CreateCheckPoint(flags); -    ckpt_performed = true; +    ckpt_performed = CreateCheckPoint(flags); This change could result in the next scheduled checkpoint being

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-19 Thread Anton A. Melnikov
On 18.09.2024 21:04, Fujii Masao wrote: -    CreateCheckPoint(flags); -    ckpt_performed = true; +    ckpt_performed = CreateCheckPoint(flags); This change could result in the next scheduled checkpoint being triggered in 15 seconds if a checkpoint is skippe

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-18 Thread Fujii Masao
On 2024/09/18 23:35, Anton A. Melnikov wrote: Fujii, Alexander thanks a lot! On 17.09.2024 05:47, Fujii Masao wrote: Regarding the patch:   if (do_restartpoint)   PendingCheckpointerStats.restartpoints_performed++; +    else + 

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-18 Thread Fujii Masao
On 2024/09/18 21:22, Alexander Korotkov wrote: Patch attached. Unless there are any objections, I plan to commit this and back-patch it to v17. I've checked this patch, it looks good to me. Generally, it looks like I should be in charge for this, given I've committed previous patch by Anton

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-18 Thread Anton A. Melnikov
Fujii, Alexander thanks a lot! On 17.09.2024 05:47, Fujii Masao wrote: Regarding the patch: if (do_restartpoint) PendingCheckpointerStats.restartpoints_performed++; +    else +    PendingCheckpointerStats.num_performed++; I ex

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-18 Thread Alexander Korotkov
On Wed, Sep 18, 2024 at 1:21 PM Fujii Masao wrote: > On 2024/09/17 11:47, Fujii Masao wrote: > > > > > > On 2024/09/16 23:30, Anton A. Melnikov wrote: > >> +1 > >> This idea seems quite tenable to me. > >> > >> There is a small clarification. Now if there were no skipped restartpoints > >> then >

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-18 Thread Fujii Masao
On 2024/09/17 11:47, Fujii Masao wrote: On 2024/09/16 23:30, Anton A. Melnikov wrote: +1 This idea seems quite tenable to me. There is a small clarification. Now if there were no skipped restartpoints then restartpoints_done will be equal to restartpoints_timed + restartpoints_req. Similar

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-16 Thread Fujii Masao
On 2024/09/16 23:30, Anton A. Melnikov wrote: +1 This idea seems quite tenable to me. There is a small clarification. Now if there were no skipped restartpoints then restartpoints_done will be equal to restartpoints_timed + restartpoints_req. Similar for checkpoints. So i tried to introduce n

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-16 Thread Anton A. Melnikov
Hi! On 13.09.2024 18:20, Fujii Masao wrote: If I understand correctly, restartpoints_timed and restartpoints_done were separated because a restartpoint can be skipped. restartpoints_timed counts when a restartpoint is triggered by a timeout, whether it runs or not, while restartpoints_done only

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-09-13 Thread Fujii Masao
On 2024/03/14 9:19, Alexander Korotkov wrote: On Mon, Mar 11, 2024 at 11:48 AM Alexander Korotkov wrote: On Mon, Mar 11, 2024 at 5:43 AM Anton A. Melnikov wrote: On 11.03.2024 03:39, Alexander Korotkov wrote: Now that we distinguish stats for checkpoints and restartpoints, we need to upd

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-03-14 Thread Anton A. Melnikov
On 14.03.2024 03:19, Alexander Korotkov wrote: Pushed. Thanks! -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-03-13 Thread Alexander Korotkov
On Mon, Mar 11, 2024 at 11:48 AM Alexander Korotkov wrote: > > On Mon, Mar 11, 2024 at 5:43 AM Anton A. Melnikov > wrote: > > On 11.03.2024 03:39, Alexander Korotkov wrote: > > > Now that we distinguish stats for checkpoints and > > > restartpoints, we need to update the docs. Please, check the

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-03-11 Thread Alexander Korotkov
On Mon, Mar 11, 2024 at 5:43 AM Anton A. Melnikov wrote: > On 11.03.2024 03:39, Alexander Korotkov wrote: > > Now that we distinguish stats for checkpoints and > > restartpoints, we need to update the docs. Please, check the patch > > attached. > > Maybe bring the pg_stat_get_checkpointer_buffers

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-03-10 Thread Anton A. Melnikov
On 11.03.2024 03:39, Alexander Korotkov wrote: Now that we distinguish stats for checkpoints and restartpoints, we need to update the docs. Please, check the patch attached. Maybe bring the pg_stat_get_checkpointer_buffers_written() description in consistent with these changes? Like that: --

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-03-10 Thread Alexander Korotkov
On Sat, Mar 9, 2024 at 4:38 PM Magnus Hagander wrote: > Per the docs, the sync_time, write_time and buffers_written only apply > to checkpoints, not restartpoints. Is this correct? AFAICT from a > quick look at the code they include both checkpoints and restartpoints > in which case I think the do

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2024-03-09 Thread Magnus Hagander
On Fri, Dec 22, 2023 at 11:04 PM Alexander Korotkov wrote: > > Hi, Anton! > > On Mon, Dec 4, 2023 at 3:50 AM Anton A. Melnikov > wrote: >> >> Thanks for remarks! >> >> On 28.11.2023 21:34, Alexander Korotkov wrote: >> > After examining the second patch >> > ("v2-0001-Add-restartpoint-stats.patch

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2023-12-25 Thread Anton A. Melnikov
On 25.12.2023 02:38, Alexander Korotkov wrote: Pushed! Thanks a lot! With the best regards! -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2023-12-24 Thread Alexander Korotkov
On Sat, Dec 23, 2023 at 12:04 AM Alexander Korotkov wrote: > On Mon, Dec 4, 2023 at 3:50 AM Anton A. Melnikov > wrote: >> >> Thanks for remarks! >> >> On 28.11.2023 21:34, Alexander Korotkov wrote: >> > After examining the second patch >> > ("v2-0001-Add-restartpoint-stats.patch"), it appears th

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2023-12-22 Thread Alexander Korotkov
Hi, Anton! On Mon, Dec 4, 2023 at 3:50 AM Anton A. Melnikov wrote: > Thanks for remarks! > > On 28.11.2023 21:34, Alexander Korotkov wrote: > > After examining the second patch > > ("v2-0001-Add-restartpoint-stats.patch"), it appears that adding > > additional statistics as outlined in the patch

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2023-12-03 Thread Anton A. Melnikov
Thanks for remarks! On 28.11.2023 21:34, Alexander Korotkov wrote: After examining the second patch ("v2-0001-Add-restartpoint-stats.patch"), it appears that adding additional statistics as outlined in the patch is the most suitable approach to address the concerns raised. This solution provides

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2023-11-28 Thread Alexander Korotkov
Hi, Anton! On Thu, Mar 16, 2023 at 2:39 PM Anton A. Melnikov wrote: > On 15.03.2023 21:29, Gregory Stark (as CFM) wrote: > > > These patches that are "Needs Review" and have received no comments at > > all since before March 1st are these. If your patch is amongst this > > list I would suggest an

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2023-03-16 Thread Anton A. Melnikov
Hello! On 15.03.2023 21:29, Gregory Stark (as CFM) wrote: These patches that are "Needs Review" and have received no comments at all since before March 1st are these. If your patch is amongst this list I would suggest any of: 1) Move it yourself to the next CF (or withdraw it) 2) Post to the

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2022-12-07 Thread Anton A. Melnikov
Hello! On 06.12.2022 21:44, Andres Freund wrote: Hi, On 2022-09-19 01:29:21 +0300, Anton A. Melnikov wrote: Corrected patch is attached (v2-0001-Fix-burst-checkpoint_req-growth.patch). This patch doesn't pass the main regression tests tests successfully: https://cirrus-ci.com/task/550270001

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2022-12-06 Thread Andres Freund
Hi, On 2022-09-19 01:29:21 +0300, Anton A. Melnikov wrote: > Corrected patch is attached (v2-0001-Fix-burst-checkpoint_req-growth.patch). This patch doesn't pass the main regression tests tests successfully: https://cirrus-ci.com/task/5502700019253248 diff -U3 /tmp/cirrus-ci-build/src/test/regr

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2022-09-18 Thread Anton A. Melnikov
Hello! Thank you very much for your feedback and essential remarks. On 07.09.2022 10:39, Kyotaro Horiguchi wrote: It lets XLogPageRead run the same check with what CreateRestartPoint does, so it basically works (it is forgetting a lock, though. The reason for omitting the lock in CreateRestart

Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2022-09-07 Thread Kyotaro Horiguchi
At Tue, 6 Sep 2022 14:02:53 +0300, "Anton A. Melnikov" wrote in > Can we treat such behavior as a bug? Depends on how we see the counter value. I think this can be annoying but not a bug. CreateRestartPoint already handles that case. While standby is well catching up, startup may make requests

May be BUG. Periodic burst growth of the checkpoint_req counter on replica.

2022-09-06 Thread Anton A. Melnikov
Hello! Found a periodic spike growth of the checkpoint_req counter on replica by 20-30 units after large insert (~350Mb) on master. Reproduction on master and replica with default conf: 1) execute the command "insert into test values (generate_series(1,1E7));". This leads to the table's growth b