Re: POC: track vacuum/analyze cumulative time per relation

2025-02-04 Thread Alena Rybakina
Hi! On 28.01.2025 03:59, Michael Paquier wrote: On Mon, Jan 27, 2025 at 11:22:16AM -0600, Sami Imseih wrote: I have put my hands on this patch, and at the end I think that we should just do the attached, which is simpler and addresses your use-case. Note also that the end time is acquired whil

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-27 Thread Michael Paquier
On Mon, Jan 27, 2025 at 11:22:16AM -0600, Sami Imseih wrote: >> I have put my hands on this patch, and at the end I think that we >> should just do the attached, which is simpler and addresses your >> use-case. Note also that the end time is acquired while the entries >> are not locked in the repo

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-27 Thread Sami Imseih
> I think that this is hiding a behavior change while adding new > counters, and both changes are independent. That's a fair point. > Another thing that is > slightly incorrect to do if we take the argument of only adding the > counters is moving around the call of pgstat_progress_end_command(),

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-27 Thread Bertrand Drouvot
Hi, On Mon, Jan 27, 2025 at 09:30:16AM +0900, Michael Paquier wrote: > The addition of the extra GetCurrentTimestamp() in the report path > does not stress me much, Reading at the previous messages I see how you reached this state. I also think that makes sense and that's not an issue as we are

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-26 Thread Michael Paquier
On Fri, Jan 24, 2025 at 02:19:59PM -0600, Sami Imseih wrote: > So currently, the report of the last_(autoanalyze|analyze)_time > is set before the index_vacuum_cleanup, but for logging purposes > the elapsed time is calculated afterwards. Most users will not notice > this, but I think that is wrong

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-24 Thread Sami Imseih
> I was referring to the order of the fields in the structure itself, > but that's no big deal one way or the other. I understand your point now. I will group them with the related counters in the next rev and will use > This should be one comment for the whole block, or this should use the > sin

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-23 Thread Michael Paquier
On Mon, Jan 20, 2025 at 11:04:40AM -0600, Sami Imseih wrote: >> + PgStat_Counter total_vacuum_time; >> + PgStat_Counter total_autovacuum_time; >> + PgStat_Counter total_analyze_time; >> + PgStat_Counter total_autoanalyze_time; >> } PgStat_StatTabEntry; >> These are time val

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-20 Thread Sami Imseih
> I can get behind the idea of the patch. Thanks for the review! > + PgStat_Counter total_vacuum_time; > + PgStat_Counter total_autovacuum_time; > + PgStat_Counter total_analyze_time; > + PgStat_Counter total_autoanalyze_time; > } PgStat_StatTabEntry; > These are time val

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-19 Thread Michael Paquier
On Fri, Jan 17, 2025 at 09:42:13AM -0600, Sami Imseih wrote: > Indeed. Corrected in v5 I can get behind the idea of the patch. + PgStat_Counter total_vacuum_time; + PgStat_Counter total_autovacuum_time; + PgStat_Counter total_analyze_time; + PgStat_Counter total_autoanalyz

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-17 Thread Sami Imseih
> One comment: > > + PG_RETURN_FLOAT8(result);\ > > The "\" indentation looks wrong for this line (was ok in v3). > > Regards, > Indeed. Corrected in v5 Thanks! Sami v5-0001-Track-per-relation-cumulative-time-spent-in-vacuu.patch Description: Binar

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-17 Thread Bertrand Drouvot
Hi, On Wed, Jan 15, 2025 at 11:09:00AM -0600, Sami Imseih wrote: > > Appart from the above that LGTM. > > thanks! I took care of your comments. Thanks! > I was not sure > about the need to cast "double" but as you mention this > is consistent with other parts of pgstatfuncs.c Yup. > v4 attach

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-15 Thread Sami Imseih
> Appart from the above that LGTM. thanks! I took care of your comments. I was not sure about the need to cast "double" but as you mention this is consistent with other parts of pgstatfuncs.c v4 attached. Regards, Sami v4-0001-Track-per-relation-cumulative-time-spent-in-vacuu.patch Descriptio

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-15 Thread Bertrand Drouvot
Hi, On Tue, Jan 14, 2025 at 05:01:52PM -0600, Sami Imseih wrote: > Please see the attached v3. Thanks! A few comments: === 1 + + total_vacuum_time bigint + Those new fields should be documented as "double precision". === 2 +#define PG_STAT_GET_RELENTRY_FLOAT8(stat)

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-14 Thread Sami Imseih
thanks for the review! > The comments do not reflect the function names ("total" is missing to give > pg_stat_get_total_vacuum_time() and such). fixed > I think it's better to define the macro just before its first usage (meaning > just after pg_stat_get_vacuum_count()): that would be consistent

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-13 Thread Bertrand Drouvot
Hi, On Fri, Jan 10, 2025 at 04:26:05PM -0600, Sami Imseih wrote: > { > /* time is already in msec, just convert to double for presentation */ > PG_RETURN_FLOAT8((double) > pgstat_fetch_stat_checkpointer()->write_time); > } > > > + > > + PgStat_Counter total_vacuum_time; /* user initia

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Sami Imseih
Thanks for the review! > === 1 > > + endtime = GetCurrentTimestamp(); > pgstat_report_vacuum(RelationGetRelid(rel), > rel->rd_rel->relisshared, > Max(vacrel->new_live_tuples, > 0), >

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Sami Imseih
> > this information gives a complete picture of vacuum efficiency, because > > analyzing only total_time statistics does not give a complete picture of > > what vacuum did: was it cleaning almost huge index, cleaning tables or > > just sleeping. > > The purpose of total_time is to be able to calcu

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Sami Imseih
> this information gives a complete picture of vacuum efficiency, because > analyzing only total_time statistics does not give a complete picture of > what vacuum did: was it cleaning almost huge index, cleaning tables or > just sleeping. The purpose of total_time is to be able to calculate the av

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Bertrand Drouvot
Hi, On Thu, Jan 02, 2025 at 12:24:06PM -0600, Sami Imseih wrote: > Having the total (auto)vacuum elapsed time > along side the existing (auto)vaccum_count > allows a user to track the average time an > operating overtime and to find vacuum tuning > opportunities. > > The same can also be said for

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Alena Rybakina
Hi! On 04.01.2025 06:41, wenhui qiu wrote: Hi Sami  Thank you for your path,it seems some path monitor vacuum status,Can we synthesize their good ideas together。 I've been working on a patch that collects vacuum statistics since May [0]. It includes heap and index relation vacuum statistics,

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-03 Thread wenhui qiu
Hi Sami Thank you for your path,it seems some path monitor vacuum status,Can we synthesize their good ideas together。 On Fri, 3 Jan 2025 at 02:24, Sami Imseih wrote: > Hi, > > After a recent question regarding tracking vacuum start_time in > pg_stat_all_tables [1], it dawned on me that this vie

POC: track vacuum/analyze cumulative time per relation

2025-01-02 Thread Sami Imseih
Hi, After a recent question regarding tracking vacuum start_time in pg_stat_all_tables [1], it dawned on me that this view is missing an important cumulative metric, which is how much time is spent performing vacuum per table. Currently, the only way a user can get this information is if they ena