Re: Vacuum statistics

2025-06-02 Thread Alena Rybakina
On 02.06.2025 19:25, Alexander Korotkov wrote: On Tue, May 13, 2025 at 12:49 PM Alena Rybakina wrote: On 12.05.2025 08:30, Amit Kapila wrote: On Fri, May 9, 2025 at 5:34 PM Alena Rybakina wrote: I did a rebase and finished the part with storing statistics separately from the relation stat

Re: Vacuum statistics

2025-06-02 Thread Alexander Korotkov
On Tue, May 13, 2025 at 12:49 PM Alena Rybakina wrote: > On 12.05.2025 08:30, Amit Kapila wrote: > > On Fri, May 9, 2025 at 5:34 PM Alena Rybakina > > wrote: > >> I did a rebase and finished the part with storing statistics separately > >> from the relation statistics - now it is possible to di

Re: Vacuum statistics

2025-05-14 Thread Amit Kapila
On Tue, May 13, 2025 at 3:19 PM Alena Rybakina wrote: > > On 12.05.2025 08:30, Amit Kapila wrote: > > On Fri, May 9, 2025 at 5:34 PM Alena Rybakina > > wrote: > >> I did a rebase and finished the part with storing statistics separately > >> from the relation statistics - now it is possible to d

Re: Vacuum statistics

2025-05-13 Thread Alena Rybakina
Hi! On 12.05.2025 08:30, Amit Kapila wrote: On Fri, May 9, 2025 at 5:34 PM Alena Rybakina wrote: I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and thi

Re: Vacuum statistics

2025-05-12 Thread Amit Kapila
On Fri, May 9, 2025 at 5:34 PM Alena Rybakina wrote: > > I did a rebase and finished the part with storing statistics separately from > the relation statistics - now it is possible to disable the collection of > statistics for relationsh using gucs and > this allows us to solve the problem with

Re: Vacuum statistics

2025-05-09 Thread Alena Rybakina
Hi! On 22.04.2025 21:23, Andrei Lepikhov wrote: On 10/28/24 14:40, Alexander Korotkov wrote: On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina If I missed something or misunderstood, can you explain in more detail? Actually, I mean why do we need a possibility to return statistics for all table

Re: Vacuum statistics

2025-04-22 Thread Andrei Lepikhov
On 10/28/24 14:40, Alexander Korotkov wrote: On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina If I missed something or misunderstood, can you explain in more detail? Actually, I mean why do we need a possibility to return statistics for all tables/indexes in one function call? User anyway is su

Re: Vacuum statistics

2025-04-05 Thread Alena Rybakina
On 13.03.2025 09:42, Bertrand Drouvot wrote: Hi, On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote: The usecase I can see here is that we don't want autovac creating so much WAL traffic that it starts forcing other backends to have to write WAL out. But tracking how many times autovac w

Re: Vacuum statistics

2025-03-24 Thread Jim Nasby
On Fri, Mar 21, 2025 at 2:42 PM Alena Rybakina wrote: > On 13.03.2025 09:42, Bertrand Drouvot wrote: > > Hi, > > On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote: > > The usecase I can see here is that we don't want autovac creating so much > WAL traffic that it starts forcing other back

Re: Vacuum statistics

2025-03-21 Thread Alena Rybakina
Sorry, I forgot to provide a link to the problem [0], actually. So I provided it below. [0] https://www.postgresql.org/message-id/CAPpHfduoJEuoixPTTg2tjhnXqrdobuMaQGxriqxJ9TjN1uxOuA%40mail.gmail.com On 21.03.2025 22:42, Alena Rybakina wrote: On 13.03.2025 09:42, Bertrand Drouvot wrote: Hi,

Re: Vacuum statistics

2025-03-17 Thread Alena Rybakina
On 17.03.2025 09:42, vignesh C wrote: On Thu, 27 Feb 2025 at 23:30, Alena Rybakina wrote: Hi! On 17.02.2025 17:46, Alena Rybakina wrote: On 04.02.2025 18:22, Alena Rybakina wrote: Hi! Thank you for your review! On 02.02.2025 23:43, Alexander Korotkov wrote: On Mon, Jan 13, 2025 at 3:26 PM

Re: Vacuum statistics

2025-03-16 Thread vignesh C
On Thu, 27 Feb 2025 at 23:30, Alena Rybakina wrote: > > Hi! > On 17.02.2025 17:46, Alena Rybakina wrote: > > On 04.02.2025 18:22, Alena Rybakina wrote: > >> Hi! Thank you for your review! > >> > >> On 02.02.2025 23:43, Alexander Korotkov wrote: > >>> On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina

Re: Vacuum statistics

2025-03-15 Thread Kirill Reshke
On Thu, 27 Feb 2025 at 23:00, Alena Rybakina wrote: > > Hi! > On 17.02.2025 17:46, Alena Rybakina wrote: > > On 04.02.2025 18:22, Alena Rybakina wrote: > >> Hi! Thank you for your review! > >> > >> On 02.02.2025 23:43, Alexander Korotkov wrote: > >>> On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina

Re: Vacuum statistics

2025-03-12 Thread Bertrand Drouvot
Hi, On Wed, Mar 12, 2025 at 05:15:53PM -0500, Jim Nasby wrote: > The usecase I can see here is that we don't want autovac creating so much > WAL traffic that it starts forcing other backends to have to write WAL out. > But tracking how many times autovac writes WAL buffers won't help with that Ri

Re: Vacuum statistics

2025-03-12 Thread Jim Nasby
On Wed, Mar 12, 2025 at 2:41 PM Alena Rybakina wrote: > Hi! > > On 10.03.2025 12:13, Ilia Evdokimov wrote: > > Hi, > > > > After commit eaf5027 we should add information about wal_buffers_full. > > > > Any thoughts? > > > > -- > > Best regards, > > Ilia Evdokimov, > > Tantor Labs LLC. > > > I thi

Re: Vacuum statistics

2025-03-12 Thread Alena Rybakina
Hi! On 10.03.2025 12:13, Ilia Evdokimov wrote: Hi, After commit eaf5027 we should add information about wal_buffers_full. Any thoughts? -- Best regards, Ilia Evdokimov, Tantor Labs LLC. I think I can add it. To be honest, I haven't gotten to know this statistic yet, haven't had time to get

Re: Vacuum statistics

2025-03-12 Thread Alena Rybakina
Hi! On 10.03.2025 16:33, Kirill Reshke wrote: On Thu, 27 Feb 2025 at 23:00, Alena Rybakina wrote: Hi! On 17.02.2025 17:46, Alena Rybakina wrote: On 04.02.2025 18:22, Alena Rybakina wrote: Hi! Thank you for your review! On 02.02.2025 23:43, Alexander Korotkov wrote: On Mon, Jan 13, 2025 at

Re: Vacuum statistics

2025-03-10 Thread Ilia Evdokimov
Hi, After commit eaf5027 we should add information about wal_buffers_full. Any thoughts? -- Best regards, Ilia Evdokimov, Tantor Labs LLC.

Re: Vacuum statistics

2025-02-05 Thread Alena Rybakina
On 05.02.2025 09:59, Alexander Korotkov wrote: What is the point for disabling pgstat_track_vacuum_statistics then? I don't see it saves any valuable resources. The original point by Masahiko Sawada was growth of data structures in times [1] (and corresponding memory consumption especially with

Re: Vacuum statistics

2025-02-04 Thread Alexander Korotkov
On Tue, Feb 4, 2025 at 5:22 PM Alena Rybakina wrote: > > Hi! Thank you for your review! > > On 02.02.2025 23:43, Alexander Korotkov wrote: > > On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina > > wrote: > >> I noticed that the cfbot is bad, the reason seems to be related to the > >> lack of a para

Re: Vacuum statistics

2025-02-04 Thread Alena Rybakina
On 04.02.2025 18:22, Alena Rybakina wrote:   Also, should pgstat_track_vacuum_statistics also affect per database statistics? According to my original idea, I thought that we could collect extended statistics on relationships depending on whether the hook is enabled, and always on databases

Re: Vacuum statistics

2025-02-02 Thread Alexander Korotkov
On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina wrote: > I noticed that the cfbot is bad, the reason seems to be related to the lack > of a parameter in src/backend/utils/misc/postgresql.conf.sample. I added it, > it should help. The patch doesn't apply cleanly. Please rebase. I see you introd

Re: Vacuum statistics

2025-01-13 Thread Alena Rybakina
On 10.01.2025 17:51, Alena Rybakina wrote: Hi! I thought about this problem again and I think I have a solution. On 02.12.2024 23:12, Alena Rybakina wrote: * I've read the previous discussion on how important to keep all these fields regarding vacuum statistics including points by Andrei and J

Re: Vacuum statistics

2025-01-13 Thread Andrei Zubkov
Hi Sami, Thank you for your attention to our patch and for your own work. On Sun, 2025-01-05 at 20:00 -0600, Sami Imseih wrote: > > You make valid points. I now think because track_vacuum_statistics is > optional, we should track total_time in 2 places. First place in the > new > view being prop

Re: Vacuum statistics

2025-01-10 Thread Alena Rybakina
Sorry, I made a typo due to lack of sleep, I've marked below where exactly just in case. On 10.01.2025 15:04, Alena Rybakina wrote: Hi, I have updated the patch. Fix minor mistakes in the document, added the wraparound_failsafe_count statistics - it accounts the number of times when the vacu

Re: Vacuum statistics

2025-01-10 Thread Alena Rybakina
Hi! I thought about this problem again and I think I have a solution. On 02.12.2024 23:12, Alena Rybakina wrote: * I've read the previous discussion on how important to keep all these fields regarding vacuum statistics including points by Andrei and Jim. It still worrying me that statistics volu

Re: Vacuum statistics

2025-01-05 Thread Sami Imseih
> I am yet to look very closely, but I think some additional columns that > will be useful is the number of failsafe autovacuums occurred. > > Do you mean when the autovacuum started to prevent workaround? > Specifically vacuum_failsafe_age [1] when autovacuum automatically performs a vacuum witho

Re: Vacuum statistics

2025-01-04 Thread Alena Rybakina
Hi, thank you for your attention to this patch. On 02.01.2025 23:12, Sami Imseih wrote: Hi, Thanks for the work you have done here. Exposing cumulative metrics at this level of detail for vacuum is surely useful to find vacuum bottlenecks and to determine the effectiveness of vacuum tuning. Yes

Re: Vacuum statistics

2025-01-03 Thread Sami Imseih
> I guess one question is how realistic it is to try and put everything about > (auto)vacuum in a single view. > Given the complexity, the answer to that might just be “no”. In that case > leaving existing fields in pg_stat_all_tables > is a lot more reasonable. Agree. I also think the total_tim

Re: Vacuum statistics

2025-01-03 Thread Jim Nasby
On Jan 2, 2025, at 4:33 PM, Sami Imseih wrote: > >> While backwards compatibility is important, there’s definitely precedent for >> changing >> what shows up in the catalog. IMHO it’s better to bite the bullet and move >> those fields >> instead of having vacuum stats spread across two differen

Re: Vacuum statistics

2025-01-03 Thread Greg Sabino Mullane
> > While backwards compatibility is important, there’s definitely precedent > for changing what shows up in the catalog. IMHO it’s better to bite the > bullet and move those fields instead of having vacuum stats spread across > two different views. > -1. That's a huge change, and pg_stat_all_tabl

Re: Vacuum statistics

2025-01-02 Thread Sami Imseih
> While backwards compatibility is important, there’s definitely precedent for > changing > what shows up in the catalog. IMHO it’s better to bite the bullet and move > those fields > instead of having vacuum stats spread across two different views. Correct, the most recent one that I could thin

Re: Vacuum statistics

2025-01-02 Thread Jim Nasby
> On Jan 2, 2025, at 2:12 PM, Sami Imseih wrote: > > Alternatively, we can remove the vacuum related stats from pg_stat_all_tables, > but that will break monitoring tools and will leave us with the (auto)analyze > metrics alone in pg_stat_all_tables. This sounds very ugly. While backwards compa

Re: Vacuum statistics

2025-01-02 Thread Sami Imseih
Hi, Thanks for the work you have done here. Exposing cumulative metrics at this level of detail for vacuum is surely useful to find vacuum bottlenecks and to determine the effectiveness of vacuum tuning. I am yet to look very closely, but I think some additional columns that will be useful is the

Re: Vacuum statistics

2024-12-19 Thread Alena Rybakina
Hi! On 02.12.2024 17:46, Ilia Evdokimov wrote: In my opinion, the patches are semantically correct. However, not all dead code has been removed - I'm referring to pgstat_update_snapshot(). Also, the tests need to be fixed. I fixed it [0]. Thank you! [0] https://www.postgresql.org/message-

Re: Vacuum statistics

2024-12-02 Thread Alena Rybakina
On 02.12.2024 17:46, Ilia Evdokimov wrote: In my opinion, the patches are semantically correct. However, not all dead code has been removed - I'm referring to pgstat_update_snapshot(). Also, the tests need to be fixed. Thank you, I'll fix it -- Regards, Alena Rybakina Postgres Professional

Re: Vacuum statistics

2024-12-02 Thread Alena Rybakina
Hi! Thank you for your review! On 30.11.2024 07:48, Kirill Reshke wrote: Hello! After a brief glance, I think this patch set is good. But there isn't any more time in the current CF to commit this :(. So I moved to the next CF. I agree with you. Thank you!) I also like the 0001 commit message.

Re: Vacuum statistics

2024-12-02 Thread Alena Rybakina
On 02.12.2024 11:27, Alexander Korotkov wrote: Hi, Alena! On Wed, Nov 13, 2024 at 6:21 PM Alena Rybakina wrote: Updated 0001-v13 attached, as well as the diff between v12 and v13. Thank you) And I agree with your changes. And included them in patches. Thank you for the updated patchset. S

Re: Vacuum statistics

2024-12-02 Thread Ilia Evdokimov
In my opinion, the patches are semantically correct. However, not all dead code has been removed - I'm referring to pgstat_update_snapshot(). Also, the tests need to be fixed. -- Best regards, Ilia Evdokimov, Tantor Labs LLC.

Re: Vacuum statistics

2024-12-02 Thread Alexander Korotkov
Hi, Alena! On Wed, Nov 13, 2024 at 6:21 PM Alena Rybakina wrote: > Updated 0001-v13 attached, as well as the diff between v12 and v13. > > Thank you) > > And I agree with your changes. And included them in patches. Thank you for the updated patchset. Some points from me. * I've read the previo

Re: Vacuum statistics

2024-11-29 Thread Kirill Reshke
On Wed, 13 Nov 2024 at 21:21, Alena Rybakina wrote: > > Hi! Thank you for your contribution to this thread! > > On 13.11.2024 03:24, Jim Nasby wrote: > > On Nov 10, 2024, at 2:09 PM, Alena Rybakina wrote: > > > On 08.11.2024 22:34, Jim Nasby wrote: > > > On Nov 2, 2024, at 7:22 AM, Alena Rybakina

Re: Vacuum statistics

2024-11-12 Thread Jim Nasby
On Nov 10, 2024, at 2:09 PM, Alena Rybakina wrote: On 08.11.2024 22:34, Jim Nasby wrote: On Nov 2, 2024, at 7:22 AM, Alena Rybakina wrote:

Re: Vacuum statistics

2024-11-10 Thread Ilia Evdokimov
On 08.11.2024 22:23, Alena Rybakina wrote: Hi! Thank you for review! On 07.11.2024 17:49, Ilia Evdokimov wrote: Thank you for fixing it. 1) I have found some typos in the test output files (out-files) when running 'make check' and 'make check-world'. These typos might cause minor discrepan

Re: Vacuum statistics

2024-11-08 Thread Jim Nasby
> On Nov 2, 2024, at 7:22 AM, Alena Rybakina wrote: > >>> The second is the interrupts field. It is needed for monitoring to know >>> do we have them or not, so tracking them on the database level will do >>> the trick. Interrupt is quite rare event, so once the monitoring system >>> will catch

Re: Vacuum statistics

2024-11-08 Thread Alena Rybakina
Hi! Thank you for review! On 07.11.2024 17:49, Ilia Evdokimov wrote: Thank you for fixing it. 1) I have found some typos in the test output files (out-files) when running 'make check' and 'make check-world'. These typos might cause minor discrepancies in test results. You may already be awar

Re: Vacuum statistics

2024-11-07 Thread Ilia Evdokimov
On 22.10.2024 22:30, Alena Rybakina wrote: Hi! On 16.10.2024 14:01, Alena Rybakina wrote: Thank you for rebasing. I have noticed that when I create a table or an index on this table, there is no information about the table or index in pg_stat_vacuum_tables and pg_stat_vacuum_indexes until

Re: Vacuum statistics

2024-11-02 Thread Alena Rybakina
Hi! On 29.10.2024 14:02, Alena Rybakina wrote: On 28.10.2024 16:40, Alexander Korotkov wrote: On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina wrote: I didn't understand correctly - did you mean that we don't need SRF if we need to display statistics for a specific object? Otherwise, we need

Re: Vacuum statistics

2024-10-29 Thread Jim Nasby
On Oct 29, 2024, at 7:40 AM, Andrei Zubkov wrote: > > Hi, > > Thanks for your attention to our patch! > > On Mon, 2024-10-28 at 16:03 -0500, Jim Nasby wrote: >>> Yes, but as Masahiko-san pointed out, PgStat_TableCounts is almost >>> tripled in space. That a huge change from having no statistic

Re: Vacuum statistics

2024-10-29 Thread Andrei Zubkov
Hi, Thanks for your attention to our patch! On Mon, 2024-10-28 at 16:03 -0500, Jim Nasby wrote: > > Yes, but as Masahiko-san pointed out, PgStat_TableCounts is almost > > tripled in space.  That a huge change from having no statistics on > > vacuum to have it in much more detail than everything e

Re: Vacuum statistics

2024-10-29 Thread Alena Rybakina
On 28.10.2024 16:40, Alexander Korotkov wrote: On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina wrote: I didn't understand correctly - did you mean that we don't need SRF if we need to display statistics for a specific object? Otherwise, we need this when we display information on all database

Re: Vacuum statistics

2024-10-28 Thread Jim Nasby
> On Oct 28, 2024, at 2:07 PM, Alexander Korotkov wrote: > >> I suppose that if we turn off statistics collection for a certain object, we >> can miss it. In addition, the user may not enable the parameter for the >> object in time, because he will forget about it. > > I agree with this poin

Re: Vacuum statistics

2024-10-28 Thread Alexander Korotkov
On Sun, Sep 29, 2024 at 12:22 AM Alena Rybakina wrote: > Hi! Thank you for your interesting for this patch! > > I took a very brief look at this and was wondering if it was worth > having a way to make the per-table vacuum statistics opt-in (like a > table storage parameter) in order to decrease t

Re: Vacuum statistics

2024-10-28 Thread Alexander Korotkov
On Sun, Aug 25, 2024 at 6:59 PM Alena Rybakina wrote: > I didn't understand correctly - did you mean that we don't need SRF if > we need to display statistics for a specific object? > > Otherwise, we need this when we display information on all database > objects (tables or indexes): > > while ((e

Re: Vacuum statistics

2024-10-16 Thread Andrei Zubkov
Hi Ilia, On Wed, 2024-10-16 at 13:31 +0300, Ilia Evdokimov wrote: > BTW, I recommend renaming the view pg_stat_vacuum_database to > pg_stat_vacuum_databaseS  for consistency with pg_stat_vacuum_tables > and pg_stat_vacuum_indexes Such renaming doesn't seems correct to me because pg_stat_vacuum_da

Re: Vacuum statistics

2024-10-16 Thread Alena Rybakina
Hi! On 16.10.2024 13:31, Ilia Evdokimov wrote: On 08.10.2024 19:18, Alena Rybakina wrote: Made a rebase on a fresh master branch. -- Regards, Alena Rybakina Postgres Professional Thank you for rebasing. I have noticed that when I create a table or an index on this table, there is no infor

Re: Vacuum statistics

2024-10-16 Thread Ilia Evdokimov
On 08.10.2024 19:18, Alena Rybakina wrote: Made a rebase on a fresh master branch. -- Regards, Alena Rybakina Postgres Professional Thank you for rebasing. I have noticed that when I create a table or an index on this table, there is no information about the table or index in pg_stat_vacuum

Re: Vacuum statistics

2024-09-28 Thread Alena Rybakina
Hi! Thank you for your interesting for this patch! I took a very brief look at this and was wondering if it was worth having a way to make the per-table vacuum statistics opt-in (like a table storage parameter) in order to decrease the shared memory footprint of storing the stats. I'm not sure h

Re: Vacuum statistics

2024-09-27 Thread Masahiko Sawada
On Fri, Sep 27, 2024 at 12:19 PM Melanie Plageman wrote: > > On Fri, Sep 27, 2024 at 2:16 PM Masahiko Sawada wrote: > > > > Hi, > > > > On Thu, Sep 5, 2024 at 2:01 PM Alena Rybakina > > wrote: > > > > > > Hi! Thank you for your review! > > > > > > On 05.09.2024 15:47, jian he wrote: > > > > > >

Re: Vacuum statistics

2024-09-27 Thread Andrei Zubkov
Hi, On Fri, 2024-09-27 at 11:15 -0700, Masahiko Sawada wrote: > I'm concerned that a pg_stat_vacuum_tables view has some duplicated > statistics that we already collect in different ways. For instance, > total_blks_{read,hit,dirtied,written} are already tracked at > system-level by pg_stat_io, pg

Re: Vacuum statistics

2024-09-27 Thread Melanie Plageman
On Fri, Sep 27, 2024 at 2:16 PM Masahiko Sawada wrote: > > Hi, > > On Thu, Sep 5, 2024 at 2:01 PM Alena Rybakina > wrote: > > > > Hi! Thank you for your review! > > > > On 05.09.2024 15:47, jian he wrote: > > > > On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina > > wrote: > > > > Hi, all! > > > >

Re: Vacuum statistics

2024-09-27 Thread Masahiko Sawada
Hi, On Thu, Sep 5, 2024 at 2:01 PM Alena Rybakina wrote: > > Hi! Thank you for your review! > > On 05.09.2024 15:47, jian he wrote: > > On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina > wrote: > > Hi, all! > > I have attached the new version of the code and the diff files > (minor-vacuum.no-cbot)

Re: Vacuum statistics

2024-09-05 Thread jian he
On Thu, Sep 5, 2024 at 1:23 AM Alena Rybakina wrote: > > Hi, all! > > I have attached the new version of the code and the diff files > (minor-vacuum.no-cbot). > hi. still have white space issue when using "git apply", you may need to use "git diff --check" to find out where. /* -- dif

Re: Vacuum statistics

2024-08-26 Thread Alena Rybakina
Just in case, I have attached a diff file to show the changes for the latest version attached here [0] to make the review process easier. [0] https://www.postgresql.org/message-id/c4e4e305-7119-4183-b49a-d7092f4efba3%40postgrespro.ru -- Regards, Alena Rybakina Postgres Professional: http://ww

Re: Vacuum statistics

2024-08-25 Thread Alena Rybakina
On 22.08.2024 07:29, Kirill Reshke wrote: On Thu, 22 Aug 2024 at 07:48, jian he wrote: On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina wrote: We check it there: "tabentry->vacuum_ext.type != type". Or were you talking about something else? On 19.08.2024 12:32, jian he wrote: in pg_stats_va

Re: Vacuum statistics

2024-08-25 Thread Alena Rybakina
On 22.08.2024 05:47, jian he wrote: On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina wrote: We check it there: "tabentry->vacuum_ext.type != type". Or were you talking about something else? On 19.08.2024 12:32, jian he wrote: in pg_stats_vacuum if (type == PGSTAT_EXTVAC_INDEX || type ==

Re: Vacuum statistics

2024-08-22 Thread Alexander Korotkov
On Wed, Aug 21, 2024 at 1:39 AM Alena Rybakina wrote: > > I think you've counted the above system tables from the database, but > I'll double-check it. Thank you for your review! > > On 19.08.2024 19:28, Ilia Evdokimov wrote: > > Are you certain that all tables are included in > > `pg_stat_vacuum_

Re: Vacuum statistics

2024-08-21 Thread Kirill Reshke
On Thu, 22 Aug 2024 at 07:48, jian he wrote: > > On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina > wrote: > > > > We check it there: "tabentry->vacuum_ext.type != type". Or were you talking > > about something else? > > > > On 19.08.2024 12:32, jian he wrote: > > > > in pg_stats_vacuum > > if

Re: Vacuum statistics

2024-08-21 Thread jian he
On Wed, Aug 21, 2024 at 6:37 AM Alena Rybakina wrote: > > We check it there: "tabentry->vacuum_ext.type != type". Or were you talking > about something else? > > On 19.08.2024 12:32, jian he wrote: > > in pg_stats_vacuum > if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP) > {

Re: Vacuum statistics

2024-08-20 Thread Alena Rybakina
I think you've counted the above system tables from the database, but I'll double-check it. Thank you for your review! On 19.08.2024 19:28, Ilia Evdokimov wrote: Are you certain that all tables are included in `pg_stat_vacuum_tables`? I'm asking because of the following: SELECT count(*) FROM

Re: Vacuum statistics

2024-08-20 Thread Alena Rybakina
We check it there: "tabentry->vacuum_ext.type != type". Or were you talking about something else? On 19.08.2024 12:32, jian he wrote: in pg_stats_vacuum if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP) { Oidrelid = PG_GETARG_OID(1);

Re: Vacuum statistics

2024-08-20 Thread Alena Rybakina
Hi! Thank you very much for your review! Sorry for my late response I was overwhelmed by tasks. On 16.08.2024 14:12, jian he wrote: On Thu, Aug 15, 2024 at 4:49 PM Alena Rybakina wrote: Hi! I've applied all the v5 patches. 0002 and 0003 have white space errors. + +Number of

Re: Vacuum statistics

2024-08-19 Thread Ilia Evdokimov
Are you certain that all tables are included in `pg_stat_vacuum_tables`? I'm asking because of the following: SELECT count(*) FROM pg_stat_all_tables ;  count ---    108 (1 row) SELECT count(*) FROM pg_stat_vacuum_tables ;  count ---     20 (1 row) -- Regards, Ilia Evdokimov, Tantor L

Re: Vacuum statistics

2024-08-19 Thread jian he
in pg_stats_vacuum if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP) { Oidrelid = PG_GETARG_OID(1); /* Load table statistics for specified database. */ if (OidIsValid(relid)) { tabentry = fetch_dbstat_tabentry(dbi

Re: Vacuum statistics

2024-08-16 Thread jian he
On Thu, Aug 15, 2024 at 4:49 PM Alena Rybakina wrote: > > Hi! I've applied all the v5 patches. 0002 and 0003 have white space errors. + +Number of times blocks of this index were already found +in the buffer cache by vacuum operations, so that a read was not necessary +

Re: Vacuum statistics

2024-08-15 Thread Ilia Evdokimov
On 15.8.24 11:49, Alena Rybakina wrote: I have some suggestions: 1. pgstatfuncs.c in functions tuplestore_put_for_database() and tuplestore_put_for_relation you can remove 'nulls' array if you're sure that columns cannot be NULL. We need to use this for tuplestore_putvalues function.

Re: Vacuum statistics

2024-08-13 Thread Ilia Evdokimov
And I have one suggestion for pg_stat_vacuum_database: I suppose we should add database's name column after 'dboid' column because it is difficult to read statistics without database's name. We could call it 'datname' just like in 'pg_stat_database' view. Regards, Ilia Evdokimov, Tantor Labs

Re: Vacuum statistics

2024-08-13 Thread Ilia Evdokimov
On 10.8.24 22:37, Andrei Zubkov wrote: Hi, Ilia! Do you consider not to create new table in pg_catalog but to save statistics in existing table? I mean pg_class or pg_stat_progress_analyze, pg_stat_progress_vacuum? Thank you for your interest on our patch! *_progress views is not our case.

Re: Vacuum statistics

2024-08-10 Thread Kirill Reshke
On Wed, 12 Jun 2024 at 11:38, Alena Rybakina wrote: > > Hi! > > On 11.06.2024 16:09, Alena Rybakina wrote: > > On 08.06.2024 09:30, Alena Rybakina wrote: > > I am currently working on dividing this patch into three parts to simplify > the review process: one of them will contain code for collecti

Re: Vacuum statistics

2024-08-10 Thread Andrei Zubkov
Hi, Ilia! > Do you consider not to create new table in pg_catalog but to save > statistics in existing table? I mean pg_class or > pg_stat_progress_analyze, pg_stat_progress_vacuum? > Thank you for your interest on our patch! *_progress views is not our case. They hold online statistics whil

Re: Vacuum statistics

2024-08-10 Thread Ilia Evdokimov
Hello, everyone! Thank you for your interesting patch with extended information statistics about autovacuum. Do you consider not to create new table in pg_catalog but to save statistics in existing table? I mean pg_class or pg_stat_progress_analyze, pg_stat_progress_vacuum? P.S. If I sent

Re: Vacuum statistics

2024-06-27 Thread Andrei Zubkov
Hello! On Thu, 27/06/2024 at 10:39 +0900, Masahiko Sawada: > On Fri, May 31, 2024 at 4:19 AM Andrei Zubkov > wrote: > > As the vacuum process is a backend it has a workload > > instrumentation. > > We have all the basic counters available such as a number of blocks > > read, hit and written, time

Re: Vacuum statistics

2024-06-26 Thread Masahiko Sawada
On Fri, May 31, 2024 at 4:19 AM Andrei Zubkov wrote: > > Hi, > > Th, 30/05/2024 at 10:33 -0700, Alena Rybakina wrote: > > I suggest gathering information about vacuum resource consumption for > > processing indexes and tables and storing it in the table and index > > relationships (for example, Pg

Re: Vacuum statistics

2024-06-17 Thread Alena Rybakina
I have written the documentary and attached the patch. On 08.06.2024 09:30, Alena Rybakina wrote: Iam currentlyworkingondividingthispatchintothreepartstosimplifythe reviewprocess:oneofthemwillcontaincodeforcollectingvacuumstatisticsontables,the secondonindexesandthe lastondatabases.I alsowrit

Re: Vacuum statistics

2024-06-11 Thread Alena Rybakina
Hi! On 11.06.2024 16:09, Alena Rybakina wrote: On 08.06.2024 09:30, Alena Rybakina wrote: Iam currentlyworkingondividingthispatchintothreepartstosimplifythe reviewprocess:oneofthemwillcontaincodeforcollectingvacuumstatisticsontables,the secondonindexesandthe lastondatabases. I have divided

Re: Vacuum statistics

2024-06-06 Thread Dilip Kumar
On Thu, May 30, 2024 at 11:57 PM Alena Rybakina wrote: > > On 30.05.2024 10:33, Alena Rybakina wrote: > > > > I suggest gathering information about vacuum resource consumption for > > processing indexes and tables and storing it in the table and index > > relationships (for example, PgStat_StatTab

Re: Vacuum statistics

2024-05-30 Thread Andrei Zubkov
Hi, Th, 30/05/2024 at 10:33 -0700, Alena Rybakina wrote: > I suggest gathering information about vacuum resource consumption for > processing indexes and tables and storing it in the table and index > relationships (for example, PgStat_StatTabEntry structure like it has > realized for usual stati

Re: Vacuum statistics

2024-05-30 Thread Alena Rybakina
On 30.05.2024 10:33, Alena Rybakina wrote: I suggest gathering information about vacuum resource consumption for processing indexes and tables and storing it in the table and index relationships (for example, PgStat_StatTabEntry structure like it has realized for usual statistics). It will al