On Fri, Aug 02, 2024 at 05:53:31AM +0900, Michael Paquier wrote:
> Attached is a rebased set of the rest, with 0001 now introducing the
> pluggable core part.
So, I have been able to spend a few more days on all that while
travelling across three continents, and I have applied the core patch
follo
On Sun, Jul 28, 2024 at 10:03:56PM +0200, Dmitry Dolgov wrote:
> So far I've got nothing against :)
I've looked again at the first patch of this series, and applied the
first one. Another last-minute edit I have done is to use more
consistently PgStat_Kind in the loops for the stats kinds across
> On Sun, Jul 28, 2024 at 10:20:45PM GMT, Michael Paquier wrote:
> I would like to apply this new infrastructure stuff and move on to the
> problems related to the scability of pg_stat_statements. So, are
> there any objections with all that?
So far I've got nothing against :)
On Sat, Jul 27, 2024 at 03:49:42PM +0200, Dmitry Dolgov wrote:
> Agree, looks good. I've tried to quickly sketch out such a fixed
> statistic for some another extension, everything was fine and pretty
> straightforward.
That's my hope. Thanks a lot for the feedback.
> One question, why don't you
On Tue, Jul 16, 2024 at 10:27:25AM +0900, Michael Paquier wrote:
> Perhaps we should have a few more inline functions like
> pgstat_get_entry_len() to retrieve the parts of the custom data in the
> snapshot and shmem control structures for fixed-numbered stats. That
> would limit what extensions n
On Fri, Jul 12, 2024 at 03:44:26PM +0200, Dmitry Dolgov wrote:
> I think it's fine. Although this solution feels a bit uncomfortable,
> after thinking back and forth I don't see any significantly better
> option. Worth noting that since the main goal is to maintain uniqueness,
> fixing the kind IDs
> On Thu, Jul 11, 2024 at 04:42:22PM GMT, Michael Paquier wrote:
>
> So we are down to the remaining parts of the patch, and this is going
> to need a consensus about a few things because this impacts the
> developer experience when implementing one's own custom stats:
> - Are folks OK with the poi
On Thu, Jul 11, 2024 at 01:29:08PM +, Bertrand Drouvot wrote:
> Please find attached a patch to do so (attached as .txt to not perturb the
> cfbot).
+ * Reads in existing statistics file into the shared stats hash (for non fixed
+ * amount stats) or into the fixed amount stats.
Thanks. I hav
Hi,
On Fri, Jul 05, 2024 at 09:35:19AM +0900, Michael Paquier wrote:
> On Thu, Jul 04, 2024 at 11:30:17AM +, Bertrand Drouvot wrote:
> >
> > /*
> > * Reads in existing statistics file into the shared stats hash.
> >
> > This comment above pgstat_read_statsfile() is not correct, fixed stats
On Wed, Jul 10, 2024 at 09:00:31AM +, Bertrand Drouvot wrote:
> On Wed, Jul 10, 2024 at 08:28:56AM +, Bertrand Drouvot wrote:
>> v5-0001 LGTM.
Thanks. I've applied this refactoring piece.
> /*
> * We found an existing statistics file. Read it and put all the hash
> * table
Hi,
On Wed, Jul 10, 2024 at 08:28:56AM +, Bertrand Drouvot wrote:
> Hi,
>
> On Tue, Jul 09, 2024 at 03:54:37PM +0900, Michael Paquier wrote:
> > On Mon, Jul 08, 2024 at 02:07:58PM +, Bertrand Drouvot wrote:
> > > It looks pretty straightforward, just one comment:
> > >
> > > +
Hi,
On Tue, Jul 09, 2024 at 03:54:37PM +0900, Michael Paquier wrote:
> On Mon, Jul 08, 2024 at 02:07:58PM +, Bertrand Drouvot wrote:
> > It looks pretty straightforward, just one comment:
> >
> > + ptr = ((char *) ctl) + kind_info->shared_ctl_off;
> > +
On Tue, Jul 09, 2024 at 05:23:03AM +, Bertrand Drouvot wrote:
> I gave a second thought on it, and I think that this is the "data" part that
> lead
> to the confusion (as too generic), what about?
>
> shared_data_len -> shared_stats_len
> shared_data_off -> shared_stats_off
>
> That looks ok
On Mon, Jul 08, 2024 at 02:07:58PM +, Bertrand Drouvot wrote:
> It looks pretty straightforward, just one comment:
>
> + ptr = ((char *) ctl) + kind_info->shared_ctl_off;
> + kind_info->init_shmem_cb((void *) ptr);
>
> I don't think we need to cast
Hi,
On Tue, Jul 09, 2024 at 10:45:05AM +0900, Michael Paquier wrote:
> On Mon, Jul 08, 2024 at 07:22:32AM +, Bertrand Drouvot wrote:
> > Yeah, what I meant to say is that one could think for example that's the
> > PgStatShared_Archiver size while in fact it's the PgStat_ArchiverStats size.
> >
On Mon, Jul 08, 2024 at 07:22:32AM +, Bertrand Drouvot wrote:
> Yeah, what I meant to say is that one could think for example that's the
> PgStatShared_Archiver size while in fact it's the PgStat_ArchiverStats size.
> I think it's more confusing when writing the stats. Here we are manipulating
Hi,
On Mon, Jul 08, 2024 at 07:22:32AM +, Bertrand Drouvot wrote:
> Except the above (which is just a Nit), 0001 LGTM.
>
Looking at 0002:
It looks pretty straightforward, just one comment:
+ ptr = ((char *) ctl) + kind_info->shared_ctl_off;
+ kin
Hi,
On Mon, Jul 08, 2024 at 03:49:34PM +0900, Michael Paquier wrote:
> On Mon, Jul 08, 2024 at 06:39:56AM +, Bertrand Drouvot wrote:
> > + for (int kind = PGSTAT_KIND_FIRST_VALID; kind <= PGSTAT_KIND_LAST;
> > kind++)
> > + {
> > + char *ptr;
> > +
On Mon, Jul 08, 2024 at 06:39:56AM +, Bertrand Drouvot wrote:
> + for (int kind = PGSTAT_KIND_FIRST_VALID; kind <= PGSTAT_KIND_LAST;
> kind++)
> + {
> + char *ptr;
> + const PgStat_KindInfo *info = pgstat_get_kind_info(kind);
>
> I wonder if we co
Hi,
On Mon, Jul 08, 2024 at 02:30:23PM +0900, Michael Paquier wrote:
> On Fri, Jul 05, 2024 at 09:35:19AM +0900, Michael Paquier wrote:
> > On Thu, Jul 04, 2024 at 11:30:17AM +, Bertrand Drouvot wrote:
> >> On Wed, Jul 03, 2024 at 06:47:15PM +0900, Michael Paquier wrote:
> >>> - PgStat_Snapsho
On Fri, Jul 05, 2024 at 09:35:19AM +0900, Michael Paquier wrote:
> On Thu, Jul 04, 2024 at 11:30:17AM +, Bertrand Drouvot wrote:
>> On Wed, Jul 03, 2024 at 06:47:15PM +0900, Michael Paquier wrote:
>>> - PgStat_Snapshot holds an array of void* also indexed by
>>> PGSTAT_NUM_KINDS, pointing to th
On Sun, Jul 07, 2024 at 12:21:26PM +0200, Dmitry Dolgov wrote:
> From what I understand, coordinating custom RmgrIds via a wiki page was
> made under the assumption that implementing a table AM with custom WAL
> requires significant efforts, which limits the demand for ids. This
> might not be same
> On Fri, Jun 21, 2024 at 01:28:11PM +0900, Michael Paquier wrote:
> On Fri, Jun 21, 2024 at 01:09:10PM +0900, Kyotaro Horiguchi wrote:
> > At Thu, 13 Jun 2024 16:59:50 +0900, Michael Paquier
> > wrote in
> >> * The kind IDs may change across restarts, meaning that any stats data
> >> associated
On Thu, Jul 04, 2024 at 11:30:17AM +, Bertrand Drouvot wrote:
> On Wed, Jul 03, 2024 at 06:47:15PM +0900, Michael Paquier wrote:
>> among the following lines:
>> - PgStat_ShmemControl holds an array of void* indexed by
>> PGSTAT_NUM_KINDS, pointing to shared memory areas allocated for each
>> f
On Thu, Jul 04, 2024 at 01:56:52PM -0700, Andres Freund wrote:
> On 2024-07-03 18:47:15 +0900, Michael Paquier wrote:
>> - PgStat_ShmemControl holds an array of void* indexed by
>> PGSTAT_NUM_KINDS, pointing to shared memory areas allocated for each
>> fixed-numbered stats. Each entry is allocated
On Thu, Jul 04, 2024 at 02:08:25PM -0700, Andres Freund wrote:
> I forgot to say: In general I am highly supportive of this effort and thankful
> to Michael for tackling it. The above was just about that one aspect.
Thanks. Let's discuss how people want this stuff to be shaped, and
how much we wa
On Thu, Jul 04, 2024 at 02:00:47PM -0700, Andres Freund wrote:
> On 2024-06-13 16:59:50 +0900, Michael Paquier wrote:
>> * Making custom stats data persistent is an interesting problem, and
>> there are a couple of approaches I've considered:
>> ** Allow custom kinds to define callbacks to read and
Hi,
On 2024-07-04 14:00:47 -0700, Andres Freund wrote:
> On 2024-06-13 16:59:50 +0900, Michael Paquier wrote:
> > * Making custom stats data persistent is an interesting problem, and
> > there are a couple of approaches I've considered:
> > ** Allow custom kinds to define callbacks to read and wri
Hi,
On 2024-06-13 16:59:50 +0900, Michael Paquier wrote:
> * Making custom stats data persistent is an interesting problem, and
> there are a couple of approaches I've considered:
> ** Allow custom kinds to define callbacks to read and write data from
> a source they'd want, like their own file th
Hi,
On 2024-07-03 18:47:15 +0900, Michael Paquier wrote:
> While looking at a different patch from Tristan in this area at [1], I
> still got annoyed that this patch set was not able to support the case
> of custom fixed-numbered stats, so as it is possible to plug in
> pgstats things similar to t
Hi,
On Wed, Jul 03, 2024 at 06:47:15PM +0900, Michael Paquier wrote:
> While looking at a different patch from Tristan in this area at [1], I
> still got annoyed that this patch set was not able to support the case
> of custom fixed-numbered stats, so as it is possible to plug in
> pgstats things
On Thu, Jul 04, 2024 at 10:11:02AM +0700, Andrei Lepikhov wrote:
> It is a feature my extensions (which usually change planning behaviour)
> definitely need. It is a problem to show the user if the extension does
> something or not because TPS smooths the execution time of a single query
> and perf
On 6/13/24 14:59, Michael Paquier wrote:
This will hopefully spark a discussion, and I was looking for answers
regarding these questions:
- Should the pgstat_kind_infos array in pgstat.c be refactored to use
something similar to pgstat_add_kind?
- How should the persistence of the custom stats be
On Fri, Jun 21, 2024 at 08:13:15AM +0900, Michael Paquier wrote:
> On Thu, Jun 20, 2024 at 02:27:14PM +, Bertrand Drouvot wrote:
>> On Thu, Jun 20, 2024 at 09:46:30AM +0900, Michael Paquier wrote:
>> I think it makes sense to follow the same "behavior" as the custom
>> wal resource managers. Th
On Fri, Jun 21, 2024 at 01:09:10PM +0900, Kyotaro Horiguchi wrote:
> At Thu, 13 Jun 2024 16:59:50 +0900, Michael Paquier
> wrote in
>> * The kind IDs may change across restarts, meaning that any stats data
>> associated to a custom kind is stored with the *name* of the custom
>> stats kind. De
At Thu, 13 Jun 2024 16:59:50 +0900, Michael Paquier wrote
in
> * The kind IDs may change across restarts, meaning that any stats data
> associated to a custom kind is stored with the *name* of the custom
> stats kind. Depending on the discussion happening here, I'd be open
> to use the same co
On Thu, Jun 20, 2024 at 02:27:14PM +, Bertrand Drouvot wrote:
> On Thu, Jun 20, 2024 at 09:46:30AM +0900, Michael Paquier wrote:
> I think it makes sense to follow the same "behavior" as the custom
> wal resource managers. That, indeed, looks much more simpler than v1.
Thanks for the feedback.
On Thu, Jun 20, 2024 at 01:05:42PM +, Bertrand Drouvot wrote:
> On Thu, Jun 13, 2024 at 04:59:50PM +0900, Michael Paquier wrote:
>> * Making custom stats data persistent is an interesting problem, and
>> there are a couple of approaches I've considered:
>> ** Allow custom kinds to define callba
Hi,
On Thu, Jun 20, 2024 at 09:46:30AM +0900, Michael Paquier wrote:
> On Thu, Jun 13, 2024 at 04:59:50PM +0900, Michael Paquier wrote:
> > - How should the persistence of the custom stats be achieved?
> > Callbacks to give custom stats kinds a way to write/read their data,
> > push everything int
Hi,
On Thu, Jun 13, 2024 at 04:59:50PM +0900, Michael Paquier wrote:
> Hi all,
>
> 2) Make the shmem pgstats pluggable so as it is possible for extensions
> to register their own stats kinds.
Thanks for the patch! I like the idea of having custom stats (it has also been
somehow mentioned in [1])
On Thu, Jun 13, 2024 at 04:59:50PM +0900, Michael Paquier wrote:
> - How should the persistence of the custom stats be achieved?
> Callbacks to give custom stats kinds a way to write/read their data,
> push everything into a single file, or support both?
> - Should this do like custom RMGRs and ass
41 matches
Mail list logo
