Hi, I found an issue while implementing KIP-877 for server plugins.
Several server plugins can be instantiated multiple times. This can happen: - when a server is running in combined mode (for example Authorizer is instantiated by both brokers and controllers) - when using multiple listeners (for example KafkaPrincipalBuilder is instantiated per processor per listener) In the voted proposal, I did not include enough tags for some plugins to uniquely identify each instance and avoid metric name collisions. For that reason, I updated the KIP to specify extra tags (in addition of "config" and "class") for the following plugin types: - KafkaPrincipalBuilder - Authorizer - ClientQuotaCallback - ConfigProvider - AuthenticateCallbackHandler - Login - SslEngineFactory You can find the exact tags in the Proposed Changes section of the KIP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges Let me know if you have questions or concerns. Thanks, Mickael On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <mickael.mai...@gmail.com> wrote: > > Hi, > > While implementing the KIP I noticed the Kafka Connect specifics had > not been updated correctly based on the latest discussions. The > consensus was that connector and tasks should use their context to > retrieve the PluginMetrics instance if they want to register metrics. > While it's a slightly different mechanism than for the other plugins > (they need to implement Monitorable), this allows connector to support > this new feature and still work when deployed on an older Connect > runtime. > > I have updated the KIP and I have 2 PRs ready for review: > - Adding the APIs and support for producer and consumer plugins: > https://github.com/apache/kafka/pull/17511 > - Adding support for Connect plugins: > https://github.com/apache/kafka/pull/17804 > > Thanks, > Mickael > > > On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <mickael.mai...@gmail.com> > wrote: > > > > Hi, > > > > I'm closing the vote. The vote has passed with > > + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote > > from Hector. > > > > Thanks, > > Mickael > > > > On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <show...@gmail.com> wrote: > > > > > > Hi Mickael, > > > > > > Thanks for the KIP. > > > +1 (binding) from me. > > > > > > Thanks. > > > Luke > > > > > > On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tbent...@redhat.com> wrote: > > > > > > > +1 (binding). > > > > > > > > Thanks Mickael! > > > > > > > > On Thu, 1 Aug 2024 at 05:12, Mickael Maison <mickael.mai...@gmail.com> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > Bumping this thread to get some more votes and/or feedback. > > > > > > > > > > As I restarted the vote on June 10 after major changes, I'm only > > > > > counting votes since then. > > > > > So we have 1 binding (Chris) and 1 non-binding (Hector) votes. > > > > > > > > > > Thanks, > > > > > Mickael > > > > > > > > > > On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) > > > > > <hgerald...@bloomberg.net> wrote: > > > > > > > > > > > > This will help eliminate some boilerplate code we have for our > > > > > connectors. > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To: > > > > > dev@kafka.apache.org > > > > > > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to > > > > > register metrics > > > > > > > > > > > > Bumping this thread. > > > > > > > > > > > > Let me know if you have any feedback. > > > > > > > > > > > > Thanks, > > > > > > Mickael > > > > > > > > > > > > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton > > > > > > <fearthecel...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > +1 (binding), thanks Mickael! > > > > > > > > > > > > > > On Mon, Jun 10, 2024, 04:24 Mickael Maison > > > > > > > <mickael.mai...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > Following the feedback in the DISCUSS thread, I made significant > > > > > > > > changes to the proposal. So I'd like to restart a vote for > > > > > > > > KIP-877: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi > > > > > > ns+and+connectors+to+register+metrics > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Mickael > > > > > > > > > > > > > > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley > > > > > > > > <tbent...@redhat.com> > > > > > wrote: > > > > > > > > > > > > > > > > > > Hi Mickael, > > > > > > > > > > > > > > > > > > You'll have seen that I left some comments on the discussion > > > > > thread, but > > > > > > > > > they're minor enough that I'm happy to vote +1 here. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Tom > > > > > > > > > > > > > > > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison < > > > > > mickael.mai...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Bumping this thread since I've not seen any feedback. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Mickael > > > > > > > > > > > > > > > > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison > > > > > > > > > > <mickael.mai...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > I'd like to start a vote on KIP-877: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi > > > > > > ns+and+connectors+to+register+metrics > > > > > > > > > > > > > > > > > > > > > > Let me know if you have any feedback. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Mickael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
