Hi Raghav, As you know, KIP-372 added "version" tag to RequestsPerSec metric to monitor requests for each version. As mentioned in the KIP, to get total count per request (across all versions), we need to aggregate over all versions. You may need to automate this by fetching all metrics using wildcard and aggregating over all versions. We may need to use JMX readers which support wildcard queries.
All possible protocol request versions are available here: https://kafka.apache.org/protocol.html#The_Messages_Produce On Sat, Mar 9, 2019 at 5:59 AM Raghav <raghavas...@gmail.com> wrote: > Hello Allen, Ted, Mani, Gwen, James > > https://www.mail-archive.com/dev@kafka.apache.org/msg86226.html > > I read this email thread and I see that you guys were split in going ahead > with this change. This has broken our dashboard in 2.1 Kafka, which is ok > as long as we know how to fix it. > > Can you please help us figure out the answer for the email below ? It will > be greatly appreciated. We just want to know how to find the version number > ? > > Many thanks. > > R > > On Fri, Dec 14, 2018 at 5:16 PM Raghav <raghavas...@gmail.com> wrote: > >> I got it to work. I fired up a console and then saw what beans are >> registered there and then queries using the code. It works then. >> >> But the apiVersion is different and how do we know what apiVersion a >> producer is using ? Our dashboards cannot have a-priori knowledge of the >> version numbers, and wildcards don't work. See the screenshot below, >> apiVersion is 7. Where did this come from ? Can someone please help to >> understand. >> >> [image: jmx.png] >> >> >> >> On Fri, Dec 14, 2018 at 4:29 PM Raghav <raghavas...@gmail.com> wrote: >> >>> Is this a test case for this commit: >>> https://github.com/apache/kafka/pull/4506 ? I have tried using all >>> possible cases but cannot get it to work. >>> >>> I cannot get it to work using mbean reader via JMX. Can any one please >>> help? Or atleast confirm that it is broken. Thanks >>> >>> R >>> >>> On Fri, Dec 14, 2018 at 6:34 AM Raghav <raghavas...@gmail.com> wrote: >>> >>>> Thanks Ismael. How to query it in 2.1 ? I tried all possible ways >>>> including using version, but I am still getting the same exception message. >>>> >>>> Thanks for your help. >>>> >>>> On Thu, Dec 13, 2018 at 7:19 PM Ismael Juma <isma...@gmail.com> wrote: >>>> >>>>> The metric was changed to include the API version. I believe this was >>>>> in >>>>> the upgrade notes for 2.0.0. >>>>> >>>>> Ismael >>>>> >>>>> On Thu, Dec 13, 2018, 3:35 PM Raghav <raghavas...@gmail.com wrote: >>>>> >>>>> > Hi >>>>> > >>>>> > We are trying to move from Kafka 1.1.0 to Kafka 2.1.0. We used to >>>>> monitor >>>>> > our 3 node Kafka using JMX. Upon moving to 2.1.0, we have observed >>>>> that the >>>>> > *below* mentioned metric can't be retrie >>>>> > and we get the below exception: >>>>> > >>>>> > >>>>> *"kafka.network:type=RequestMetrics,name=RequestsPerSec,request=Produce"* >>>>> > >>>>> > javax.management.InstanceNotFoundException: >>>>> > kafka.network:type=RequestMetrics,name=RequestsPerSec,request=Produce >>>>> > at >>>>> > >>>>> > >>>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095) >>>>> > at >>>>> > >>>>> > >>>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1375) >>>>> > at >>>>> > >>>>> > >>>>> com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:920) >>>>> > at >>>>> > >>>>> > >>>>> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1462) >>>>> > at >>>>> > >>>>> > >>>>> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) >>>>> > at >>>>> > >>>>> > >>>>> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309) >>>>> > at >>>>> > >>>>> > >>>>> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401) >>>>> > at >>>>> > >>>>> > >>>>> javax.management.remote.rmi.RMIConnectionImpl.getMBeanInfo(RMIConnectionImpl.java:905) >>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> > at >>>>> > >>>>> > >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>> > at >>>>> > >>>>> > >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>> > at java.lang.reflect.Method.invoke(Method.java:498) >>>>> > at >>>>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:324) >>>>> > at sun.rmi.transport.Transport$1.run(Transport.java:200) >>>>> > at sun.rmi.transport.Transport$1.run(Transport.java:197) >>>>> > at java.security.AccessController.doPrivileged(Native Method) >>>>> > at sun.rmi.transport.Transport.serviceCall(Transport.java:196) >>>>> > at >>>>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) >>>>> > at >>>>> > >>>>> > >>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) >>>>> > at >>>>> > >>>>> > >>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) >>>>> > at java.security.AccessController.doPrivileged(Native Method) >>>>> > at >>>>> > >>>>> > >>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) >>>>> > at >>>>> > >>>>> > >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>> > at >>>>> > >>>>> > >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>> > at java.lang.Thread.run(Thread.java:745) >>>>> > at >>>>> > >>>>> > >>>>> sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276) >>>>> > at >>>>> > >>>>> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253) >>>>> > at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162) >>>>> > at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source) >>>>> > at >>>>> javax.management.remote.rmi.RMIConnectionImpl_Stub.getMBeanInfo(Unknown >>>>> > Source) >>>>> > at >>>>> > >>>>> > >>>>> javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getMBeanInfo(RMIConnector.java:1079) >>>>> > >>>>> > >>>>> > We even tried adding version to it, but no avail. >>>>> > >>>>> > Can someone please guide us how to >>>>> > get >>>>> "kafka.network:type=RequestMetrics,name=RequestsPerSec,request=Produce" >>>>> > value ? >>>>> > >>>>> > R >>>>> > >>>>> >>>> >>>> >>>> -- >>>> Raghav >>>> >>> >>> >>> -- >>> Raghav >>> >> >> >> -- >> Raghav >> > > > -- > Raghav >
