Hi Guozhang, Thank you for the KIP.
1) As far as I understand, the StreamsMetrics interface is there for user-defined processors. Would it make sense to also add a method to the interface to specify a sensor that records skipped records? 2) What are the semantics of active-task-process and standby-task-process 3) How do dropped-late-records and expired-window-record-drop relate to each other? I guess the former is for records that fall outside the grace period and the latter is for records that are processed after the retention period of the window. Is this correct? 4) Is there an actual difference between skipped and dropped records? If not, shall we unify the terminology? 5) What happens with removed metrics when the user sets the version of "built.in.metrics.version" to 2.2- Best, Bruno On Thu, Jun 27, 2019 at 6:11 PM Guozhang Wang <wangg...@gmail.com> wrote: > > Hello folks, > > As 2.3 is released now, I'd like to bump up this KIP discussion again for > your reviews. > > > Guozhang > > > On Thu, May 23, 2019 at 4:44 PM Guozhang Wang <wangg...@gmail.com> wrote: > > > Hello Patrik, > > > > Since we are rolling out 2.3 and everyone is busy with the release now > > this KIP does not have much discussion involved yet and will slip into the > > next release cadence. > > > > This KIP itself contains several parts itself: 1. refactoring the existing > > metrics hierarchy to cleanup some redundancy and also get more clarity; 2. > > add instance-level metrics like rebalance and state metrics, as well as > > other static metrics. > > > > > > Guozhang > > > > > > > > On Thu, May 23, 2019 at 5:34 AM Patrik Kleindl <pklei...@gmail.com> wrote: > > > >> Hi Guozhang > >> Thanks for the KIP, this looks very helpful. > >> Could you please provide more detail on the metrics planned for the state? > >> We were just considering how to implement this ourselves because we need > >> to > >> track the history of stage changes. > >> The idea was to have an accumulated "seconds in state x" metric for every > >> state. > >> The new rebalance metric might solve part of our use case, but it is > >> interesting what you have planned for the state metric. > >> best regards > >> Patrik > >> > >> On Fri, 29 Mar 2019 at 18:56, Guozhang Wang <wangg...@gmail.com> wrote: > >> > >> > Hello folks, > >> > > >> > I'd like to propose the following KIP to improve the Kafka Streams > >> metrics > >> > mechanism to users. This includes 1) a minor change in the public > >> > StreamsMetrics API, and 2) a major cleanup on the Streams' own built-in > >> > metrics hierarchy. > >> > > >> > Details can be found here: > >> > > >> > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%3A+Augment+metrics+for+Kafka+Streams > >> > > >> > I'd love to hear your thoughts and feedbacks. Thanks! > >> > > >> > -- > >> > -- Guozhang > >> > > >> > > > > > > -- > > -- Guozhang > > > > > -- > -- Guozhang
