Re: Kafka Streams processor node metrics process rate with multiple stream threads

2018-07-17 Thread Guozhang Wang
Thanks Sam! Please feel free to assign the ticket to yourself and I will review your PR if you created one: https://cwiki.apache.org/confluence/display/KAFKA/Contributing+Code+Changes#ContributingCodeChanges-PullRequest On Tue, Jul 17, 2018 at 6:29 PM, Sam Lendle wrote: > https://issues.apache.

Re: Kafka Streams processor node metrics process rate with multiple stream threads

2018-07-17 Thread Sam Lendle
https://issues.apache.org/jira/browse/KAFKA-7176 If I have a change I will give trunk a try. On 7/16/18, 2:14 PM, "Guozhang Wang" wrote: Hmm.. this seems new to me. Checked on the source code it seems right to me. Could you try out the latest trunk (build from source code) and see

Re: Kafka Streams processor node metrics process rate with multiple stream threads

2018-07-16 Thread Guozhang Wang
Hmm.. this seems new to me. Checked on the source code it seems right to me. Could you try out the latest trunk (build from source code) and see if it is the same issue for you? > In addition to that, though, I also see state store metrics for tasks that have been migrated to another instance, an

Re: Kafka Streams processor node metrics process rate with multiple stream threads

2018-07-12 Thread Sam Lendle
Ah great, thanks Gouzhang. I also noticed a similar issue with state store metrics, where rate metrics for each thread/task appear to be the total rate across all threads/tasks on that instance. In addition to that, though, I also see state store metrics for tasks that have been migrated to an

Re: Kafka Streams processor node metrics process rate with multiple stream threads

2018-07-11 Thread Guozhang Wang
Hello Sam, It is a known issue that should have been fixed in 2.0, the correlated fix has also been cherry-picked to the 1.1.1 bug fix release as well: https://github.com/apache/kafka/pull/5277 Guozhang On Wed, Jul 11, 2018 at 11:42 AM, Sam Lendle wrote: > Hello! > > Using kafka-streams 1.1.

Kafka Streams processor node metrics process rate with multiple stream threads

2018-07-11 Thread Sam Lendle
Hello! Using kafka-streams 1.1.0, I noticed when I sum the process rate metric for a given processor node, the rate is many times higher than the number of incoming messages. Digging further, it looks like the rate metric associated with each thread in a given application instance is always the