Re: Minimizing global store restoration time

Hi Taylor You are right, the parallel processing is not mentioned in this issue, if I remember correctly it was in the thread that lead to it as a possibility when changing to the restoration listeners. Best regards Patrik > Am 07.02.2019 um 00:47 schrieb Taylor P : > > Hi Patrik, > > I am no

Kafka Streams KGroupedTable.count() method returning negative values.

Hi All, One of my Kafka streams application is returning negative values for count() method. How is that possible? is there any known issue? I cannot think of any reason that this count could be negative. Is it possible if the state store is corrupted? idAndJobTransaction .filter((k,v)

Re: Kafka Streams KGroupedTable.count() method returning negative values.

Hi Ankur, Could you add an additional peek method logging what's going into the groupBy call and share the logs? Thanks, Bill On Fri, Feb 8, 2019 at 7:48 AM Ankur Rana wrote: > Hi All, > > One of my Kafka streams application is returning negative values for > count() method. > How is that poss

Re: Kafka Streams KGroupedTable.count() method returning negative values.

Hi Bill, I will try to make that change but since the negative values are really rare, It would be very difficult to capture the logs at that point. I am seeing this issue only in production. I will see what I can do. Also, the negative count is not limited to smaller values, but I have seen valu

Troubleshooting custom implementation for a org.apache.kafka.common.metrics.MetricsReporter

Hi I was checking out how to export metrics from a Kafka Steams App directly to the Prometheus Registry without using JMX, just by implementing a custom MetricsReporter. After some investigation, it became clear that a metric with the same name could be used by multiple entities, e.g. with differ

Re: Kafka Streams KGroupedTable.count() method returning negative values.

Hi Ankur, I understand. Let's see if we can narrow things down some without any logging. Where exactly are you seeing the negative number from the code above? Have you confirmed the count is negative by observing the results of the count().toStream() before the mapValues call? Thanks! Bill On

Re: Kafka streams exactly_once auto commit timeout transaction issue

Hello Xander, Upon committing the state with `exactly_once`, Streams will commit the transaction by going through the commit protocol (details can be found here [1]). So I guess the following happened in time: 1) one record gets read in. 2) processing the record by traversing the topology, not ye

[VOTE] 2.1.1 RC2

Hi all, This is the third candidate for release of Apache Kafka 2.1.1. This release includes many bug fixes for Apache Kafka 2.1. Compared to rc1, this release includes the following changes: * MINOR: release.py: fix some compatibility problems. * KAFKA-7897; Disable leader epoch cache when old

Re: [VOTE] 2.1.1 RC2

+1 Passes librdkafka test suite. Den fre 8 feb. 2019 kl 21:02 skrev Colin McCabe : > Hi all, > > This is the third candidate for release of Apache Kafka 2.1.1. This > release includes many bug fixes for Apache Kafka 2.1. > > Compared to rc1, this release includes the following changes: > * MINO

Re: Kafka Streams KGroupedTable.count() method returning negative values.

No, I do not have any logs before call to mapValues call but I will add them and let you know. Although I did see the negative values in the peek method above. Job Summary class is a very simple class and doesn't do any processing. [image: image.png] Here's the constructor for JobSummary class us