Re: Kafka Partition Leader -1

2019-11-10 Thread SenthilKumar K
Thanks. Keeping Broker iD 0 has caused some problems for me in the past ==> Our brokers start with id 1. Refer the attached image. you mentioned you “Fixed the broker” in the past. What exactly did you fix? - broker restart helped. On Fri, Nov 8, 2019 at 5:08 AM M. Manna wrote: > Hi, > > O

Re: Subject: [VOTE] 2.2.2 RC2

2019-11-10 Thread Jakub Scholz
+1 (non-binding) ... I used the prepared artifacts (Scala 2.12) and run my tests against them. All seems to be fine. On Sun, Nov 10, 2019 at 2:10 AM Matthias J. Sax wrote: > Thanks for running the release Randall! > > - verified all signatures > - build from source > - ran unit/integration tests

AW: Consumer Lags and receive no records anymore

2019-11-10 Thread Oliver Eckle
Hi, just still having some questions. Are you talking about lagging consumer, or about a consumer that completely stops to get new records? Because my issue is about the stopping of getting new records - which results in some lagging. Its not about the lagging itself - just to make shure we talk

AW: [External] AW: Consumer Lags and receive no records anymore

2019-11-10 Thread Oliver Eckle
Hi Dave, thank you . saw some tutorial where they told it otherwise .. which confuses me a litte. If its done round-robin .. my "world view" makes sense again 😊 Oliver -Ursprüngliche Nachricht- Von: Tauzell, Dave Gesendet: Freitag, 8. November 2019 16:18 An: users@kafka.apache.org B

Re: kafka stream ktable with suppress operator

2019-11-10 Thread Matthias J. Sax
Sorry for the late reply. > You mentioned that, reduce() could use RocksDB as stores by default > while suppress() is in memory. Is that the reason that reduce() has > both -repartition and -changelog topics while suppress() only has > -changelog topic? No. The repartitioning would happen if you

Re: kafka streams replication.factor=3 triples the count of tasks for internal topics

2019-11-10 Thread Matthias J. Sax
The replication factor should not change the number of create tasks. Those only depend on the number of partitions (and your topology). If you have two repartition topics I assume that your topology has 3 sub-topologies. Hence, 2 repartition topic with 68 partitons each (and I assume an input top

Re: Message order and retries question

2019-11-10 Thread Matthias J. Sax
As you enable idempotance, you should set retries to `Integer.MAX_VALUES` -- for newer version in which the default is MAX_VALUES you can of course remove the config. This will give you strict ordering guarantees, assuming that your topic is configured correctly, ie, `min.insync.replicats=2` and `

Re: Running Kafka Stream Application in YARN

2019-11-10 Thread Matthias J. Sax
I would checkout the YARN docs on how to run a Java application. There should not be a difference. (Of course, Kafka Streams might be stateful though). -Matthias On 11/9/19 3:26 AM, Debraj Manna wrote: > Anyone any update on this? > > On Fri, 8 Nov 2019, 15:56 Debraj Manna, wrote: > >> Hi >>