Re: Partition assignment not well distributed over threads

2020-07-30 Thread Giselle Van Dongen
Hey Sophie, This was indeed the issue. An environment variable got passed through wrong. Thank you for your tip that made me check this. Giselle On 2020/07/29 17:41:43, Sophie Blee-Goldman wrote: > Hey Giselle, > > How many stream threads is each instance configured with? If the total > num

Re: Partition assignment not well distributed over threads

2020-07-29 Thread Sophie Blee-Goldman
Hey Giselle, How many stream threads is each instance configured with? If the total number of threads across all instances exceeds the total number of tasks, then some threads won't get any assigned tasks. There's a known bug where tasks might not get evenly distributed over all instances in this

Partition assignment not well distributed over threads

2020-07-29 Thread Giselle Van Dongen
We have a Kafka Streams (2.4) app consisting of 5 instances. It reads from a Kafka topic with 20 partitions (5 brokers). We notice that the partition assignment does not always lead to well distributed load over the different threads. We notice this at startup as well as after a recovery of a