Hi Navneeth, To know the reason why there's more than one partition in the same stream task, we should know why the rebalance triggered. That might have to look into the logs.
> I have configured standby to be 1 which means there will be one more copy of the state store and warm up by default is 2. What's the difference, will there be 2 copies now? You should have "at most" 3 copies now, which is 1 standby + 2 warmup when having enough Kafka stream instances. Thank you. Luke On Sat, Feb 19, 2022 at 2:18 PM Navneeth Krishnan <reachnavnee...@gmail.com> wrote: > Hi Guozhang, > > Thanks and sorry for the late reply. I'm overriding the > GROUP_INSTANCE_ID_CONFIG > & APPLICATION_SERVER_CONFIG. > Rest all are defaults. Even then I see more than one partition being > allocated to the same stream task. > > Also I have an additional question regarding the replicas. The default > values for the configs num.standby.replicas & max.warmup.replicas are 0 & 2 > respectively. I have configured standby to be 1 which means there will be > one more copy of the state store and warm up by default is 2. What's the > difference, will there be 2 copies now? > > Thanks > > On Fri, Feb 4, 2022 at 1:13 AM Guozhang Wang <wangg...@gmail.com> wrote: > > > Hello Navneeth, > > > > Could you describe how you ended up with more than one partition > > assigned to the same thread after certain rebalance(s)? Do you override > any > > default config values such as instance.id (for static consumer members), > > etc? > > > > Also I'd suggest upgrading to a newer version --- we just released 3.1.0 > > --- since we've made many improvements / fix bugs around rebalances and > > assignment logic since 2.6. > > > > > > Guozhang > > > > On Wed, Feb 2, 2022 at 9:37 AM Navneeth Krishnan < > reachnavnee...@gmail.com > > > > > wrote: > > > > > Hi All, > > > > > > We are facing an issue with our kafka streams application due to uneven > > > task allocation. There are 100 partitions in the input topic with 100 > > > stream threads processing the data. Everything works well when each > task > > > gets assigned with 1 partition. But when more than one partition is > > > assigned to the same thread then it causes delay in processing causing > a > > > huge backlog. > > > > > > How is everyone handling this? This creates a huge impact to the > > > application SLA and we would like to minimize such cases. Any > suggestions > > > would be appreciated. > > > > > > Note: we are on version 2.6.2 > > > > > > Thanks, > > > Navneeth > > > > > > > > > -- > > -- Guozhang > > >
