> > If a new consumer joins an existing group, it triggers a rebalance even > if it's > consuming from a new topic (which matches my symptoms). >
Not sure I understand the issue. Rebalances are triggered when either 1) group membership changes, 2) a consumer's subscription changes, or 3) when the number of partitions for a topic changes. -Jason On Thu, Jan 7, 2016 at 1:38 PM, Cliff Rhyne <crh...@signal.co> wrote: > I'll give the 0.9.0 trunk a try. > > By the way, it looks to me that this might be a separate issue. I just > setup unique group IDs for the stop/start topics from the always-on topic > and my test passed. I think the issue is that the > GroupCoordinator.doJoinGroup() only tracks the group and not the topic when > deciding whether to rebalance (GroupCoordinator.prepareRebalance()). If a > new consumer joins an existing group, it triggers a rebalance even if it's > consuming from a new topic (which matches my symptoms). > > What do you think? > > On Thu, Jan 7, 2016 at 3:27 PM, Jason Gustafson <ja...@confluent.io> > wrote: > > > There have been bugs affecting both the client and the server. The one I > > mentioned above only affected the client, so you could try updating it > > alone if that's easier, but it would be better to do both. > > > > I'll leave it to others to comment on the release timeline. I haven't > seen > > any major consumer-related bugs in the past couple weeks, so my feeling > is > > that it's starting to stabilize. It would be nice to get KIP-41 into the > > next release though. > > > > -Jason > > > > On Thu, Jan 7, 2016 at 1:18 PM, Cliff Rhyne <crh...@signal.co> wrote: > > > > > Hi Jason, > > > > > > I'm just on the 0.9.0.0 release. Are the fixes in the client, the > kafka > > > service, or both? I'll give it a try. > > > > > > Is there a timeline for when 0.9.0.1 would be released? > > > > > > Thanks, > > > Cliff > > > > > > On Thu, Jan 7, 2016 at 3:14 PM, Jason Gustafson <ja...@confluent.io> > > > wrote: > > > > > > > Hey Cliff, > > > > > > > > Are you using the 0.9.0.0 release? We've fixed a few problems in the > > > 0.9.0 > > > > branch, some of which might explain the behavior you're seeing. There > > was > > > > one bug in particular which resulted in the consumer not fetching > data > > > for > > > > a set of partitions after a rebalance. > > > > > > > > -Jason > > > > > > > > On Thu, Jan 7, 2016 at 1:01 PM, Cliff Rhyne <crh...@signal.co> > wrote: > > > > > > > > > Hi, > > > > > > > > > > I'm testing out some changes with the 0.9.0.0 new KafkaConsumer > API. > > > We > > > > > re-use the same consumer group ID across different components of > the > > > > > application (which consume from different topics). One topic is > > always > > > > > being consumed from, the rest are turned on and off. > > > > > > > > > > If I only run the always-on consumer, I have a very low occurrence > > rate > > > > of > > > > > the log message below: > > > > > > > > > > org.apache.kafka.clients.consumer.internals.AbstractCoordinator - > > > Attempt > > > > > to heart beat failed since the group is rebalancing, try to re-join > > > > group. > > > > > > > > > > > > > > > If I run both types of consumers, the log message occurs frequently > > and > > > > the > > > > > alway-on consumer eventually doesn't succeed in rejoining (I see > the > > > > > attempt in the logs to rejoin but nothing happens after that). I > > only > > > > have > > > > > logs on the client side to work with; there's nothing showing up in > > the > > > > > kafka logs to show why the group's state isn't stable. > > > > > > > > > > Thanks, > > > > > Cliff > > > > > > > > > > -- > > > > > Cliff Rhyne > > > > > Software Engineering Lead > > > > > e: crh...@signal.co > > > > > signal.co > > > > > ________________________ > > > > > > > > > > Cut Through the Noise > > > > > > > > > > This e-mail and any files transmitted with it are for the sole use > of > > > the > > > > > intended recipient(s) and may contain confidential and privileged > > > > > information. Any unauthorized use of this email is strictly > > prohibited. > > > > > ©2015 Signal. All rights reserved. > > > > > > > > > > > > > > > > > > > > > -- > > > Cliff Rhyne > > > Software Engineering Lead > > > e: crh...@signal.co > > > signal.co > > > ________________________ > > > > > > Cut Through the Noise > > > > > > This e-mail and any files transmitted with it are for the sole use of > the > > > intended recipient(s) and may contain confidential and privileged > > > information. Any unauthorized use of this email is strictly prohibited. > > > ©2015 Signal. All rights reserved. > > > > > > > > > -- > Cliff Rhyne > Software Engineering Lead > e: crh...@signal.co > signal.co > ________________________ > > Cut Through the Noise > > This e-mail and any files transmitted with it are for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. Any unauthorized use of this email is strictly prohibited. > ©2015 Signal. All rights reserved. >
