Hi Jason, Thanks for your detailed explain. Face to this situation, I wanna discuss more a little bit, I try two approach to avoid it in Kafka 0.9.0.1, and both work correctly.
1. Using subscribe(topics, listener) and implements onPartitionsAssigned(partitions) , it manually run consumer.commitSyc() when onPartitionsAssigned be called 2. Using consumer.assign(topicpartitions) and assign partition number directly I think first approach is better. It has more flexible for dynamic partition situation and suitable for persistent consumer service. If migrate form high level api to new consumer ,this approach is more straightly. Second approach only use for specific requirement, but it has to control more detail information. It is suitable for target clear job or web service to get given length offsets how do you think? 2016-03-08 1:54 GMT+08:00 Jason Gustafson <ja...@confluent.io>: > Hi Ken, > > It looks like what happened is this: > > 1. First thread joins the group and is assigned partitions 0 and 1 > 2. First thread races through a bunch of messages from these partitions. > 3. Second thread joins and is assigned partition 1 (leaving partition 0 for > the first thread) > 4. Both threads read whatever is left from their respective partitions > > This is normal (expected) behavior. It might seem a little odd that the > first thread gets assigned both partitions initially even though both > threads were started at about the same time. This is the result of the > coordinator somewhat optimistically creating the group and assigning > partitions immediately when the first member joins. When the second member > joins, it forces a partition rebalance, but there will be some latency > before the first member rejoins. We have considered adding a delay on > initial group creation to give time for more members to join, but as of > yet, we haven't implemented it. > > -Jason > > On Sun, Mar 6, 2016 at 7:11 PM, Ken Cheng <kk543...@gmail.com> wrote: > > > > > > > hi > > > > > > > > I'm trying new consumer features in my local PC and VM > > > > and reference code from > > > http://www.confluent.io/blog/tutorial-getting-started-with-the-new-apache-kafka-0.9-consumer-client > > > > ,but execute result is different > > > > > > > > I use 2 threads consumer with same group.id consume one topic with 2 > > partition > > > > ,but one partition assign to two consumer instance ! > > > > > > > > e.g. > > > > If you check the attachement exelog.txt > > > > 1. you can find term "0: {partition=0, offset=0}" twice, means that > > instance pool same offset again > > > > 2. "1: {partition=1, offset=0}" and "0: {partition=1, offset=0}" will be > > found , means same partition assign to two consumer instance > > > > > > > > It is really strange for me > > > > does anyone know why ? > > > > > > > > plz give me a help or tips thanks. > > > > > > > > > > > > > > > > my broker setup: > > > > Kafka version: confluent platform 2.0.1, standalone > > > > zookeeper version : 3.4.6, 3 node cluster > > > > topic partition : 2 > > > > > > > > > > > > Best Regards, > > > > Ken > > > > > > >
