>From what I understood, it seems that stickiness is preserved only for the remaining live consumers.
Say a consumer owns some partitions and then dies. Those partitions will get redistributed to the rest of the group. Now if the consumer comes back up, based on the algorithm described with the concept of "reassignable partitions", then the consumer may get different partitions than what it had before. Is my understanding right? Put another way: once coming back up, can the consumer load its UserData with the assignment it had before dying? On Wed, Jun 22, 2016 at 4:41 PM, Jason Gustafson <ja...@confluent.io> wrote: > Hey Vahid, > > Thanks for the updates. I think the lack of comments on this KIP suggests > that the motivation might need a little work. Here are the two main > benefits of this assignor as I see them: > > 1. It can give a more balanced assignment when subscriptions do not match > in a group (this is the same problem solved by KIP-49). > 2. It potentially allows applications to save the need to cleanup partition > state when rebalancing since partitions are more likely to stay assigned to > the same consumer. > > Does that seem right to you? > > I think it's unclear how serious the first problem is. Providing better > balance when subscriptions differ is nice, but are rolling updates the only > scenario where this is encountered? Or are there more general use cases > where differing subscriptions could persist for a longer duration? I'm also > wondering if this assignor addresses the problem found in KAFKA-2019. It > would be useful to confirm whether this problem still exists with the new > consumer's round robin strategy and how (whether?) it is addressed by this > assignor. > > The major selling point seems to be the second point. This is definitely > nice to have, but would you expect a lot of value in practice since > consumer groups are usually assumed to be stable? It might help to describe > some specific use cases to help motivate the proposal. One of the downsides > is that it requires users to restructure their code to get any benefit from > it. In particular, they need to move partition cleanup out of the > onPartitionsRevoked() callback and into onPartitionsAssigned(). This is a > little awkward and will probably make explaining the consumer more > difficult. It's probably worth including a discussion of this point in the > proposal with an example. > > Thanks, > Jason > > > > On Tue, Jun 7, 2016 at 4:05 PM, Vahid S Hashemian < > vahidhashem...@us.ibm.com > > wrote: > > > Hi Jason, > > > > I updated the KIP and added some details about the user data, the > > assignment algorithm, and the alternative strategies to consider. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-54+-+Sticky+Partition+Assignment+Strategy > > > > Please let me know if I missed to add something. Thank you. > > > > Regards, > > --Vahid > > > > > > >
