Hey Dana, The intention of the UserData field is to allow custom partition assignments to leverage member-specific metadata. For example, it might include the rack name of the host for a rack-aware assignment strategy or the number of cpus for a resource-based assignment strategy. The strategies shipped with Kafka just leave this field empty.
I think the main use case for having different topic subscriptions in a group is a rolling upgrade which involves a change in the consumed topics. I don't know of any cases where different subscriptions are needed in a stable state. By the way, I spent a little time updating the protocol documentation here: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-GroupMembershipAPI. It's still a little incomplete, so I'll keep working on it. Feel free to ask further questions here if something is unclear. -Jason On Wed, Nov 25, 2015 at 2:23 AM, Dana Powers <dana.pow...@rd.io> wrote: > Thanks, Jason. I see the range and roundrobin assignment strategies > documented in the source. I don't see userdata used by either -- is that > correct (I may be misreading)? The notes suggest userdata for something > more detailed in the future, like rack-aware placements? > > One other question: in what circumstances would consumer processes in a > single group want to use different topic subscriptions rather than > configure a new group? > > Thanks again, > > -Dana > On Nov 25, 2015 8:59 AM, "Jason Gustafson" <ja...@confluent.io> wrote: > > > Hey Dana, > > > > Have a look at this wiki, which has more detail on the consumer's > embedded > > protocol: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Client-side+Assignment+Proposal > > . > > > > At the moment, the group protocol supports consumer groups and kafka > > connect groups. Kafka tooling currently depends on the structure for > these > > protocol types, so reuse of the same names might cause problems. I will > > look into updating the protocol documentation to standardize the protocol > > formats that are in use and to provide guidance for client > implementations. > > My own view is that unless there's a good reason not to, all consumer > > implementation should use the same consumer protocol format so that > tooling > > will work correctly. > > > > Thanks, > > Jason > > > > > > > > On Tue, Nov 24, 2015 at 4:16 PM, Dana Powers <dana.pow...@gmail.com> > > wrote: > > > > > Hi all - I've been reading through the wiki docs and mailing list > threads > > > for the new JoinGroup/SyncGroup/Heartbeat APIs, hoping to add > > functionality > > > to the python driver. It appears that there is a shared notion of group > > > "protocols" (client publishes supported protocols, coordinator picks > > > protocol for group to use), and their associated metadata. Is there any > > > documentation available for existing protocols? Will there be an > official > > > place to document that supporting protocol X means foo? I think I can > > > probably construct a simple working protocol, but if I pick a protocol > > name > > > that already exists, will things break? > > > > > > -Dana > > > > > >
