Re: Kafka KIP meeting Aug. 18

2015-08-18 Thread Jun Rao
Yes, that's related to the zombie consumer. Basically, if a consumer is in a long GC, it may be taken out of the group. However, when the consumer wakes up from the GC, it may still be able to consume data that it's no longer responsible for. We protect this a little bit by not allowing a zombie co

Re: Kafka KIP meeting Aug. 18

2015-08-18 Thread Flavio Junqueira
Thanks for the notes, Jun. There was a mention of single-writer behavior, if I understood correctly, in the context of the new consumer, and I was wondering if that's specific to the new consumer work or if it's going to have any dependency or relation to KIP-27. -Flavio > On 18 Aug 2015, at 2

Re: Kafka KIP meeting Aug. 18

2015-08-18 Thread Jun Rao
The the following are the notes for today's KIP discussion. * client-side assignment strategy: We discussed concerns about rebalancing time due to metadata inconsistency, especially when lots of topics are subscribed. Will discuss a bit more on the mailing list. * CopyCat data api: The discussion

Kafka KIP meeting Aug. 18

2015-08-17 Thread Jun Rao
Hi, Everyone, We will have a Kafka KIP meeting tomorrow at 11am PST. If you plan to attend but haven't received an invite, please let me know. The following is the agenda. Agenda 1. Discuss the proposal on client-side assignment strategy ( https://cwiki.apache.org/confluence/display/KAFKA/Kafka+C