Re: Partition ownership with high-level consumer

2015-10-07 Thread Ewen Cheslack-Postava
And you can get the current assignment in the new consumer after the rebalance completes too: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L593 On Tue, Oct 6, 2015 at 5:27 PM, Gwen Shapira wrote: > Ah, so the new consumer g

Re: Partition ownership with high-level consumer

2015-10-06 Thread Gwen Shapira
Ah, so the new consumer got that one: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceListener.java#L88 On Tue, Oct 6, 2015 at 3:55 PM, Joey Echeverria wrote: > I really only want them for the partitions I own. The client should

Re: Partition ownership with high-level consumer

2015-10-06 Thread Joey Echeverria
I really only want them for the partitions I own. The client should know that in order to acquire the zookeeper locks and could potentially execute a callback to tell me the partitions I own after a rebalance. -Joey On Tue, Oct 6, 2015 at 4:08 PM, Gwen Shapira wrote: > I don't think so. AFAIK,

Re: Partition ownership with high-level consumer

2015-10-06 Thread Gwen Shapira
I don't think so. AFAIK, even the new API won't send this information to every consumer, because in some cases it can be huge. On Tue, Oct 6, 2015 at 1:44 PM, Joey Echeverria wrote: > But nothing in the API? > > -Joey > > On Tue, Oct 6, 2015 at 3:43 PM, Gwen Shapira wrote: > > > Zookeeper wil

Re: Partition ownership with high-level consumer

2015-10-06 Thread Joey Echeverria
But nothing in the API? -Joey On Tue, Oct 6, 2015 at 3:43 PM, Gwen Shapira wrote: > Zookeeper will have this information under /consumers//owners > > > > On Tue, Oct 6, 2015 at 12:22 PM, Joey Echeverria wrote: > > > Hi! > > > > Is there a way to track current partition ownership when using the

Re: Partition ownership with high-level consumer

2015-10-06 Thread Gwen Shapira
Zookeeper will have this information under /consumers//owners On Tue, Oct 6, 2015 at 12:22 PM, Joey Echeverria wrote: > Hi! > > Is there a way to track current partition ownership when using the > high-level consumer? It looks like the rebalance callback only tells me the > partitions I'm (pot