Re: Programmatic Kafka version detection/extraction?

gt;> > > >> > > On Mon, Nov 10, 2014 at 7:17 PM, Otis Gospodnetic < >> > > otis.gospodne...@gmail.com> wrote: >> > > >> > > > Hi, >> > > > >> > > > Is there a way to detect which version of Kafka one is running? >> > > > Is there an API for that, or a constant with this value, or maybe an >> > > MBean >> > > > or some other way to get to this info? >> > > > >> > > > Thanks, >> > > > Otis >> > > > -- >> > > > Monitoring * Alerting * Anomaly Detection * Centralized Log >> Management >> > > > Solr & Elasticsearch Support * http://sematext.com/ >> > > > >> > > >> > >> -- Joey Echeverria

Partition ownership with high-level consumer

Hi! Is there a way to track current partition ownership when using the high-level consumer? It looks like the rebalance callback only tells me the partitions I'm (potentially) losing. -Joey

Re: Partition ownership with high-level consumer

But nothing in the API? -Joey On Tue, Oct 6, 2015 at 3:43 PM, Gwen Shapira wrote: > Zookeeper will have this information under /consumers//owners > > > > On Tue, Oct 6, 2015 at 12:22 PM, Joey Echeverria wrote: > > > Hi! > > > > Is there a way to track cu

Re: Partition ownership with high-level consumer

o. AFAIK, even the new API won't send this information to > every consumer, because in some cases it can be huge. > > > > On Tue, Oct 6, 2015 at 1:44 PM, Joey Echeverria wrote: > > > But nothing in the API? > > > > -Joey > > > > On Tue, Oct 6, 2

Re: Consumer with more stable partition assignments?

I don't have a solution, but I thought I'd chime in with interest in finding a solution to this problem. We have a use case where we're partitioning the dataset we write to according to Kafka partitions and having to close all writers and re-open after a rebalance is a pain point. -Joey > On

Handling long commits during a rebalance

We've been playing around with the new Consumer API and have it an unfortunate bump in the road. When our onPartitionsRevoked() callback is called we'd like to be able to commit any data that we were processing to stable storage so we can then commit the offsets back to Kafka. This way we don't thr

Re: Handling long commits during a rebalance

would check to > see if there were offsets that needed to committed, if so it would commit > them. > > It was a bunch of work to get it working and it's still not perfect, but > it's getting the job done. > > -craig > > On Tue, Jul 26, 2016 at 3:13 PM, Joey E

Re: Handling long commits during a rebalance

queue is cleared...again may not > ideal, but works for this use case. > > On Tue, Jul 26, 2016 at 8:53 PM, Joey Echeverria wrote: > >> That's the direction we're looking at for normal commit processing, >> but how do you handle commits during a rebalanc