Re: How to set offset for a consumer in Kafka 0.10.0.X

Hi Glen, Maybe have a look at this, slightly different version. Not doing anything more than what you have seen in gist. You should pass broker url as first parameter and and a json containing desired offsets as second parameter. https://github.com/goibibo/woof/tree/master/scripts/offset_commit_

Re: timeindex file timestamp mismatch (0.10.1.0)

Hi, I think you are facing this issue. https://issues.apache.org/jira/browse/KAFKA-3163 Thanks Sudev On Fri, Jan 20, 2017 at 9:50 PM Meghana Narasimhan < mnarasim...@bandwidth.com> wrote: > Hi, > I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters A > and B. I have upgraded

Re: Storing Kafka Message JSON to deep storage like S3

HI Aseem, You can run Apache Flume to consume messages from Kafka and write them to s3/HDFS in (micro batches)streaming fashion. Writes to s3/HDFS should be in micro batches, you can do it for every message (not every sure if s3 supports append) but it won't be performant. https://flume.apache.o

Re: Messages intermittently get lost

Hi Zak, Why don't you try using zookeeper four letter admin tools ? echo stat | nc zookeeper-ip zookeeper-port echo stat | nc localhost 2181 These commands will tell you the current status of zookeeper client. ht

Re: Kafka Mirrormaker in sync mode

Hi Mudit, No, you can't. On Thu, Oct 27, 2016 at 4:35 PM, Mudit Agarwal wrote: > Hi, > Can we run Kafka MirrorMaker in sync mode where both source producer and > consumer both are running in sync node ? > Thanks,Mudit -- Thanks Sudev A C

Re: Scala: Kafka Consumer (kafka-clients 0.9.0.1)

and correctly, I can use the same KafkaConsumer instance with > multiple threads against multiple partitions. > If that is the case, how can I commit the offset specific to my 'Runnable' > instance that just processed a single message without affecting other > threads and partitions? > > Thanks in advance, > Amir Zuker > -- Thanks Sudev A C Data Team

Re: Kafka topic-partition won't serve if prefered election fails for a partition.

lly "preferred leader election” would fail if/when one or more > > brokers still did not come back online after being down for some time. Is > > that your scenario? > > > > -Zakee > > > > > On Aug 11, 2016, at 12:42 AM, Sudev A C > wrote: >

Kafka topic-partition won't serve if prefered election fails for a partition.

Hi, With *auto.leader.rebalance.enable=true* Kafka runs preferred election tool in regular intervals rebalancing topic-partition leaders. Now if preferred election fails to elect the preferred leader for a topic partition it falls back to any other leader in ISR. But for us in production when thi