connection closed by kafka

2016-10-27 Thread Jianbin Wei
In our environment we notice that sometimes Kafka would close the connection after one message is sent over. The client does not detect that and tries to send another message again. That triggers a RST packet. Any idea why the Kafka broker would close the connection? Attached you can find the

Duplicate messages when reading with multiple consumers in same group

2016-10-27 Thread Gabriel Reid
Hi, I've run into an issue using the old high-level consumer, and I'd like to verify that it is indeed a bug (I'm pretty sure it is), or see if I'm just doing something wrong. The situation is that I have two consumers (A and B) in the same consumer group reading from a single topic. One consumer

#Kafka #Zookeeper #Kerberos

2016-10-27 Thread Mohammed Amine GARMES (contractor)
Hello Kafka, I am Mohammed Amine GARMES from AMADEUS SAS. First of all I want to thank you for your solution is very helpful. Indeed, this is my first message to Kafka team, so I have an issue with Kafka security : I have 2 kafka servers configured to start with kafka security, I try to start

Re: Mysterious timeout

2016-10-27 Thread Martin Gainty
MG>can u write simpleConsumer to determine when lead broker times-out.. then you'll need to tweak connection settings https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example MG>to debug the response determine the leadBroker and the reason for fetch failure as seen here: if

Mysterious timeout

2016-10-27 Thread Mike Kaplinskiy
Hey folks, We're observing a very peculiar behavior on our Kafka cluster. When one of the Kafka broker instances goes down, we're seeing the producer block (at .flush) for right about `request.timeout.ms` before returning success (or at least not throwing an exception) and moving on. We're runnin

Re: Kafka Mirrormaker in sync mode

2016-10-27 Thread Sudev A C
Hi Mudit, No, you can't. On Thu, Oct 27, 2016 at 4:35 PM, Mudit Agarwal wrote: > Hi, > Can we run Kafka MirrorMaker in sync mode where both source producer and > consumer both are running in sync node ? > Thanks,Mudit -- Thanks Sudev A C

Kafka Mirrormaker in sync mode

2016-10-27 Thread Mudit Agarwal
Hi, Can we run Kafka MirrorMaker in sync mode where both source producer and consumer both are running in sync node ? Thanks,Mudit