Kafka Socket error

2017-05-09 Thread Arya Ketan
Hi, We got this below error in our production set-up. We then tried to read from kafka consumer shell, we got this error again 2-3 times, but after that it started working and now, we arent able to reproduce. Any indicator on why this error occurs and how it auto corrected.. Googling got me to som

Re: Introducing Dory

2016-06-12 Thread Arya Ketan
Hi Dave, Dory looks pretty interesting. I had a few further questions on it a) How does Dory handle kernel panics? b) What kind of message guarantees does dory provide and also if you can share some design decisions taken to enable the guarantees whatever they are. Thanks Arya Arya On Sun, Jun

Re: [ANN] sqlstream: Simple MySQL binlog to Kafka stream

2015-03-17 Thread Arya Ketan
Apache Kafka." > >> > >> Mypipe currently encodes events using Avro before pushing them into > Kafka > >> and is Avro schema repository aware. The project is young; and patches > for > >> improvements are appreciated (= > >> > >>

Re: [ANN] sqlstream: Simple MySQL binlog to Kafka stream

2015-03-16 Thread Arya Ketan
Great work. Sorry for kinda hijacking this thread, but I though that we had built some-thing on mysql bin log event propagator and wanted to share it . You guys can also look into Aesop ( https://github.com/Flipkart/aesop). Its a change propagation frame-work. It has relays which listens to bin log

Re: Kafka with multiple zookeper

2014-04-21 Thread Arya Ketan
you can give zkconnect as host1:port1,host2:port2,hostn:portn... If you use paths, then host1:port1,host2:port2,hostn:portn/ Arya On Mon, Apr 21, 2014 at 6:46 PM, Dhiraj Thakur < dhiraj.tha...@northgate-is.com> wrote: > Hi, > As per the documentation we can have multiple broker of kafka on mul

Re: Issue with kafka cluster with broker not able to send messages.

2014-04-10 Thread Arya Ketan
limit? > > cat /proc//limits > > On Thu, Apr 10, 2014 at 09:20:51AM +0530, Arya Ketan wrote: > > *Issue : *Kafka cluster goes to an un-responsive state after some time > with > > producers getting Socket time-outs on every request made. > > > > *Kafka Version* -

Re: Issue with kafka cluster with broker not able to send messages.

2014-04-10 Thread Arya Ketan
> The broker seems dead due to too many open file handlers, which are likely > due to too many open sockets. Hhow many producer clientss do you have on > these 5 machines, and could you check if there is any socket leak? > > Guozhang > > > On Wed, Apr 9, 2014 at 8:50 PM, A

Issue with kafka cluster with broker not able to send messages.

2014-04-09 Thread Arya Ketan
*Issue : *Kafka cluster goes to an un-responsive state after some time with producers getting Socket time-outs on every request made. *Kafka Version* - 0.8.1 *Machines* : VMs , 2 cores 8gb RAM, linux , 3 node cluster. ulimit -a core file size (blocks, -c) 0 data seg size (kby