Re: Kafka-Storm Run-time Exception

2014-06-05 Thread Andrew Neilson
it's possible you have some other dependency using an earlier version of Scala. A common one to check for when using Kafka is jline 0.9.94, which comes through the zookeeper 3.3.4 dependency included with kafka_2.9.2-0.8.1 and has more than one dependency that uses scala 2.8.x. If this is where it

2 kafkas 1 zookeeper

2013-06-05 Thread Andrew Neilson
We are interested in having two kafka clusters (one 0.7.2, one 0.8) share the same 3 node zookeeper tier (3.4.3). Has anyone else tried something like this in the past? Should we anticipate any issues? Thanks! Andrew

Re: seeing poor consumer performance in 0.7.2

2013-04-26 Thread Andrew Neilson
the consumer is lagging, this indicates a problem on the consumer side. > > > > > > Thanks, > > > > > > Jun > > > > > > > > > On Mon, Apr 22, 2013 at 9:13 PM, Andrew Neilson > > >wrote: > > > > > > > Hmm it is highly u

Re: seeing poor consumer performance in 0.7.2

2013-04-26 Thread Andrew Neilson
> > Thanks, > > Jun > > > On Mon, Apr 22, 2013 at 9:13 PM, Andrew Neilson >wrote: > > > Hmm it is highly unlikely that that is the culprit... There is lots of > > bandwidth available for me to use. I will definitely keep that in mind > > though. I was working o

Re: Occasional batch send errors

2013-04-23 Thread Andrew Neilson
Hey Karl, I have a very similar setup (3 kafka 0.7.2 brokers, 3 ZK 3.4.3 nodes) that I'm running right now and am getting the same error on the producers. Haven't resolved it yet: ERROR ProducerSendThread--1585663279 kafka.producer.async.ProducerSendThread - Error in handling batch of 200 events j

Re: seeing poor consumer performance in 0.7.2

2013-04-22 Thread Andrew Neilson
Oh... and at this point I'm talking about consumers that do no processing and don't even produce any output. They simply send udp packets to graphite. On Mon, Apr 22, 2013 at 9:13 PM, Andrew Neilson wrote: > Hmm it is highly unlikely that that is the culprit... There is lots o

Re: seeing poor consumer performance in 0.7.2

2013-04-22 Thread Andrew Neilson
Jun Rao wrote: > Is your network shared? Is so, another possibility is that some other apps > are consuming the bandwidth. > > Thanks, > > Jun > > > On Sun, Apr 21, 2013 at 12:23 PM, Andrew Neilson >wrote: > > > Thanks very much for the reply Neha! So I swapped

Re: seeing poor consumer performance in 0.7.2

2013-04-21 Thread Andrew Neilson
and see how it performs > ? > > Thanks, > Neha > > On Sun, Apr 21, 2013 at 1:59 AM, Andrew Neilson > wrote: > > I am currently running a deployment with 3 brokers, 3 ZK, 3 producers, 2 > > consumers, and 15 topics. I should first point out that this is my first >

seeing poor consumer performance in 0.7.2

2013-04-21 Thread Andrew Neilson
I am currently running a deployment with 3 brokers, 3 ZK, 3 producers, 2 consumers, and 15 topics. I should first point out that this is my first project using Kafka ;). The issue I'm seeing is that the consumers are only processing about 15 messages per second from what should be the largest topic