Re: Controller is not being failed over 0.8.1

2014-04-17 Thread Jun Rao
The following suggest that the controlled shutdown failed. Since you are many partitions, you probably need to increase controlled.shutdown.retry.backoff.ms and controlled.shutdown.max.retries to give enough time for the controlled shutdown to complete. Also, try increasing controller.socket.timeou

Re: confirm subscribe to users@kafka.apache.org

2014-04-17 Thread Harsha
On Thu, Apr 17, 2014, at 05:37 PM, users-h...@kafka.apache.org wrote: > Hi! This is the ezmlm program. I'm managing the > users@kafka.apache.org mailing list. > > I'm working for my owner, who can be reached > at users-ow...@kafka.apache.org. > > To confirm that you would like > >ka...@har

RE: Cluster design distribution and JBOD vs RAID

2014-04-17 Thread Bello, Bob
Some feedback from your feedback. BERT> We have several uses cases we are looking at kafka for. Today we are just using the file system to buffer data between our systems. We are looking at uses cases that have varying message sizes of 200, 300, 1000, 2200 bytes BOB> Since you are using small m

Re: Reaction to Reactive Streams?

2014-04-17 Thread Scott Clasen
Would be great to get a reactive consumer api (in 0.9 ??) On Thu, Apr 17, 2014 at 11:58 AM, Richard Rodseth wrote: > Announcement from Typesafe et al: > > http://www.reactive-streams.org/ > https://typesafe.com/blog/typesafe-announces-akka-streams >

Re: Unable to get off the ground following the "quick start" section

2014-04-17 Thread Manoj Khangaonkar
Hi I was able to get the quickstart instructions working recently but I used the (1) binary download (2) I did not use the zookeeper packaged with Kafka. I installed zookeeper using a download from the zookeeper projects. ( I did get a lot of exceptions with the packaged zookeeper) regards O

Reaction to Reactive Streams?

2014-04-17 Thread Richard Rodseth
Announcement from Typesafe et al: http://www.reactive-streams.org/ https://typesafe.com/blog/typesafe-announces-akka-streams

Re: How to calculate number of partitions?

2014-04-17 Thread Cassa L
Thanks. that helps. LCassa On Wed, Apr 16, 2014 at 3:39 PM, Joel Koshy wrote: > Does this help: > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowdoIchoosethenumberofpartitionsforatopic > ? > > > On Wed, Apr 16, 2014 at 03:34:28PM -0700, Cassa L wrote: > > Hi, > > I am sizin

RE: Controller is not being failed over 0.8.1

2014-04-17 Thread Bello, Bob
I setup a 2 node cluster. With replication-factor 2. Create 10 topics with 131 partitions each then tried to do a controlled shutdown of broker #2 (which is the current elected leader). (Sorry, our Broker #1 and Broker #2 server times are 6h off, I will have to get that fixed). According to Z

Re: zkclient version

2014-04-17 Thread Jun Rao
0.3 is the right version. Are you sure the 0.3 jar is included? That method only exists in 0.3. Thanks, Jun On Thu, Apr 17, 2014 at 6:32 AM, Yonghui Zhao wrote: > Hi > > I am trying kafka 0.8.1, and use zkclient version: > > > com.101tec > zkclient >

RE: Controller is not being failed over 0.8.1

2014-04-17 Thread Seshadri, Balaji
The ZK data was showing controller as still broker-1. @Bob, Can you please send errors you see in the state-change.log. Thanks, Balaji From: Jun Rao [jun...@gmail.com] Sent: Wednesday, April 16, 2014 9:37 PM To: users@kafka.apache.org Subject: Re: Contro

Re: Unable to get off the ground following the "quick start" section

2014-04-17 Thread Stephen Boesch
Thanks I will look into that resource. Using the tgz is working it turns out I was symlinked back to the src dir when encountering the issue. 2014-04-17 6:37 GMT-07:00 Bert Corderman : > I cant speak to the quick start , however I found the following very > helpful when I was getting started. >

Re: Unable to get off the ground following the "quick start" section

2014-04-17 Thread Bert Corderman
I cant speak to the quick start , however I found the following very helpful when I was getting started. http://www.michael-noll.com/blog/2013/03/13/running-a-multi-broker-apache-kafka-cluster-on-a-single-node/ On Thu, Apr 17, 2014 at 9:22 AM, Stephen Boesch wrote: > I have tried to use kafka

zkclient version

2014-04-17 Thread Yonghui Zhao
Hi I am trying kafka 0.8.1, and use zkclient version: com.101tec zkclient 0.3 But I found exceptions in console, is zkversion is wrong? *java.lang.NoSuchMethodError: org.I0Itec.zkclient.ZkClient.writeDataReturnStat(Ljava/lang/String;Ljava/la

Unable to get off the ground following the "quick start" section

2014-04-17 Thread Stephen Boesch
I have tried to use kafka both building form source via gradle as well as un tgz-ing the tarball. Either way, none of the scripts work in the way described in the Quick Start section. Following is an example : 6:15:40/kafka:91 $bin/zookeeper-server-start.sh config/zookeeper.properties Exception i

Re: Cluster design distribution and JBOD vs RAID

2014-04-17 Thread Bert Corderman
Hey Bob, thanks for your detailed response. I have added comments inline. On Wed, Apr 16, 2014 at 7:41 PM, Bello, Bob wrote: > Perhaps as you consider the size of your cluster, a few questions about > the kind of messaging you are looking at? I can use an example of what we > do in our produc