Re: Re: leader:none question

2013-10-30 Thread linghongbo008
It seems only this way. Thanks ! by the way, you have a plan to develop this functionality in a future version ? From: Neha Narkhede Date: 2013-10-30 13:13 To: users@kafka.apache.org; linghongbo008 Subject: Re: Re: leader:none question We forgot to delete that script from the 0.8 beta distro

Topic related API

2013-10-30 Thread 小宇
Hi All, why not provide CreateTopic and DeleteTopic api, we know when auto.create.topics.enable set true, kafka can auto create the topic, but the problem is it always take some time before we can lookup the topic in zookeeper, will this can be solved in the future version of kafka? Thanks! Best

Re: Topic related API

2013-10-30 Thread 小宇
Anyone help? 2013/10/30 小宇 > Hi All, > > why not provide CreateTopic and DeleteTopic api, we know when > auto.create.topics.enable set true, kafka can auto create the topic, but > the problem is it always take some time before we can lookup the topic in > zookeeper, will this can be solved in

Re: Topic related API

2013-10-30 Thread Viktor Kolodrevskiy
Why do you want to get topic list from zookeeper? As alternative you can send TopicMetadataRequest to SimpleConsumer and get TopicMetadataResponse that contains list of topics. Thanks, Viktor 2013/10/30 小宇 : > Anyone help? > > > 2013/10/30 小宇 > >> Hi All, >> >> why not provide CreateTopic and D

Re: Re: leader:none question

2013-10-30 Thread Neha Narkhede
Yes, this will be included in either 0.8.1 or 0.8.2. Thanks, Neha On Oct 30, 2013 1:09 AM, "linghongbo008" wrote: > ** > It seems only this way. Thanks ! > by the way, you have a plan to develop this functionality in a future > version ? > -- > > *From:* Neha Narkh

Re: High level consumer stop reading data

2013-10-30 Thread Jun Rao
Are you running the latest code from the 0.8 branch? Thanks, Jun On Tue, Oct 29, 2013 at 10:47 PM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > No, it doesn't throw any exception. Consumer goes to halt state and after > restarting it again starts consuming the data. > > > On Wed,

Re: Plan for Scala 2.10+ support

2013-10-30 Thread Hisham Mardam-Bey
We have 0.7.x with Scala 2.10 and sbt 0.12 here if that's useful to anyone else: https://github.com/mate1/kafka/commits/0.7.2-m1-sbt-0.12 On Wed, Oct 30, 2013 at 12:08 AM, chetan conikee wrote: > Thanks Joe. > We have been maintaining an 0.8+2.10 release in our private repo for the > past few

Questions about producer API

2013-10-30 Thread Roger Hoover
Hi, I'm still getting started with Kafka and was curious why there is an asymmetry between the producer and consumer APIs. Why does the producer config take a list of brokers where as the consumer config takes a list of brokers? Thanks, Roger

Re: High level consumer stop reading data

2013-10-30 Thread Hanish Bansal
No. We are using binary of kafka "0.8.0 Beta1 Release". On Wed, Oct 30, 2013 at 8:26 PM, Jun Rao wrote: > Are you running the latest code from the 0.8 branch? > > Thanks, > > Jun > > > On Tue, Oct 29, 2013 at 10:47 PM, Hanish Bansal < > hanish.bansal.agar...@gmail.com> wrote: > > > No, it doesn

Re: Plan for Scala 2.10+ support

2013-10-30 Thread Ngu, Bob
I am using Maven to build my Scala app but to resolve the bad POM definition issue, https://issues.apache.org/jira/browse/KAFKA-1064, I added all the required dependencies to my POM and thought I had to use Scala 2.9.2 but it seems to build just fine with 2.10.2, this should be fine, right? Thanks

Re: High level consumer stop reading data

2013-10-30 Thread Neha Narkhede
Could you try using 0.8 HEAD? We fixed quite a few consumer bugs from 0.8.0 Beta1 to 0.8 HEAD. Thanks, Neha On Wed, Oct 30, 2013 at 10:29 AM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > No. We are using binary of kafka "0.8.0 Beta1 Release". > > > On Wed, Oct 30, 2013 at 8:26 PM,

Kafka 0.8, Scala 2.10.2, OSGi

2013-10-30 Thread Ngu, Bob
Hi, I decided to start a new thread regarding Kafka 0.8 beta1 support for Scala 2.10.2 and OSGi. I asked about 2.10.2 in a different thread I am using Maven to build my Scala library but to resolve the bad POM definition issue, https://issues.apache.org/jira/browse/KAFKA-1064, I added all the re

Re: Kafka 0.8, Scala 2.10.2, OSGi

2013-10-30 Thread Ngu, Bob
Also I have to upgrade scalatest to 1.9 because 1.8 no longer exists. However, I got a bunch of compile errors building with 2.10.2, any insights will be much appreciated, thanks. [error] /Users/rngu/ij/kafka-0.8.0-beta1-src/core/src/main/scala/kafka/utils/Annota tions.scala:31: not found: type S

Re: Kafka 0.8, Scala 2.10.2, OSGi

2013-10-30 Thread Joe Stein
Can you go off the 0.8 branch please? We're in the process of the 0.8.0 release so if you still have problems that were not resolved we can know what they are. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.st

Re: Kafka 0.8, Scala 2.10.2, OSGi

2013-10-30 Thread Ngu, Bob
Thanks, I did just that, I found the kafka github repo, grabbed the 0.8 branch and it builds fine now. Any thoughts on the OSGi support question? On 10/30/13, 4:00 PM, "Joe Stein" wrote: >Can you go off the 0.8 branch please? We're in the process of the 0.8.0 >release so if you still have proble

Re: Kafka 0.8, Scala 2.10.2, OSGi

2013-10-30 Thread Ngu, Bob
Hi Joe, I ran into a number of missing PGP key error with “sbt publish-local” [error] (Kafka/*:packaged-artifacts) java.io.FileNotFoundException: /Users/rngu/.sbt/gpg/secring.asc (No such file or directory) [error] (perf/*:packaged-artifacts) java.io.FileNotFoundException: /Users/rngu/.sbt/gpg/sec

Re: Kafka 0.8, Scala 2.10.2, OSGi

2013-10-30 Thread Joe Stein
you need a pgp keypair. if you use sbt you could try http://www.scala-sbt.org/sbt-pgp/usage.html or without sbt get OpenPGP installed http://linux.die.net/man/1/gpg2 and then gpg2 --gen-key On Wed, Oct 30, 2013 at 8:05 PM, Ngu, Bob wrote: > Hi Joe, > > I ran into a number of missing PGP key

Re: Kafka 0.8, Scala 2.10.2, OSGi

2013-10-30 Thread Joe Stein
In regards to OSGi maybe it is something you would propose and help out with for the client rewrite https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite I don't know what is entailed for coding and long term support and what the community benefit would be. If you could help with any of

Re: Questions about producer API

2013-10-30 Thread Neha Narkhede
Agree that it is somewhat awkward to use zookeeper for broker discovery on consumer, but a broker list on the producer. There were a couple of discussions on the mailing list suggesting using zookeeper on the producer, at least for discovering the brokers for the first time. However, we are startin

Roadmap

2013-10-30 Thread Richard Rodseth
Hi I just started looking at Kafka today. Installing 0.8 Beta 1 went well and I was able to run the command-line producer and consumer. When I tried to write some Java code, I ran into problems with Maven dependencies (it was looking for Scala.Product). I used the appropriate from Maven central.

Why Apache Kafka is bettern than any other Message System?

2013-10-30 Thread 이연옥
Hi, all. I just got curiosity why Apache Kafka is better than any other Message System in terms of throughput, and durability. What’s the fact to let Kafka have better performance? I read some documents and researched, however, I am not quite sure about that.. And Zookeeper in Apache Kafka seems l

Re: Why Apache Kafka is bettern than any other Message System?

2013-10-30 Thread Philip O'Toole
On Wed, Oct 30, 2013 at 8:13 PM, Lee, Yeon Ok (이연옥) wrote: > Hi, all. > I just got curiosity why Apache Kafka is better than any other Message > System in terms of throughput, and durability. > Because it's brilliant, that's why. :-) > What’s the fact to let Kafka have better performance? > Be

Connection reset by peer Error in Kafka server.log

2013-10-30 Thread Banerjee, Aparup
Hi, I keep getting this error in Kafka server.log. I don't see anything in my producer or consumer log. Any idea what can be going on here? Aparup 2013-10-31 02:37:13,917] INFO Closing socket connection to /x.x.x.x. (kafka.network.Processor) [2013-10-31 02:37:21,645] INFO Closing socket connec

Re: Why Apache Kafka is bettern than any other Message System?

2013-10-30 Thread Neha Narkhede
You can find Kafka use cases described here - http://kafka.apache.org/documentation.html#uses Also, the design document is a good read to understand why Kafka is more efficient - http://kafka.apache.org/documentation.html#design Thanks, Neha On Wed, Oct 30, 2013 at 8:24 PM, Philip O'Toole wrot