ConsumerConnector connecting to Zookeeper: EndOfStreamException

2014-05-06 Thread Sebastian Mattheis
Hello, I get an error when trying to 'consume' messages from Kafka (2.9.2-0.8.1) with a Zookeer stand-alone (3.4.5). You can see the source code below as well as the error message and logfile from Zookeeper. I'm not sure if the Java libraries are incompatible, because I added dependency kafka_

kafka dynamic topic discovery

2014-05-06 Thread Kannadhasan Sekar
Hi friends, I am working on Kafka based pub/sub project.I would like know more about Kafka dynamic topic consumer.My producer will publish data into any topic. My consumer need to consume all the data from all the topic. In current Kafka version is it support wild card topic consumer? ​ Than

still looking for Maven POM for Java Kafka

2014-05-06 Thread David Novogrodsky
All, I am running some examples from the packet book Apache Kafka. I am creating a HighLevelconsumer. I am getting an error when I try to run the code: -- C:\Users\david.j.novogrodsky\Documents\TestingKafka\target>java -cp Kafka_test-1.0-SNAPSHOT.jar com.cat.HighLevelConsumer test Exception in t

Re: still looking for Maven POM for Java Kafka

2014-05-06 Thread Yury Ruchin
Looks like Kafka classes is not on your classpath. You should either assemble an uber-jar from your project (e. g. using Maven Assembly plugin with jar-with-dependencies descriptor ref) or add location of Kafka classes to your classpath. 2014-05-06 19:11 GMT+04:00 David Novogrodsky : > All, > >

Re: kafka dynamic topic discovery

2014-05-06 Thread Guozhang Wang
Hello, You may find this doc helpful in using a wildcard topic subscription: https://kafka.apache.org/documentation.html#highlevelconsumerapi Guozhang On Tue, May 6, 2014 at 7:00 AM, Kannadhasan Sekar wrote: > Hi friends, >I am working on Kafka based pub/sub project.I would like know more

Re: JAVA HEAP settings for KAFKA in production

2014-05-06 Thread Jay Kreps
Hey Todd, Doc patch? :-) svn co http://svn.apache.org/repos/asf/kafka/site/081/ Don't stress about html or formatting, I'm happy to do that part. I would love to give people more authoritative advice. Right now everything is a bit obsolete and wrong. -Jay On Mon, May 5, 2014 at 10:36 PM, Todd

Re: still looking for Maven POM for Java Kafka

2014-05-06 Thread Steven A Robenalt
Hi Dave, Here's a POM I use to build the examples. It's pretty simple, including kafka 0.8.0 with scala 2.9.2 libs, and targeting Java 7. I also made a couple of mods to make things easier: 1) I use log4j 1.2.17 to eliminate a log4j 1.2.15 dependency on the sun jdmk. 2) I use the maven dependency

Re: kafka dynamic topic discovery

2014-05-06 Thread Neha Narkhede
Yes, wildcard consumption is supported in the latest stable release of Kafka (0.8.1.1). Thanks, Neha On Tue, May 6, 2014 at 7:00 AM, Kannadhasan Sekar wrote: > Hi friends, >I am working on Kafka based pub/sub project.I would like know more about > Kafka dynamic topic consumer.My producer w

Re: JAVA HEAP settings for KAFKA in production

2014-05-06 Thread Neha Narkhede
Updated the docs with Todd's recommendation - http://kafka.apache.org/081/documentation.html#java Thanks, Neha On Tue, May 6, 2014 at 9:40 AM, Jay Kreps wrote: > Hey Todd, > > Doc patch? :-) > > svn co http://svn.apache.org/repos/asf/kafka/site/081/ > > Don't stress about html or formatting, I

Re: still looking for Maven POM for Java Kafka

2014-05-06 Thread David Novogrodsky
Yury, Thank you for the reply. It was helpful. David Novogrodsky david.novogrod...@gmail.com http://www.linkedin.com/in/davidnovogrodsky On Tue, May 6, 2014 at 10:47 AM, Yury Ruchin wrote: > Looks like Kafka classes is not on your classpath. You should either > assemble an uber-jar from your

Re: QOS on Producer Side

2014-05-06 Thread Bhavesh Mistry
So to retrieve the message that was not transmitted due to error, I will have to store the message in each call back the message and store this in a file.Is there an alternative solution ? What is best way to achieve QOS on producer side ? Also, will new Producer API will work with Broker ve