Re: Recovering High Level Consumer from exceptions

Normally, consumers don't timeout unless you have configured it. You just need to make sure you don't kill the consumer thread if it hits an application error. Thanks, Jun On Mon, Oct 28, 2013 at 5:21 PM, Joseph Lawson wrote: > Hi everyone, > > how should a high level consumer implementation

Re: Re: leader:none question

Hi, Neha ! I use kafka0.8 that offered to delete the function of the topic. It can delete the topic by running the shell script "bin/kafka-delete-topic.sh " and create the topic by running the script "kafka-create-topic.sh ". Now, how do I fix this error to the topic of leader isn't "leader :

Recovering High Level Consumer from exceptions

Hi everyone, how should a high level consumer implementation handle consumer exceptions. Using the high level consumer wiki example, the primary thread calls an example.run() and then waits for a thread.sleep and then shuts down. If each ConsumerTest is a thread, what is the best way for the

Re: producer exceptions when broker dies

I think you are right. I was too quick saying that pre 0.7 we also have "at-least-once". Guozhang On Sun, Oct 27, 2013 at 9:51 AM, Jason Rosenberg wrote: > Perhaps, it's a matter of semantics. > > But, I think I'm not talking only about failure, but normal operation. > It's normal to take a c

Re: Limitation on kafka topic size

Hello Suchi, Each topic will have its own partitions and each partition is represented as a list of file segments. Hence having multiple smaller topics means you will have # topics * avg. # partitions_per_topic file handlers. Volume is usually not a problem for Kafka topics, however, if you have

Re: Cannot start Kafka 0.8, exception in leader election

Hello Nicholas, The log shows the controller cannot connect with one of the brokers due to "ava.net.ConnectException: Connection refused". Are all your brokers in the same cluster? Guozhang On Mon, Oct 28, 2013 at 3:37 PM, Nicholas Tietz wrote: > Hi everyone, > > I'm having some problems gett

Cannot start Kafka 0.8, exception in leader election

Hi everyone, I'm having some problems getting Kafka 0.8 running. (I'm running this on Mac OS X 10.8 with the latest updates; Java version 1.6.0_51.) I followed the instructions from the quickstart for 0.8, both from-source and from-binary (cleaned up all related files each time). However, I am get

Re: Broker bind address versus published hostname in ZooKeeper

OK. I've submitted a patch along with unit tests. This change is backward compatible with two new optional parameters called advertise.host.name and advertise.port. The reason I called them that instead of zkHost.name and zkHost.port is that the zkHost.* names sound like they represent the host

Limitation on kafka topic size

All Are there any limitations on max topic size in Kafka. How large can a topic get? What are the implications on resources, latencies, performance and feasability? What are the pros and cons of using a single large topic vs multiple smaller topics? Suchi

Re: Limitation on kafka topic size

We are using kafka 0.7.2 Suchi On Tue, Oct 29, 2013 at 12:53 AM, Suchi Amalapurapu wrote: > All > Are there any limitations on max topic size in Kafka. How large can a > topic get? > What are the implications on resources, latencies, performance and > feasability? > What are the pros and cons of

Re: How to run a single unit test with ./sbt

Joel, Thank you! This is very helpful. What I notice now is that it works for Test classes that extend org.scalatest.junit.JUnit3Suite. There are other tests in the codebase that use a @Test annotation as in the example below. Any idea how to run those? import org.junit.{Test, After, Before}

Re: leader:none question

There is no way to delete topics in Kafka yet. So trying to delete topics like this might cause failures and cause the topics to be in a bad state. Thanks, Neha On Oct 28, 2013 4:46 AM, "linghongbo008" wrote: > Hi All > I encountered an error. I delete a certain topic and recreate it , but > w

leader:none question

Hi All I encountered an error. I delete a certain topic and recreate it , but when I run the shell "kafka-list-topic.sh ", the topic of all partitions "leader:none". I have no idea! Please tell me what happened. Thanks. topic: mqttmsglog partition: 0leader: nonereplicas: 11,