High level consumer stop reading data

Hi All, We are running kafka-0.8, If kafka node machine's network is restarted or lost for some time, then high level consumer stop reading data from kafka even after the network is restarted/working. -- *Thanks & Regards* *Hanish Bansal*

0.8 Head: Simple consumer no receiving messages

Hi, I see the following scenario: 1. Send messages under some topic X, able to see the log folder in Kafka Broker with name X-0 (Zeroth partition) and having files xxx.log and xxx.index under them. So guess this is fine 2. THen I fire up the consumer for topic X, it is able to find two streams

Re: Ganglia Metrics Reporter

Hi Maxime, I'm using this at the Wikimedia Foundation to send Kafka Broker metrics to Ganglia. However, we use Ganglia in multicast mode. This mostly seems to work with your code, but the ttl on the multicast packets gets set at 1. We have sometimes have multiple levels of ganglia aggregator

Re: Ganglia Metrics Reporter

Hi Andrew, how do you plan to use metrics 3.0.1? The current Kafka 0.8 version uses 2.2 AFAIK so this is going to require a new Kafka version. I don't really use multicast TTL muself but you are right that this should be configurable and I'll definitely accept pull request going in that direction.

Re: Controlled shutdown failure, retry settings

Here's another exception I see during controlled shutdown (this time there was not an unclean shutdown problem). Should I be concerned about this exception? Is any data loss possible with this? This one happened after the first "Retrying controlled shutdown after the previous attempt failed..." m

Re: Controlled shutdown failure, retry settings

I've filed: https://issues.apache.org/jira/browse/KAFKA-1108 On Tue, Oct 29, 2013 at 4:29 PM, Jason Rosenberg wrote: > Here's another exception I see during controlled shutdown (this time there > was not an unclean shutdown problem). Should I be concerned about this > exception? Is any data lo

Re: Recovering High Level Consumer from exceptions

Thanks. i ended up catching any exception in the consumer thread and retrying the iteration unless i specifically toggled it to dump and quit on error. Sent from my Droid Charge on Verizon 4G LTE Jun Rao wrote: Normally, consumers don't timeout unless you have configured it. You just need to make

Robust of cassandra

Hi, Everyone For now, I have a kafka cluster, and there is only 1 partition in each node. i just also produce lots of messages to 1 topic only. Then, I have a problem, I used it for a while, the topic is crashed, the error message is "NoAvailableLeader", but it is good for other topics that I do no

Re: Robust of cassandra

What is the replication factor and have you checked if there are leader elections around the time you see the NoAvailableLeader exception? It is important to figure out the root cause of NoAvailableLeader, but it is transient and should fix itself. Are you using 0.8 HEAD ? On Tue, Oct 29, 2013 at

Plan for Scala 2.10+ support

Hi, We are building pub-sub applications over kafka in our company. Some of our packages have been build on Scala 2.10+. Though only Kafka_2.9.2 is available on maven repository. Kafka_2.9.2 uses some deprecated scala classes (viz. ClassManifest), which causes runtime exception for our applicatio

Re: Plan for Scala 2.10+ support

The latest 0.8 branch has support for scala 2.10. We use it in our projects. Once 0.8 is released, I believe you should be able to see 2.10 artifacts in maven repository. Are you using 0.7 or 0.8? On 30 October 2013 08:58, Abhinav Anand wrote: > Hi, > We are building pub-sub applications over

Re: Plan for Scala 2.10+ support

Are there any public maven repos hosting 0.8 with 2.10+ support.? Sent from my iPhone > On Oct 29, 2013, at 8:32 PM, Aniket Bhatnagar > wrote: > > The latest 0.8 branch has support for scala 2.10. We use it in our > projects. Once 0.8 is released, I believe you should be able to see 2.10 > ar

Re: Plan for Scala 2.10+ support

I think there was the plan to make kafka producer and consumer pure in Java, so scala version wouldn't matter. And I think that's mostly why people want certain scala version, not because of kafka itself, but they just need producer/consumer libraries. On Tue, Oct 29, 2013 at 8:32 PM, Aniket Bhat

Re: Plan for Scala 2.10+ support

I haven't been able to find one which we why had to build Kafka 0.8 from source. It's not super hard though to build Kafka. On 30 October 2013 09:07, chetan conikee wrote: > Are there any public maven repos hosting 0.8 with 2.10+ support.? > > Sent from my iPhone > > > On Oct 29, 2013, at 8:32

Re: Plan for Scala 2.10+ support

For now from the 0.8 branch you can build what your producer/consumer build needs and publish to a local repository ./sbt "++2.10.2 publish-local" The 0.8.0 release vote is being held up with an error uploading/posting artifacts https://issues.apache.org/jira/browse/INFRA-6927 and once that is re

Re: Plan for Scala 2.10+ support

Thanks Joe. We have been maintaining an 0.8+2.10 release in our private repo for the past few months. > On Oct 29, 2013, at 8:46 PM, Joe Stein wrote: > > For now from the 0.8 branch you can build what your producer/consumer build > needs and publish to a local repository > > ./sbt "++2.10.2

Re: High level consumer stop reading data

Any exception/error from the consumer? Thanks, Jun On Tue, Oct 29, 2013 at 4:50 AM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > Hi All, > > We are running kafka-0.8, If kafka node machine's network is restarted or > lost for some time, then high level consumer stop reading data f

Re: Re: leader:none question

We forgot to delete that script from the 0.8 beta distro. Since we don't have a way to delete the topic, the cleanest way to come out of this would be to wipe out the cluster and start fresh. If that's too much overhead, you can try bringing the cluster down, deleting the /brokers/topics/ data from

Re: Cannot start Kafka 0.8, exception in leader election

Guozhang, In this case, it seems like the controller is trying to talk to itself as the controller establishes a channel with every broker in the cluster. Thanks, Neha On Mon, Oct 28, 2013 at 4:26 PM, Guozhang Wang wrote: > Hello Nicholas, > > The log shows the controller cannot connect with

Re: High level consumer stop reading data

No, it doesn't throw any exception. Consumer goes to halt state and after restarting it again starts consuming the data. On Wed, Oct 30, 2013 at 9:37 AM, Jun Rao wrote: > Any exception/error from the consumer? > > Thanks, > > Jun > > > On Tue, Oct 29, 2013 at 4:50 AM, Hanish Bansal < > hanish.b