Re: Repost: Busy CPU while negotiating contentBuffer size at BoundedByteBufferReceive.scala:54

You can enable some trace/debug level logging to see if the thread is indeed hanging in BoundedByteBufferReceive. Thanks, Jun On Wed, Sep 24, 2014 at 8:30 AM, Jagbir Hooda wrote: > Hi Jun, > > Thanks for looking into it. We use the following steps to start the > consumer. > > 1) Create consume

Re: Questions re: partition reassignment

Okay, so just to clarify, if I have a partition where the leader is broker 0, the ISR is [0, 1] and I make a partition reassignment with a new AR list of [1, 0], broker 1 won't take over leadership? I was under the impression that the "preferred" replica would become the leader, and that that would

Re: Producer errors (failed to send producer request, failed to send requests for topics)

Kyle, the new producer will handle this exception with the specific exception type, and the callback handling function can treat it accordingly. Could you give the new producer client a try and see if that is better handled now? On Tue, Sep 23, 2014 at 8:30 PM, Kyle Banker wrote: > Thanks so muc

Re: Replay Strategies

The consumer iterator returns MessageAndMetadata which includes the offset. You would need logic in your application to check the offset if it needs to stop processing after a certain offset. On Wed, Sep 24, 2014 at 12:09:40PM -0600, Kyle Banker wrote: > What are the best ways to replay a portion

Re: JMX on Kafka 0.8.1

0.8.1.1 I attached some examples thanks On Wed, Sep 24, 2014 at 5:43 PM, Neha Narkhede wrote: > That is odd. Which version of Kafka are you using? > > On Wed, Sep 24, 2014 at 5:37 PM, Jinder Aujla > wrote: > > > Hi > > > > I'm trying to get some JMX stats from a running Kafka instance, I can

Re: Producer errors (failed to send producer request, failed to send requests for topics)

Kyle, We have developed a new (pure java) producer in trunk. It should have better error messaging. Could you give it a try and see if it points out the problem clearer to you? Thanks, Jun On Tue, Sep 23, 2014 at 8:30 PM, Kyle Banker wrote: > Thanks so much, Jun. That seems to have fixed the

Re: Questions about Kafka 0.9 API changes

Valentin, As Guozhang mentioned, to use the new consumer in the SimpleConsumer way, you would subscribe to a set of topic partitions and the issue poll(). You can change subscriptions on every poll since it's cheap. The benefit you get is that it does things like leader discovery and maintaining c

Re: JMX on Kafka 0.8.1

That is odd. Which version of Kafka are you using? On Wed, Sep 24, 2014 at 5:37 PM, Jinder Aujla wrote: > Hi > > I'm trying to get some JMX stats from a running Kafka instance, I can > connect using jconsole and I can see various topic specific information, I > can see the "..LogEndOffset" value

JMX on Kafka 0.8.1

Hi I'm trying to get some JMX stats from a running Kafka instance, I can connect using jconsole and I can see various topic specific information, I can see the "..LogEndOffset" value changing as I add data, but for some reason the value for "kafka.server":type="BrokerTopicMetrics",name="AllTopics

Re: BadVersion state in Kafka Logs

>From the logs you've attached, my guess is it's most likely due to KAFKA-1382. Thanks, Neha On Wed, Sep 24, 2014 at 10:48 AM, Seshadri, Balaji wrote: > Hi, > > > > We got the below error in our logs and our consumers stopped consuming any > data ?.It worked only after restart. > > > > We would

Re: Connection reset by peer

Hello, With reference to this[1] discussion, I am facing a similar issue with the following stack trace interchangeably with broken pipe: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.ja

Replay Strategies

What are the best ways to replay a portion of a topic in the past? I'm using Kafka to process a real-time data stream. Suppose I discover that all data inside of a six-hour window one day ago was incorrectly processed. Since I'm checkpointing offsets in an external database, I can create a new co

RE: BadVersion state in Kafka Logs

Please find the log attached. From: Seshadri, Balaji [mailto:balaji.sesha...@dish.com] Sent: Wednesday, September 24, 2014 11:48 AM To: 'users@kafka.apache.org' Subject: BadVersion state in Kafka Logs Hi, We got the below error in our logs and our consumers stopped consuming any data ?.It worke

BadVersion state in Kafka Logs

Hi, We got the below error in our logs and our consumers stopped consuming any data ?.It worked only after restart. We would like to confirm that it's because we are running with 0.8-beta version and not 0.8 release version to convince "THE MGMT" guys. Please let me know if it's this KAFKA-138

Re: Repost: Busy CPU while negotiating contentBuffer size at BoundedByteBufferReceive.scala:54

Hi Jun, Thanks for looking into it. We use the following steps to start the consumer. 1) Create consumer connector ConsumerConnector consumerConnector = kafka.consumer.Consumer.createJavaConsumerConnector(createConsumerConfig()) 2) Create message streams List> streamList = consumerConnecto