Re: Fetch Response for invalid offset returns success.

You may have mis-understood the api. getOffsetsBefore() returns a valid offset before a given timestamp, not a given offset. Thanks, Jun On Fri, Feb 28, 2014 at 5:02 PM, Vadim Keylis wrote: > Hello Guozhang. The following is slightly modified version of the example. > Where offset can be prev

Re: Fetch Response for invalid offset returns success.

Hello Guozhang. The following is slightly modified version of the example. Where offset can be previously valid offset, but does not exist in kafka for one or the other valid reason. TopicAndPartition topicAndPartition = new TopicAndPartition(topic, partition); Map requestInfo = new HashMap

Re: New Consumer API discussion

1. "The new consumer API is non blocking and instead of returning a blocking iterator, the consumer provides a poll() API that returns a list of records. " So this means the consumer polls, and if there are new messages it pulls them down and then disconnects? Not really. The point I was trying t

Re: Fetch Response for invalid offset returns success.

Hi Vadim, Which example in the kafka doc are you following? Guozhang On Fri, Feb 28, 2014 at 3:18 PM, Vadim Keylis wrote: > Good afternoon. I am writing my own client that is based on simple > consumer. I implemented method that is based on example in the kafka > documents that sends request

Re: ConsumerRebalanceFailedException

This actually doesn't look like the entire log. Could you attach your entire log in pastebin.com or something similar? I'm looking for something like this in your log - "Rebalancing attempt failed... " Thanks, Neha On Fri, Feb 28, 2014 at 12:44 PM, Yu, Libo wrote: > Here is a snapshot of our l

Re: Fetch Response for invalid offset returns success.

I am using Kafka 0.8 On Fri, Feb 28, 2014 at 4:30 PM, Neha Narkhede wrote: > What version of Kafka are you using? > > > On Fri, Feb 28, 2014 at 3:18 PM, Vadim Keylis > wrote: > > > Good afternoon. I am writing my own client that is based on simple > > consumer. I implemented method that is base

Re: Fetch Response for invalid offset returns success.

What version of Kafka are you using? On Fri, Feb 28, 2014 at 3:18 PM, Vadim Keylis wrote: > Good afternoon. I am writing my own client that is based on simple > consumer. I implemented method that is based on example in the kafka > documents that sends request to kafka to find an offset. I am t

Re: Kafka Sync vs. Async etc.

a. Sync producer b. request.required.acks = -1 c. compression.codec="gzip" with send(List) I'm not quite sure what you meant by 2, but committed all or nothing can somewhat be achieved by placing all messages in a batch in one compressed message. So I suggested c) above. Thanks, Neha On Fri, F

Kafka Sync vs. Async etc.

Hi: If someone's intention is to implement a producer such that: 1. The call to send() blocks until the write is successful on the configured number or replicas or max num of retries. 2. The call to send() takes in a configurable batch of messages that is committed all or nothing. what combinat

Fetch Response for invalid offset returns success.

Good afternoon. I am writing my own client that is based on simple consumer. I implemented method that is based on example in the kafka documents that sends request to kafka to find an offset. I am trying to find offsets after given offset. The api returns success even those offset does not exist b

RE: ConsumerRebalanceFailedException

Here is a snapshot of our logs. We've known that our three brokers somehow were offline and that caused the exception. key LOAN.SMC.134096888 takes 2 ms [06/02/14 08:54:06:006 AM EST] 198 INFO consumer.kafkaConsumerImpl: commitOddLots happen, topicName = credit.cache.smc.debt .topic, selector =

Re: New Consumer API discussion

Few clarifications: 1. "The new consumer API is non blocking and instead of returning a blocking iterator, the consumer provides a poll() API that returns a list of records. " So this means the consumer polls, and if there are new messages it pulls them down and then disconnects? 2. " The consum

Re: Puppet module for deploying Kafka released

Andrew, It's will be great to add a link to your prebuilt deb patch on your repo! ;) What should I do if I am to build the debian package my self? François Langelier Étudiant en génie Logiciel - École de Technologie Supérieure Capitaine Club Capra