Hello Erik, Are you referring to the java doc for KafkaConsumer ( http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html )?
Currently the description of the timeout value is "timeout - The time, in milliseconds, spent waiting in poll if data is not available. If 0, returns immediately with any records that are available now. Must not be negative." We can clarify its behavior a bit more clear if people find it still confusing. Guozhang On Thu, Nov 26, 2015 at 6:34 AM, Erik Pragt <erik.pr...@jworks.nl> wrote: > Hi Jay, > > Thanks for the quick response. That was not the behavior I expected when > doing a poll(100), but I understand how it works now. I'm not sure if > updating the documentation would have prevented my confusion, but maybe > adding the "return the instant there are new messages, but if no messages > come return after the specified timeout passes" would have helped :-) > > Thanks! Erik > > On Thu, Nov 26, 2015 at 3:02 PM, Jay Kreps <j...@confluent.io> wrote: > > > Actually this may be something we can improve in the documentation. > Calling > > poll(1000) doesn't mean "check for new messages every second" but rather > > "return the instant there are new messages, but if no messages come > return > > after a one second timeout passes". > > > > So in that respect both the old and new consumer are neither polling nor > > streaming but rather a kind of "long poll" if you're familiar with that > > term. > > > > -Jay > > > > On Thursday, November 26, 2015, Erik Pragt <erik.pr...@jworks.nl> wrote: > > > > > Hi all, > > > > > > We're using Apache Kafka 0.9, and in our 0.8.2 Kafka, we had a consumer > > > which used a ConsumerConnector to stream all the messages to be > > processed. > > > > > > The new KafkaConsumer seems to use a polling mechanism instead of > > > streaming, and I was wondering: what is the 'right' way to go forward? > > > Should I poll with 0 millis? Should I poll every 100ms and accept the > > > polling delay? Should I stick to the previous way and keep using the > > > ConsumerConnector to stream the messages per topic? > > > > > > I'm just wondering what the best way forward is, and I hope somebody > can > > > give me a bit of advice in this. > > > > > > Thanks, > > > > > > Erik > > > > > > -- -- Guozhang
