Hi Jay, Thanks for the quick response. That was not the behavior I expected when doing a poll(100), but I understand how it works now. I'm not sure if updating the documentation would have prevented my confusion, but maybe adding the "return the instant there are new messages, but if no messages come return after the specified timeout passes" would have helped :-)
Thanks! Erik On Thu, Nov 26, 2015 at 3:02 PM, Jay Kreps <j...@confluent.io> wrote: > Actually this may be something we can improve in the documentation. Calling > poll(1000) doesn't mean "check for new messages every second" but rather > "return the instant there are new messages, but if no messages come return > after a one second timeout passes". > > So in that respect both the old and new consumer are neither polling nor > streaming but rather a kind of "long poll" if you're familiar with that > term. > > -Jay > > On Thursday, November 26, 2015, Erik Pragt <erik.pr...@jworks.nl> wrote: > > > Hi all, > > > > We're using Apache Kafka 0.9, and in our 0.8.2 Kafka, we had a consumer > > which used a ConsumerConnector to stream all the messages to be > processed. > > > > The new KafkaConsumer seems to use a polling mechanism instead of > > streaming, and I was wondering: what is the 'right' way to go forward? > > Should I poll with 0 millis? Should I poll every 100ms and accept the > > polling delay? Should I stick to the previous way and keep using the > > ConsumerConnector to stream the messages per topic? > > > > I'm just wondering what the best way forward is, and I hope somebody can > > give me a bit of advice in this. > > > > Thanks, > > > > Erik > > >
