Subject=0.9.0 Client - Streaming vs Polling

2016-05-05 Thread Matthew Hersee
What did you conclude? We also use the streaming methods (consumerConnector.createMessageStreams), I can't see what the equivalents are in 0.9 Cheers

Re: 0.9.0 Client - Streaming vs Polling

2015-11-26 Thread Guozhang Wang
Hello Erik, Are you referring to the java doc for KafkaConsumer ( http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html )? Currently the description of the timeout value is "timeout - The time, in milliseconds, spent waiting in poll if data is not ava

Re: 0.9.0 Client - Streaming vs Polling

2015-11-26 Thread Erik Pragt
Hi Jay, Thanks for the quick response. That was not the behavior I expected when doing a poll(100), but I understand how it works now. I'm not sure if updating the documentation would have prevented my confusion, but maybe adding the "return the instant there are new messages, but if no messages c

Re: 0.9.0 Client - Streaming vs Polling

2015-11-26 Thread Jay Kreps
Actually this may be something we can improve in the documentation. Calling poll(1000) doesn't mean "check for new messages every second" but rather "return the instant there are new messages, but if no messages come return after a one second timeout passes". So in that respect both the old and ne

0.9.0 Client - Streaming vs Polling

2015-11-26 Thread Erik Pragt
Hi all, We're using Apache Kafka 0.9, and in our 0.8.2 Kafka, we had a consumer which used a ConsumerConnector to stream all the messages to be processed. The new KafkaConsumer seems to use a polling mechanism instead of streaming, and I was wondering: what is the 'right' way to go forward? Shoul