Re: Kafka 0.8.2 Consumer Poll Mechanism

2016-02-10 Thread Joe San
any takers for my question? I will anyways try it out today! On Wed, Feb 10, 2016 at 11:32 PM, Joe San wrote: > So basically what I'm doing is the following: > > 1. I'm checking if the time to read the stream has lapsed. If yes, then I > come out of the recursion. > 2. If not, I read the stream

Re: Kafka 0.8.2 Consumer Poll Mechanism

2016-02-10 Thread Joe San
So basically what I'm doing is the following: 1. I'm checking if the time to read the stream has lapsed. If yes, then I come out of the recursion. 2. If not, I read the stream by getting the next element in the iterator What will be the offset that will be committed when I do consumer.commitOffs

Kafka 0.8.2 Consumer Poll Mechanism

2016-02-10 Thread Joe San
I tried to mimic the poll method that we have with the new consumer API in the 0.9.0.0 version. Here is what I have: def readFromKafka() = { val streams = consumerStreamsMap.get(consumerConfig.topic) @tailrec def poll(pollConfig: PollConfig, messages: Seq[String]): Seq[String] = { val i