Re: Non-blocking Kafka stream iterators

2013-01-20 Thread navneet sharma
Is there any property to make consumer work for lets say only 10 mins (ie some kind of timer) So, i want to close the consumer after 10 mins reading from broker.. Thanks, Navneet Sharma On Mon, Jan 21, 2013 at 11:48 AM, Ryan LeCompte wrote: > Perfect. Thanks Jun! > > Ryan > > > On Sun, Jan 20,

Re: Non-blocking Kafka stream iterators

2013-01-20 Thread Ryan LeCompte
Perfect. Thanks Jun! Ryan On Sun, Jan 20, 2013 at 10:16 PM, Jun Rao wrote: > Yes, see consumer.timeout.ms in http://kafka.apache.org/configuration.html > > Thanks, > > Jun > > On Sun, Jan 20, 2013 at 7:14 PM, Ryan LeCompte wrote: > > > Hello all, > > > > Does Kafka still only support blocking

Re: Non-blocking Kafka stream iterators

2013-01-20 Thread Jun Rao
Yes, see consumer.timeout.ms in http://kafka.apache.org/configuration.html Thanks, Jun On Sun, Jan 20, 2013 at 7:14 PM, Ryan LeCompte wrote: > Hello all, > > Does Kafka still only support blocking stream iterstors? It would be great > to pass a timeout or have a poll() operation for fetching i

Re: Consumer Question

2013-01-20 Thread Jun Rao
When the consumption stops in GroupA, were there any error/exception in the consumer logic? One common cause for a consumer to stop is that the application code hits an exception while consuming an event. In particular, if you are using java thread pool and haven't attached an exception handler, t

Non-blocking Kafka stream iterators

2013-01-20 Thread Ryan LeCompte
Hello all, Does Kafka still only support blocking stream iterstors? It would be great to pass a timeout or have a poll() operation for fetching items. Right now I'm always blocking in this call: for (m <- stream) ... Thanks! Ryan

Re: Consumer Question

2013-01-20 Thread Bo Sun
0.7.1 thanks On Thu, Jan 17, 2013 at 11:26 PM, Jun Rao wrote: > What version of Kafka are you using? > > Thanks, > > Jun > > On Wed, Jan 16, 2013 at 10:10 PM, Bo Sun wrote: > > > I'v got a problem like this. > > 1. I use the groupname "GourpA" to consume the kafka topic "topicA" . > > several d

Re: state of replication in 0.8

2013-01-20 Thread Jun Rao
Stan, Quite a few people have been asking the same question. I created this wiki ( https://cwiki.apache.org/confluence/display/KAFKA/Changes+in+Kafka+0.8) to document the significant changes in 0.8. The replication design in 0.8 leveraged some ideas in the paper that you mentioned, but is not the