RE: Consumer sensitive expiration of topic

code because i know what my consumers are doing. -Original Message- From: Gwen Shapira [mailto:gshap...@cloudera.com] Sent: Thursday, August 28, 2014 3:24 PM To: users@kafka.apache.org Subject: Re: Consumer sensitive expiration of topic Maybe I misunderstand the proposal, but it sounds li

Re: Consumer sensitive expiration of topic

r...@gmail.com] > Sent: Thursday, August 28, 2014 12:28 PM > To: users@kafka.apache.org > Subject: Re: Consumer sensitive expiration of topic > > Hey Dominique, > > What you describe makes sense, and it would certainly be possible for > the broker to more aggressively discard data

RE: Consumer sensitive expiration of topic

] Sent: Thursday, August 28, 2014 12:28 PM To: users@kafka.apache.org Subject: Re: Consumer sensitive expiration of topic Hey Dominique, What you describe makes sense, and it would certainly be possible for the broker to more aggressively discard data once it sees that the consumer has read it once

Re: Consumer sensitive expiration of topic

ich data > isn't relevant anymore without recreating it ? > > Thanks, > > -Original Message- > From: Neha Narkhede [mailto:neha.narkh...@gmail.com] > Sent: Wednesday, August 27, 2014 11:36 PM > To: users@kafka.apache.org > Subject: Re: Consumer sensitive expirati

RE: Consumer sensitive expiration of topic

hich data isn't relevant anymore without recreating it ? Thanks, -Original Message- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Wednesday, August 27, 2014 11:36 PM To: users@kafka.apache.org Subject: Re: Consumer sensitive expiration of topic Kafka is designed to maintai

Re: Consumer sensitive expiration of topic

l Message- > From: Prunier, Dominique [mailto:dominique.prun...@emc.com] > Sent: Friday, August 15, 2014 11:02 AM > To: users@kafka.apache.org > Subject: RE: Consumer sensitive expiration of topic > > Hi, > > Thanks for the answer. > > The topics themselves won&#x

RE: Consumer sensitive expiration of topic

Any idea on this usecase guys ? Thanks, -Original Message- From: Prunier, Dominique [mailto:dominique.prun...@emc.com] Sent: Friday, August 15, 2014 11:02 AM To: users@kafka.apache.org Subject: RE: Consumer sensitive expiration of topic Hi, Thanks for the answer. The topics

RE: Consumer sensitive expiration of topic

s nothing to do with the broker itself) but also set the expiration of the topic using the same offsets (which could be done less frequently). Does it make sense ? Thanks, -Original Message- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Thursday, August 14, 2014 8:10 PM T

Re: Consumer sensitive expiration of topic

On Thu, Aug 14, 2014 at 09:56:11PM +, Prunier, Dominique wrote: > Hi, > > I'm playing around with Kafka with the idea to implement a general purpose > message exchanger for a distributed application with high throughput > requirements (multiple hundred thousand messages per sec). > > In th

Re: Consumer sensitive expiration of topic

By design, Kafka stores data independent of the number of publishers or subscribers connecting to it. This provides high performance as the broker does not have to manage consumers and evict data based on the consumers position. This is one of the main reasons why Kafka is much more performance com