Hey Matan, At LinkedIn we end up having lots of topics--some are very high volume and some very very low volume. It works fine for low-volume topics.
At this time we don't support any kind of per-consumer throttling, that would be a nice thing to have. -Jay On Sun, Dec 16, 2012 at 2:23 AM, Matan Safriel <ma...@cloudaloe.org> wrote: > Hi, > > I've read about Kafka being used for huge paces and volumes of messages. My > scenario is a bit different, I'm looking for mostly only persistence of a > message queue, where the queue is usually not huge. > This scenario means that messages are not typically queued for long. > > Why Kafka? What I like about Kafka is that on those occasions where > messages do accumulate beyond normal (e.g. when the receiving side is not > accessible), then things are well behaved, as the messages are queued on > disk. And that it materializes a form of resilient persistency leaner than > a database. > > I'd keenly appreciate your feedback twofold. First and foremost, have you > used or observed Kafka in low-traffic scenarios, is it efficient in memory > footprint when volume is low etc, or do you foresee any prominent problem > in that type of scenario? > > Secondly, assume that once in a while, messages do accumulate in large > amounts on the queue, such as due to a network issue, and then when the > receiving end is ready to consume them, can bandwidth utilization be capped > such that the network isn't congested by that event. > > Thanks in advance, > Matan >
