Hi, I've read about Kafka being used for huge paces and volumes of messages. My scenario is a bit different, I'm looking for mostly only persistence of a message queue, where the queue is usually not huge. This scenario means that messages are not typically queued for long.
Why Kafka? What I like about Kafka is that on those occasions where messages do accumulate beyond normal (e.g. when the receiving side is not accessible), then things are well behaved, as the messages are queued on disk. And that it materializes a form of resilient persistency leaner than a database. I'd keenly appreciate your feedback twofold. First and foremost, have you used or observed Kafka in low-traffic scenarios, is it efficient in memory footprint when volume is low etc, or do you foresee any prominent problem in that type of scenario? Secondly, assume that once in a while, messages do accumulate in large amounts on the queue, such as due to a network issue, and then when the receiving end is ready to consume them, can bandwidth utilization be capped such that the network isn't congested by that event. Thanks in advance, Matan
