I don't have benchmarks but multiple consumer groups are possible. For Kafka
the performance should be similar or close to as having multiple consumers
using a single group.
_
From: Senthil Kumar
Sent: Tuesday, January 24, 2017 10:38 PM
Subject: Kaf
Why not just have a parallel consumer read all messages from whichever topics
you're interested in and store them wherever you want to? You don't need to
"backup" Kafka messages.
_
From: Aseem Bansal
Sent: Tuesday, December 6, 2016 4:55 PM
Subject: S
Do you have only one partition in the topic? The way Kafka works is that all
messages are first distributed into partitions in the topic and then the
consumers are distributed among them and they read them sequentially.
If you have only one partition in the topic, all your messages will be in it
I'd second Tom here. Create a ram disk or just let Kafka write to disk. Use
compression and batch messages and the OS fscache would take care of the rest.
Kafka is pretty fast and you probably won't notice.
_
From: Tom Crayford
Sent: Friday, July 1
This topic comes up often on this list. Kafka can be used as a datastore if
that’s what your application wants with the caveat that Kafka isn’t designed to
keep data around forever. There is a default retention time after which older
data gets deleted. The high level consumer essentially reads d
You can have the same consumer id and Kafka will balance partitions across the
two instances automatically. When one of them dies the partitions are
rebalanced and assigned to the remaining alive consumers.
_
From: Panda, Samaresh
Sent: Friday, June 5, 2015 7:
