Thanks David!. We are not using confluent at the moment .Since the work
that needs to be done by each consumer is the same (read > write to hdfs)
I am guessing my consumer code will just look the same and will need just
one consumer group.
Thanks,
Nishanth
On Tue, Sep 12, 2017 at 8:53 AM, D
Consumers can be split up based on partitions. So, you can tell a consumer
group to listen to several topics and it will divvy up the work. Your use case
sounds very canonical. I would take a look at Kafka connect (if you’re using
the confluent stack).
-Daivd
http://docs.confluent.io/curren
All,
I am very new to kafka . We have a case where we need to ingest multiple
avro record types . These avro record types vary vastly in volume and size
and I am thinking of sending each of these message types to a different
topic and creating partitions based on volume and through put needed.
