I am consuming from kafka using KafkaSpout of Storm and also in ruby using ruby-kafka gem (both use new consumer API).
I noticed that after a rolling restart of the kafka cluster. The kafkaSpout reconsumed all kafka messages ignoring the committed offsets... What can cause this behavior ? Attached kafka logs at this time (storm consumers are storm_consumer_1 and storm_consumer_2 and ruby consumer is ruby_kafka_consumer) I see these many lines like these for storm consumer but not for ruby consumer: [2017-08-20 12:03:54,270] INFO [GroupCoordinator 0]: Group storm_consumer_2 with generation 52 is now empty (__consumer_offsets-48) (kafka.coordinator.group.GroupCoordinator) [2017-08-20 12:03:54,701] INFO [GroupCoordinator 0]: Group storm_consumer_2 with generation 56 is now empty (__consumer_offsets-48) (kafka.coordinator.group.GroupCoordinator)
