attached log file On Mon, Aug 21, 2017 at 11:06 AM, Elyahou Ittah <elyaho...@fiverr.com> wrote:
> I am consuming from kafka using KafkaSpout of Storm and also in ruby using > ruby-kafka gem (both use new consumer API). > > I noticed that after a rolling restart of the kafka cluster. The > kafkaSpout reconsumed all kafka messages ignoring the committed offsets... > > What can cause this behavior ? > > Attached kafka logs at this time (storm consumers are storm_consumer_1 and > storm_consumer_2 and ruby consumer is ruby_kafka_consumer) > > I see these many lines like these for storm consumer but not for ruby > consumer: > > [2017-08-20 12:03:54,270] INFO [GroupCoordinator 0]: Group > storm_consumer_2 with generation 52 is now empty (__consumer_offsets-48) > (kafka.coordinator.group.GroupCoordinator) > [2017-08-20 12:03:54,701] INFO [GroupCoordinator 0]: Group > storm_consumer_2 with generation 56 is now empty (__consumer_offsets-48) > (kafka.coordinator.group.GroupCoordinator) >
kafkalog.txt.gz
Description: GNU Zip compressed data
