Hi,
Is there any configuration to allow the new client design group coordinator
recover after a crash?
I have a testing topology with 3 brokers and once the Group Coordinator
crash, the topic gets correctly re balanced, producer not affected, but the
consumer group stop receiving messages.
Monitoring the console output of each broker, I don't find any reference of
a new GroupCoordinator assignment. All consumers resume receiving messages
as soon I start the original group coordinator broker.
Client Config:
> private static Consumer<String, String> createFixMessageConsumer(int id) {
> Properties props = new Properties();
> props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
> "localhost:9092,localhost:9093,localhost:9094");
> props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");
> props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "1000");
> props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, "6100");
> props.put(ConsumerConfig.GROUP_ID_CONFIG, MYCONSUMERGROUP);
> props.put(ConsumerConfig.CLIENT_ID_CONFIG, id + "");
> props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");
> props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
> StringDeserializer.class.getName());
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
> StringDeserializer.class.getName());
> return new KafkaConsumer<>(props, new StringDeserializer(), new
> FixMessageDeserializer());
> }
Following the description of the consumer group while the coordinator is
online:
Command:
> > bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server
> localhost:9093 --group MYCONSUMERGROUP --describe
Output:
> Note: This will only show information about consumers that use the Java
> consumer API (non-ZooKeeper-based consumers).
> Error: Executing consumer group command failed due to The consumer group
> command timed out while waiting for group to initialize:
> [root@adroitdesktop kafka]# bin/kafka-consumer-groups.sh --new-consumer
> --bootstrap-server localhost:9093 --group MYCONSUMERGROUP --describe
> Note: This will only show information about consumers that use the Java
> consumer API (non-ZooKeeper-based consumers).
>
> TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
> LAG CONSUMER-ID HOST
> CLIENT-ID
> MYCONSUMERGROUP 1 1792 1792 0
> 1-5641eb49-2f4b-4b45-8315-d90802034234 /192.1.1.27
> 1
> MYCONSUMERGROUP 0 1828 1829 1
> 0-c4b92c8b-5827-4c62-898e-cf64dd0148c4 /192.1.1.27
> 0
> - - -
> - - 3-083be1ba-6484-473d-8b54-6d11a9d4533d
> /192.1.1.27 3
> - - -
> - - 2-6a25deb9-e90c-4a23-a3e9-879167330c5c
> /192.1.1.27 2
> [root@adroitdesktop kafka]# bin/kafka-consumer-groups.sh --new-consumer
> --bootstrap-server localhost:9093 --group FDOITAU_CONSUMER --describe
> Note: This will only show information about consumers that use the Java
> consumer API (non-ZooKeeper-based consumers).
>
> TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
> LAG CONSUMER-ID HOST
> CLIENT-ID
> MYCONSUMERGROUP 0 5484 5484 0
> 0-04a4a7e8-f206-43bd-8190-a611e1a2326b /192.1.1.27
> 0
> MYCONSUMERGROUP 1 5447 5447 0
> 1-25ee2bd2-66bc-40c8-b439-1cbc92e071cf /192.1.1.27
> 1
> - - -
> - -
> 3-7c612ba7-0f99-4c48-8a14-42a1efd8a3c7 /192.1.1.27
> 3
> - - -
> - -
> 2-aead1762-d4cc-4455-97e4-68a348940af6 /192.1.1.27
> 2
Regards,
Yamada
