Hi All, We are setting up kafka cluster on AWS. We are using cloud formation templates to bring up kafka broker service on AWS nodes in a auto scaling group. Kafka cluster size is same as auto scaling group size. We set broker.id same as AMI-LAUNCH-INDEX and start the service. This works fine during the launch for the first time.
Lets say for some reason one of the broker node’s go down and auto scaling group brings up another node, the AMI-LAUNCH-INDEX for the new node is always 0 even when node 1 or node 2 go down. This can have serious side effects with same broker id for multiple nodes in a kafka cluster. So I am wondering if there is a better way to handle broker ids in AWS auto scaling world? Any help is highly appreciated. I am also open to listen from folks who run kafka on AWS about their set up to handle ASG scenario. Thanks, Raj Tanneru Information contained in this e-mail message is confidential. This e-mail message is intended only for the personal use of the recipient(s) named above. If you are not an intended recipient, do not read, distribute or reproduce this transmission (including any attachments). If you have received this email in error, please immediately notify the sender by email reply and delete the original message.
