Wow, that seems like an anti-pattern. Replication itself should be enough to resurrect the cluster in case of node failures. Technically, you shouldn't have to maintain the same broker id. There must be something else going on with replication.
On Thu, Nov 15, 2018 at 12:28 AM Andrey Dyachkov <andrey.dyach...@gmail.com> wrote: > You can attach EBS volume, which will store data and metadata(e.g. broker > id), and then attach it to the new AWS instance and start Kafka, it will > pick the broker id plus you won’t need to rebalance the cluster. > > On Wed 14. Nov 2018 at 19:48, naresh Goud <nareshgoud.du...@gmail.com> > wrote: > > > Static IP. Buying static IP may help. I am not aws expert > > > > On Wed, Nov 14, 2018 at 12:47 PM Srinivas Rapolu <cnu.t...@gmail.com> > > wrote: > > > > > Hello Kafka experts, > > > > > > We are running Kafka on AWS, main question is what is the best way to > > > retain broker.id on new instance spun-up in-place of instance/broker > > > failed. > > > > > > We are currently running Kafka in AWS with broker.id gets auto > > generated. > > > But we are having issues when a broker is failed, new broker/instance > > > spun-up in AWS get assigned with new broker.id. The issue is, with > this > > > approach, we need to re-assign the topics/replications on to the new > > broker > > > manually. > > > > > > We learned that, replication can be auto resolved by Kafka, if we can > > > manage to get the same broker.id on the new AWS instance spun-up > > in-place > > > of failed broker/instance. > > > > > > I have read, we can set broker.id.generation.enable= false, but what is > > the > > > best way to identify and retain the broker.id? Any links/help is > > > appreciated. > > > Thanks and Regards, > > > Cnu > > > > > -- > > Thanks, > > Naresh > > www.linkedin.com/in/naresh-dulam > > http://hadoopandspark.blogspot.com/ > > > -- > Thanks, > Andrey > -- Thanks and Regards, Amit Pal
