[
https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086320#comment-14086320
]
Sriharsha Chintalapani commented on KAFKA-1070:
-----------------------------------------------
[~charmalloc] Thanks for the comments. Currently the approach allows users to
specify MaxBrokerId and kafka starts generating a sequence from MaxBrokerId +
1. If user specifies brokerId , kafka uses that instead of generating one from
zookeeper. In the current implementation we expect user configured broker.id
to be less than MaxBrokerId.
The tricky case here is if the user have a cluster where some of the brokers
have broker.id present in their config and others don't. In this case we use
MaxBrokerId (defaults to 1000) and start generating a sequenceId from there.
The issue with brokerId exception list is we do not know from which point to
generate a sequenceId.
In your example " in a cluster there may be broker.id that is 1721632 and
another 172164875 and another 172162240 "
and user added another broker to the cluster without broker.id in config. If
the user specifies that MaxReserveBrokerId is 172164875 than we can assign +1
to that new broker. If we take the approach of looking at brokerIds that exists
in the cluster and generate a new one that doesn't exist or taken by a broker
already(exclusion list as you suggested) than the problem would be how do we
know what are all the broker.ids existed in the cluster one way to look at is
zookeeper but a broker registers at zk after it starts.
For ex: if a broker1 starts with 1721632 and broker2 starts with no broker.id
than we generate 1721633 from zk and there is no guarantee that another broker
has this same id in their config. Of course we can have a exclusion list of
brokerids in a config file but that feels cumbersome for the user to list out
all the brokerids.
> Auto-assign node id
> -------------------
>
> Key: KAFKA-1070
> URL: https://issues.apache.org/jira/browse/KAFKA-1070
> Project: Kafka
> Issue Type: Bug
> Reporter: Jay Kreps
> Assignee: Sriharsha Chintalapani
> Labels: usability
> Attachments: KAFKA-1070.patch, KAFKA-1070_2014-07-19_16:06:13.patch,
> KAFKA-1070_2014-07-22_11:34:18.patch, KAFKA-1070_2014-07-24_20:58:17.patch,
> KAFKA-1070_2014-07-24_21:05:33.patch
>
>
> It would be nice to have Kafka brokers auto-assign node ids rather than
> having that be a configuration. Having a configuration is irritating because
> (1) you have to generate a custom config for each broker and (2) even though
> it is in configuration, changing the node id can cause all kinds of bad
> things to happen.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
