Alexis,
Yeah I agree with you the admin operations should be reasonably convenient,
for example in partition assignment, it should definitely do something more
clever than requiring a user-provided json. That is also something that the
community has been discussing and working on to improve.
Agai
I think the "scripts for operations" should be APIs that provides higher
level operations. It would be simpler and more powerful.
On Thu, Mar 10, 2016 at 10:54 AM Guozhang Wang wrote:
> Thanks Alexis, these are good points.
>
> We are aware of the partition assignment json problem and trying to
Thanks Alexis, these are good points.
We are aware of the partition assignment json problem and trying to improve
it; as for packaging / configs etc I think the goal of the OS Kafka package
itself is to provide universal and simple barebone scripts for operations,
and users can wrap / customize th
thank you Todd & Alexis!
I'll try to use the recommended Kafka tools
Regards,
On Mon, Mar 7, 2016 at 1:16 AM, Alexis Midon <
alexis.mi...@airbnb.com.invalid> wrote:
> also:
> . the command line tool packaging is not ideal, you have to download the
> (big) kafka zip, set paths, etc.
>
> . the de
also:
. the command line tool packaging is not ideal, you have to download the
(big) kafka zip, set paths, etc.
. the details of the zk cluster and kafka cluster must be repeated on the
command line.
with kafkat, you can use `gem install`, and a store the cluster details in
a config file.
On Sun
My recollection is that you have to come up with the partition assignment
yourself, and pass the json file as an argument.
This is quite error prone, especially during an outage.
we quickly wrote kafkat to have a simple commands that would let us express
needs without having to deal with the assig
Hello Alexis,
Could you share your findings about the command line tool? We can try to
resolve if there's any issues.
Guozhang
On Fri, Mar 4, 2016 at 3:13 PM, Alexis Midon <
alexis.mi...@airbnb.com.invalid> wrote:
> The command line tool that ships with Kafka is error prone.
>
> Our standard pr
The command line tool that ships with Kafka is error prone.
Our standard procedure is:
1. spin up the new broker
2. use `kafkat drain [--brokers ]
3. shut down old broker
The `drain` command will generate and submit a partition assignment plan
where the new broker id replaces the old one. It's p
To answer your questions…
1 - Not in the way you want it to. There is a setting for automatic leader
election (which I do not recommend anyone use at this time), but all that
does is pick which of the currently assigned replicas should be the leader.
It does not reassign partitions from one broker
