How to debug why partition reassignment failure?

2017-05-30 Thread Kerry Wei
Hi, I'm using kafka-reassign-partitions.sh to move partitions around, however, sometimes I got partition reassignment failure. The cluster is healthy before the rebalance, and a retry after 10 mins resolved the problem. However, I wonder if there's a way I can check why the reassignment failed for

Reassign Partitions Programmatically

2017-05-12 Thread Kerry Wei
Hi there, Can't find Java API to do partition reassignment, so i took a look at the source code (not a deep look). It seems that the kafka-reassign-partitions.sh script created the znode /admin/reassign_partitions with the new plan (in JSON) as the data. I wonder if I could post data to zookeeper i

Rebalance and Failures

2016-07-19 Thread Kerry Wei
Hi all, Bit confused on rebalance and failures: (if understand correctly about rebalance procedure) Suppose during the middle of the rebalance, some consumer, C1, hits an unclean shutdown (i.e. crashes, or kill -9), and the coordinator won't be aware that C1 is dead until {zookeeper.session.timeou