Hi Team, I am exploring kafka 0.8 beta release to understand data flow, replication features. While testing i found that, the partition data for data for deleted topic is preseved in kafka-logs, why this behavior? suppose below case,
A topic (suppose test1) is created with partition 6 and replication 3 on a system with 4 brokers, respective log and index files will be prepared per partition in kafka-logs. If I delete the topic and recreate the same topic ‘test1’ after some time with partition 2 and replication 2. The kafka-logs directory seems to be confusing to understand why the partitions for previous topic are present. *Please help to understand this scenario*. Also, while testing the replication and leader selection feature observed leader -1 status, Original status: topic: test1 partition: 0 leader: 4 replicas: 4,2,3 isr: 4,2,3 topic: test1 partition: 1 leader: 0 replicas: 0,3,4 isr: 0,3,4 topic: test1 partition: 2 leader: 1 replicas: 1,4,0 isr: 1,4,0 if leader 4 goes down: topic: test1 partition: 0 leader: 2 replicas: 4,2,3 isr: 2,3 topic: test1 partition: 1 leader: 0 replicas: 0,3,4 isr: 0,3,4 topic: test1 partition: 2 leader: 1 replicas: 1,4,0 isr: 1,0,4 if leader 2 goes down: topic: test1 partition: 0 leader: 3 replicas: 4,2,3 isr: 3 topic: test1 partition: 1 leader: 0 replicas: 0,3,4 isr: 0,3,4 topic: test1 partition: 2 leader: 1 replicas: 1,4,0 isr: 1,0,4 if again leader 3 goes down: topic: test1 partition: 0 leader: -1 replicas: 4,2,3 isr: topic: test1 partition: 1 leader: 0 replicas: 0,3,4 isr: 0,4 topic: test1 partition: 2 leader: 1 replicas: 1,4,0 isr: 1,0,4 As per kafka protocol guide, *leader: -1 means If no leader exists because we are in the middle of a leader election this id will be -1.* *Does it mean that, the data from partition 0 will be unavailable due no leader (leader selection in progress)?* As per my understanding, can we have auto re-balancer facility to re-balance the partition replications to available brokers if one of the broker is down, as in above case of (if leader 4 goes down), we can replicate the partition 0 to broker 0/1 to re-balance the replication. Please correct me for any wrong understanding as those are my initial observations. Thanks in advance. Thanks, Yogesh Sangvikar
