Hello, I am testing kafka 0.8.1.1 in preparation for an upgrade from kafka-0.8.1-beta. I have a 4 node cluster with one broker per node, and a topic with 8 partitions and 3 replicas. Each partition has about 6 million records.
I generated a partition reassignment json that basically causes all partitions to be shifted by one broker. As the reassignment was in progress I bounced one of the servers. After the server came back up and the broker started, I waited for the server logs to stop complaining and then ran the reassignment verify script and all partitions were verified as completed reassignment. However, one of the partition offsets was reset to 0, and 4 out of 8 partitions only had 2 in-sync-replicas instead of 3 (in-sync came back to 3 but only after I again bounced the server I had previously bounced during reassignment). Is this considered a bug? I ask because we use the SimpleConsumer API so we keep track of our own offset "pointers". If it is not a bug then I could reset the pointer to "earliest" and continue reading, but I was wondering if there is a potential for data loss in my scenario. I have plenty of logs and can reproduce but before I spam I was wondering if there is already a jira task for this issue or if anybody else is aware. Thanks, Luke Forehand | Networked Insights | Software Engineer
