[jira] [Commented] (KAFKA-828) Preferred Replica Election does not delete the admin path on controller failover
[
https://issues.apache.org/jira/browse/KAFKA-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615355#comment-13615355
]
Jun Rao commented on KAFKA-828:
---
Thanks for patch v3. A couple of comments:
1. There is a compilation error
[error]
/Users/jrao/intellij_workspace/kafka_git/core/src/main/scala/kafka/controller/PartitionLeaderSelector.scala:129:
type mismatch;
[error] found : Unit
[error] required: (kafka.api.LeaderAndIsr, Seq[Int])
[error] info("Preferred replica %d is already the current leader for
partition %s".format(preferredReplica, topicAndPartition))
[error] ^
[error] one error found
2. PreferredReplicaPartitionLeaderSelector: Is there any value in logging the
following?
info("Preferred replica %d is already the current leader for partition
%s".format(preferredReplica, topicAndPartition))
The same info will be logged in
KafkaController.removePartitionsFromPreferredReplicaElection().
3. ZkUtils.parsePreferredReplicaElectionData: This is very specific to
preferred replica election. So, it's better to move it to
PreferredReplicaLeaderElectionCommand.
> Preferred Replica Election does not delete the admin path on controller
> failover
>
>
> Key: KAFKA-828
> URL: https://issues.apache.org/jira/browse/KAFKA-828
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.8
>Reporter: Swapnil Ghike
>Assignee: Swapnil Ghike
>Priority: Blocker
> Labels: bugs, kafka-0.8, p1
> Fix For: 0.8
>
> Attachments: kafka-828-v1.patch, kafka-828-v2.patch,
> kafka-828-v3.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-830: Attachment: kafka-830-v1.patch - Changed any replica list to a set. This makes sense since replica ids can never legitimately repeat. > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work stopped] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-830 stopped by Neha Narkhede. > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-830 started by Neha Narkhede. > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-830 started by Neha Narkhede. > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-830: Status: Patch Available (was: In Progress) > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-828) Preferred Replica Election does not delete the admin path on controller failover
[ https://issues.apache.org/jira/browse/KAFKA-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615416#comment-13615416 ] Neha Narkhede commented on KAFKA-828: - Agree with Jun on 1, 2 and 3. The issue with your change to the leader selector is that it doesn't achieve the objective of KAFKA-814 completely. If a preferred replica is already the leader, we *do not* want to update zookeeper and send the become leader request to the replica again. It should just continue to work on the next partition. It doesn't achieve that. One way of handling that is throwing a special exception LeaderAlreadyElectedException (or a better name) and then catching and swallowing it inside electLeaderForPartition() > Preferred Replica Election does not delete the admin path on controller > failover > > > Key: KAFKA-828 > URL: https://issues.apache.org/jira/browse/KAFKA-828 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Swapnil Ghike >Assignee: Swapnil Ghike >Priority: Blocker > Labels: bugs, kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-828-v1.patch, kafka-828-v2.patch, > kafka-828-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615448#comment-13615448 ] Jun Rao commented on KAFKA-830: --- Thanks for the patch. One problem with changing assigned replicas from a list to a set is that it breaks the preferred replica logic. We assume that the first replica is the preferred one. So the ordering of the replicas is important. Perhaps we should keep replicas as a list and make sure that we don't add duplicates when changing it. > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-830: Attachment: kafka-830-v2.patch Good catch! However, it doesn't make sense to keep it sequence since by definition those ids are unique. As for preferred replica logic, currently it is just the smallest replica id . So there are 2 options - 1. Store replica ids in a sorted set 2. For preferred replica, pick the smallest id from a traditional hash set But in patch v2, I stuck to your suggestion since it is a much smaller change for 0.8. Will fix this in 0.8.1 > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch, kafka-830-v2.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615586#comment-13615586 ] Neha Narkhede commented on KAFKA-830: - Actually, thinking about it more, even picking the smallest replica id does not make sense. This is because if we pick the first replica to be the largest broker id, the rest of the replica ids are smaller than the preferred replica in this case. We will have to keep this a sequence and be careful while adding a replica id to the assigned replica list > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch, kafka-830-v2.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615645#comment-13615645 ] Swapnil Ghike commented on KAFKA-830: - collection.mutable.LinkedHashSet will iterate the elements in the same sequence they were inserted. But using it will require changes at more places than the one line change in patch v2. > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch, kafka-830-v2.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615697#comment-13615697 ] Jun Rao commented on KAFKA-830: --- +1 on patch v2. > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch, kafka-830-v2.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-830: Resolution: Fixed Status: Resolved (was: Patch Available) > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch, kafka-830-v2.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-830. --- Thanks for the review > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch, kafka-830-v2.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-830) partition replica assignment map in the controller should be a Set
[ https://issues.apache.org/jira/browse/KAFKA-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615750#comment-13615750 ] Neha Narkhede commented on KAFKA-830: - Swapnil, thanks for pointing that out, we can remember to do this in 0.8.1 > partition replica assignment map in the controller should be a Set > -- > > Key: KAFKA-830 > URL: https://issues.apache.org/jira/browse/KAFKA-830 > Project: Kafka > Issue Type: Bug > Components: controller >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-830-v1.patch, kafka-830-v2.patch > > > partitionReplicaAssignment currently stores the list of assigned replicas as > a sequence. When a broker comes online, the replica state machine adds the > broker to the list of assigned replicas. It should do that only if the > replica is already not in the list of assigned replicas. This causes the > replication factor to be incorrectly calculated -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-819) System Test : Add validation of log segment index to offset
[ https://issues.apache.org/jira/browse/KAFKA-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-819: Attachment: kafka-819-v1.patch Uploaded kafka-819-v1.patch > System Test : Add validation of log segment index to offset > --- > > Key: KAFKA-819 > URL: https://issues.apache.org/jira/browse/KAFKA-819 > Project: Kafka > Issue Type: Task >Reporter: John Fung >Assignee: John Fung > Labels: kafka-0.8, replication-testing > Attachments: kafka-819-v1.patch > > > This can be achieved by calling DumpLogSegments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-819) System Test : Add validation of log segment index to offset
[ https://issues.apache.org/jira/browse/KAFKA-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-819: Status: Patch Available (was: Open) > System Test : Add validation of log segment index to offset > --- > > Key: KAFKA-819 > URL: https://issues.apache.org/jira/browse/KAFKA-819 > Project: Kafka > Issue Type: Task >Reporter: John Fung >Assignee: John Fung > Labels: kafka-0.8, replication-testing > Attachments: kafka-819-v1.patch > > > This can be achieved by calling DumpLogSegments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-819) System Test : Add validation of log segment index to offset
[ https://issues.apache.org/jira/browse/KAFKA-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615809#comment-13615809 ] John Fung commented on KAFKA-819: - This patch is tested with an older 0.8 branch (before kafka-804 checkin) to reproduce the mismatching index-offset: 2013-03-27 14:48:33,033 - DEBUG - executing command [ssh localhost /home/kafka/kafka_sanity_20130314_gen_invalid_index/bin/kafka-run-class.sh kafka.tools.DumpLogSegments --file /home/kafka/kafka_sanity_20130314_gen_invalid_index/system_test/replication_testsuite/testcase_0206/logs/broker-1/kafka_server_1_logs/test_1-2/.index --verify-index-only 2>&1] (kafka_system_test_utils) 2013-03-27 14:48:34,038 - DEBUG - error found [Mismatches in :.index] (kafka_system_test_utils) 2013-03-27 14:48:34,039 - DEBUG - [ Index offset: 755, log offset: 750] (kafka_system_test_utils) 2013-03-27 14:48:34,039 - DEBUG - [ Index offset: 635, log offset: 630] (kafka_system_test_utils) 2013-03-27 14:48:34,039 - DEBUG - [ Index offset: 515, log offset: 510] (kafka_system_test_utils) 2013-03-27 14:48:34,039 - DEBUG - [ Index offset: 395, log offset: 390] (kafka_system_test_utils) 2013-03-27 14:48:34,039 - DEBUG - [ Index offset: 275, log offset: 270] (kafka_system_test_utils) 2013-03-27 14:48:34,039 - DEBUG - [ Index offset: 125, log offset: 120] (kafka_system_test_utils) > System Test : Add validation of log segment index to offset > --- > > Key: KAFKA-819 > URL: https://issues.apache.org/jira/browse/KAFKA-819 > Project: Kafka > Issue Type: Task >Reporter: John Fung >Assignee: John Fung > Labels: kafka-0.8, replication-testing > Attachments: kafka-819-v1.patch > > > This can be achieved by calling DumpLogSegments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (KAFKA-829) Mirror maker needs to share the migration tool request channel
[ https://issues.apache.org/jira/browse/KAFKA-829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-829 started by Neha Narkhede. > Mirror maker needs to share the migration tool request channel > --- > > Key: KAFKA-829 > URL: https://issues.apache.org/jira/browse/KAFKA-829 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > > 0.8 mirror maker suffers from the same drawbacks that the original version of > the migration tool did. The consumer gets blocked if its producer queue is > full, even if there are other idle producers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-829) Mirror maker needs to share the migration tool request channel
[ https://issues.apache.org/jira/browse/KAFKA-829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-829: Status: Patch Available (was: In Progress) > Mirror maker needs to share the migration tool request channel > --- > > Key: KAFKA-829 > URL: https://issues.apache.org/jira/browse/KAFKA-829 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Attachments: kafka-829-v1.patch > > > 0.8 mirror maker suffers from the same drawbacks that the original version of > the migration tool did. The consumer gets blocked if its producer queue is > full, even if there are other idle producers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-829) Mirror maker needs to share the migration tool request channel
[ https://issues.apache.org/jira/browse/KAFKA-829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-829: Attachment: kafka-829-v1.patch - Mirror maker shares the producer data channel and producer thread from migration tool. - Not sure what is the best place to put producer data channel and producer thread since it is relevant only to migration tool and mirror maker. So left it inside migration tool for now - Same caveat as migration tool. If there are failures that causes producer threads to exit, the mirror maker will not shut down cleanly and will have to be killed > Mirror maker needs to share the migration tool request channel > --- > > Key: KAFKA-829 > URL: https://issues.apache.org/jira/browse/KAFKA-829 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Neha Narkhede >Priority: Blocker > Labels: kafka-0.8, p1 > Attachments: kafka-829-v1.patch > > > 0.8 mirror maker suffers from the same drawbacks that the original version of > the migration tool did. The consumer gets blocked if its producer queue is > full, even if there are other idle producers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-819) System Test : Add validation of log segment index to offset
[ https://issues.apache.org/jira/browse/KAFKA-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-819: -- Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available) Thanks for the patch. Committed to 0.8. > System Test : Add validation of log segment index to offset > --- > > Key: KAFKA-819 > URL: https://issues.apache.org/jira/browse/KAFKA-819 > Project: Kafka > Issue Type: Task >Reporter: John Fung >Assignee: John Fung > Labels: kafka-0.8, replication-testing > Fix For: 0.8 > > Attachments: kafka-819-v1.patch > > > This can be achieved by calling DumpLogSegments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-253) Refactor the async producer to have only one queue instead of one queue per broker in a Kafka cluster
[ https://issues.apache.org/jira/browse/KAFKA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13616045#comment-13616045 ] Jason Rosenberg commented on KAFKA-253: --- Was this change merged to the 0.7 branch? > Refactor the async producer to have only one queue instead of one queue per > broker in a Kafka cluster > - > > Key: KAFKA-253 > URL: https://issues.apache.org/jira/browse/KAFKA-253 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8 >Reporter: Neha Narkhede >Assignee: Jun Rao > Fix For: 0.8 > > Attachments: kafka-253.patch, kafka-253_v2.patch, kafka-253_v3.patch, > kafka-253_v4.patch, kafka-253_v5.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Today, the async producer is associated with a particular broker instance, > just like the SyncProducer. The Producer maintains a producer pool of > sync/async producers, one per broker. Since the producer pool creates one > async producer per broker, we have multiple producer queues for one Producer > instance. > With replication, a topic partition will be logical. This requires > refactoring the AsyncProducer to be broker agnostic. As a side effect of this > refactoring, we should also ensure that we have only one queue per Producer > instance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-828) Preferred Replica Election does not delete the admin path on controller failover
[ https://issues.apache.org/jira/browse/KAFKA-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-828: Attachment: kafka-828-v4.patch Agree with all comments, uploading patch v4. Defined a new LeaderElectionNotNeededException that is thrown in selectLeader() when the current leader is the same as the preferred replica. electLeaderForPartition() swallows it. Thanks for the reviews and sorry, I was quite lousy while including the patch for 814. > Preferred Replica Election does not delete the admin path on controller > failover > > > Key: KAFKA-828 > URL: https://issues.apache.org/jira/browse/KAFKA-828 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Swapnil Ghike >Assignee: Swapnil Ghike >Priority: Blocker > Labels: bugs, kafka-0.8, p1 > Fix For: 0.8 > > Attachments: kafka-828-v1.patch, kafka-828-v2.patch, > kafka-828-v3.patch, kafka-828-v4.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
