[GitHub] kafka pull request #4198: MINOR: make controller helper methods private

2017-11-08 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/4198 MINOR: make controller helper methods private You can merge this pull request into a Git repository by running: $ git pull https://github.com/onurkaraman/kafka make-controller-helper

[GitHub] kafka pull request #4189: KAFKA-6146: minimize the number of triggers enqueu...

2017-11-07 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/4189 KAFKA-6146: minimize the number of triggers enqueuing PreferredReplicaLeaderElection events We currently enqueue a PreferredReplicaLeaderElection controller event in

[GitHub] kafka pull request #4075: MINOR: reduce partition state machine debug loggin...

2017-10-16 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/4075 MINOR: reduce partition state machine debug logging PartitionStateMachine.electLeaderForPartition logs all partition states in the cluster. This leads to quadratic logging behavior since

[GitHub] kafka pull request #3860: KAFKA-5894: add the notion of max inflight request...

2017-09-14 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3860 KAFKA-5894: add the notion of max inflight requests to async ZookeepeeperClient ZookeeperClient is a zookeeper client that encourages pipelined requests to zookeeper. We want to add the notion

[GitHub] kafka pull request #2518: KAFKA-4747: add metrics for KafkaConsumer.poll

2017-09-11 Thread onurkaraman
Github user onurkaraman closed the pull request at: https://github.com/apache/kafka/pull/2518 ---

[GitHub] kafka pull request #3765: KAFKA-5642 [WIP]: Use async ZookeeperClient in Con...

2017-08-31 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3765 KAFKA-5642 [WIP]: Use async ZookeeperClient in Controller Synchronous zookeeper writes means that we wait an entire round trip before doing the next write. These synchronous writes are

[GitHub] kafka pull request #3427: KAFKA-5501 [WIP]: use async zookeeper apis everywh...

2017-06-24 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3427 KAFKA-5501 [WIP]: use async zookeeper apis everywhere Synchronous zookeeper writes means that we wait an entire round trip before doing the next write. With respect to the controller, these

[GitHub] kafka pull request #3413: KAFKA-5502: read current brokers from zookeeper up...

2017-06-22 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3413 KAFKA-5502: read current brokers from zookeeper upon processing broker change Dong Lin's testing of the 0.11.0 release revealed a controller-side performance regression in clusters with

[GitHub] kafka pull request #3144: KAFKA-5323: AdminUtils.createTopic should check to...

2017-05-25 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3144 KAFKA-5323: AdminUtils.createTopic should check topic existence upfront When a topic exists, AdminUtils.createTopic unnecessarily does N+2 zookeeper reads where N is the number of brokers. Here

[GitHub] kafka pull request #3122: KAFKA-5310: reset ControllerContext during resigna...

2017-05-22 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3122 KAFKA-5310: reset ControllerContext during resignation This ticket is all about ControllerContext initialization and teardown. The key points are: 1. we should teardown ControllerContext

[GitHub] kafka pull request #3071: KAFKA-5258: move all partition and replica state t...

2017-05-16 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3071 KAFKA-5258: move all partition and replica state transition rules into a map Today the PartitionStateMachine and ReplicaStateMachine defines and asserts the valid state transitions inline for

[GitHub] kafka pull request #3067: KAFKA-5175: Fix transient failure in ControllerInt...

2017-05-16 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3067 KAFKA-5175: Fix transient failure in ControllerIntegrationTest.testPreferredReplicaLeaderElection The transient failure came from the controller processing the preferred replica leader

[GitHub] kafka pull request #3038: KAFKA-5180: fix transient failure in ControllerInt...

2017-05-12 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3038 KAFKA-5180: fix transient failure in ControllerIntegrationTest.testControllerMoveIncrementsControllerEpoch The tests previously ignored the fact that the controller does not atomically create

[GitHub] kafka pull request #2999: KAFKA-5197: add a tool analyzing zookeeper client ...

2017-05-08 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2999 KAFKA-5197: add a tool analyzing zookeeper client performance across its various apis The raw zookeeper client offers various means of getting and setting znodes. It would be useful to have a

[GitHub] kafka pull request #2986: KAFKA-5099: Replica Deletion Regression from KIP-1...

2017-05-05 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2986 KAFKA-5099: Replica Deletion Regression from KIP-101 Replica deletion regressed from KIP-101. Replica deletion happens when a broker receives a StopReplicaRequest with delete=true. Ever since

[GitHub] kafka pull request #2927: KAFKA-5107: remove preferred replica election stat...

2017-04-27 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2927 KAFKA-5107: remove preferred replica election state from ControllerContext KAFKA-5028 moves the controller to a single-threaded model, so we would no longer have work interleaved between

[GitHub] kafka pull request #2853: KAFKA-5069: add controller integration tests

2017-04-13 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2853 KAFKA-5069: add controller integration tests Test the various controller protocols by observing zookeeper and broker state. You can merge this pull request into a Git repository by running

[GitHub] kafka pull request #2816: KAFKA-5028: convert kafka controller to a single-t...

2017-04-05 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2816 KAFKA-5028: convert kafka controller to a single-threaded event queue model The goal of this ticket is to improve controller maintainability by simplifying the controller's concur

[GitHub] kafka pull request #2748: MINOR: remove misleading scala setter in Controlle...

2017-03-28 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2748 MINOR: remove misleading scala setter in ControllerContext You can merge this pull request into a Git repository by running: $ git pull https://github.com/onurkaraman/kafka remove-scala

[GitHub] kafka pull request #2746: KAFKA-4959: remove controller concurrent access to...

2017-03-27 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2746 KAFKA-4959: remove controller concurrent access to non-threadsafe NetworkClient, Selector, and SSLEngine This brought down a cluster by causing continuous controller moves. ZkClient&#

[GitHub] kafka pull request #2678: KAFKA-4891 kafka.request.logger TRACE regression

2017-03-13 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2678 KAFKA-4891 kafka.request.logger TRACE regression Both the headers and requests have regressed to just show object ids instead of their contents from their underlying structs. I'm guessing

[GitHub] kafka pull request #2520: KAFKA-4749: fix join-time-max and sync-time-max Me...

2017-02-08 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2520 KAFKA-4749: fix join-time-max and sync-time-max MeasurableStat type GroupCoordinatorMetrics currently sets up join-time-max and sync-time-max incorrectly as a "new Avg()" MeasurableSt

[GitHub] kafka pull request #2518: KAFKA-4747: add metrics for KafkaConsumer.poll

2017-02-08 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2518 KAFKA-4747: add metrics for KafkaConsumer.poll KafkaConsumer heavily depends on KafkaConsumer.poll yet we don't have metrics directly associated with it. We probably want to ad

[GitHub] kafka pull request #2181: KAFKA-4453: add request prioritization

2016-11-28 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2181 KAFKA-4453: add request prioritization Today all requests (client requests, broker requests, controller requests) to a broker are put into the same queue. They all have the same priority. So a

[GitHub] kafka pull request #2177: KAFKA-3959: enforce offsets.topic.replication.fact...

2016-11-27 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/2177 KAFKA-3959: enforce offsets.topic.replication.factor Kafka brokers have a config called "offsets.topic.replication.factor" that specify the replication factor for the "__consumer

[GitHub] kafka pull request #1677: KAFKA-4004: NetworkReceive.complete() should not t...

2016-07-28 Thread onurkaraman
Github user onurkaraman closed the pull request at: https://github.com/apache/kafka/pull/1677 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #1677: KAFKA-4004: NetworkReceive.complete() should not t...

2016-07-28 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/1677 KAFKA-4004: NetworkReceive.complete() should not throw NullPointerException after partially reading the size NetworkReceive.readFrom can partially read the size and not allocate a buffer. A

[GitHub] kafka pull request #1484: KAFKA-3810: replication of internal topics should ...

2016-06-09 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/1484 KAFKA-3810: replication of internal topics should not be limited by replica.fetch.max.bytes From the kafka-dev mailing list discussion: [DISCUSS] scalability limits in the coordinator

[GitHub] kafka-site pull request: fix typo in site_name

2016-05-24 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka-site/pull/15 fix typo in site_name You can merge this pull request into a Git repository by running: $ git pull https://github.com/onurkaraman/kafka-site asf-site Alternatively you can review and

[GitHub] kafka pull request: KAFKA-3718: propagate all KafkaConfig __consum...

2016-05-17 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/1394 KAFKA-3718: propagate all KafkaConfig __consumer_offsets configs to OffsetConfig instantiation Kafka has two configurable compression codecs: the one used by the client (source codec) and the

[GitHub] kafka pull request: KAFKA-3661: fix NPE in o.a.k.c.c.RoundRobinAss...

2016-05-05 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/1326 KAFKA-3661: fix NPE in o.a.k.c.c.RoundRobinAssignor when topic metadata not found AbstractPartitionAssignor.assign has an ambiguous line in its documentation: > @param partitionsPerTo

[GitHub] kafka pull request: KAFKA-3494: add metric id to client mbeans

2016-05-04 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/1323 KAFKA-3494: add metric id to client mbeans KafkaConsumer and KafkaProducer mbeans currently only have a client-id granularity. Client-id represents a logical name for an application

[GitHub] kafka pull request: KAFKA-2391 [WIP]: add timeout to KafkaConsumer...

2015-10-07 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/283 KAFKA-2391 [WIP]: add timeout to KafkaConsumer blocking calls You can merge this pull request into a Git repository by running: $ git pull https://github.com/onurkaraman/kafka KAFKA-2391

[GitHub] kafka pull request: KAFKA-2557: separate REBALANCE_IN_PROGRESS and...

2015-09-18 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/222 KAFKA-2557: separate REBALANCE_IN_PROGRESS and ILLEGAL_GENERATION error codes You can merge this pull request into a Git repository by running: $ git pull https://github.com/onurkaraman

[GitHub] kafka pull request: KAFKA-2413: fix ConsumerCoordinator updateCons...

2015-08-06 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/122 KAFKA-2413: fix ConsumerCoordinator updateConsumer You can merge this pull request into a Git repository by running: $ git pull https://github.com/onurkaraman/kafka KAFKA-2413

[GitHub] kafka pull request: KAFKA-2397: leave group request

2015-08-02 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/103 KAFKA-2397: leave group request You can merge this pull request into a Git repository by running: $ git pull https://github.com/onurkaraman/kafka leave-group Alternatively you can review