Re: Review Request 33049: Patch for KAFKA-2084

2015-06-02 Thread Aditya Auradkar
ION Diff: https://reviews.apache.org/r/33049/diff/ Testing --- Thanks, Aditya Auradkar

Review Request 34966: Patch for KAFKA-2238

2015-06-02 Thread Aditya Auradkar
9efa15ca5567b295ab412ee9eea7c03eb4cdc18b core/src/test/scala/unit/kafka/server/KafkaConfigConfigDefTest.scala 8014a5a6c362785539f24eb03d77278434614fe6 Diff: https://reviews.apache.org/r/34966/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-03 Thread Aditya Auradkar
; will be throttled for a long time anyway. removed - Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/#review85465 -----

Re: Review Request 34965: Patch for KAFKA-2241

2015-06-03 Thread Aditya Auradkar
e/src/main/scala/kafka/consumer/SimpleConsumer.scala <https://reviews.apache.org/r/34965/#comment138486> I think this needs to be volatile or AtomicBoolean - Aditya Auradkar On June 2, 2015, 11:54 p.m., Dong Lin wrote: > > --

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-04 Thread Aditya Auradkar
ide metrics on per-client * API key throttle > > rate right? We already have throttle time metrics per-clientId * API key. Do you think we need rate in addition to those? - Aditya --- This is an automatically gener

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-04 Thread Aditya Auradkar
Jun Rao. Bugs: KAFKA-2084 https://issues.apache.org/jira/browse/KAFKA-2084 Repository: kafka Description (updated) --- Signed-off-by: Aditya Auradkar Addressing Joel's comments Diffs (updated) - clients/src/main/java/org/apache/kafka/common/metrics/Quota

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-04 Thread Aditya Auradkar
it/kafka/server/ThrottledRequestExpirationTest.scala PRE-CREATION Diff: https://reviews.apache.org/r/33049/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 27799: New consumer

2015-01-22 Thread Aditya Auradkar
lic from the interface methods? - Aditya Auradkar On Jan. 21, 2015, 4:47 p.m., Jay Kreps wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 27799: New consumer

2015-01-22 Thread Aditya Auradkar
> On Jan. 22, 2015, 5:35 p.m., Aditya Auradkar wrote: > > clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java, > > line 21 > > <https://reviews.apache.org/r/27799/diff/4/?file=828377#file828377line21> > > > > nit. Can we remov

Re: Review Request 27799: New consumer

2015-01-22 Thread Aditya Auradkar
> On Jan. 22, 2015, 5:35 p.m., Aditya Auradkar wrote: > > clients/src/main/java/org/apache/kafka/clients/RequestCompletionHandler.java, > > line 21 > > <https://reviews.apache.org/r/27799/diff/4/?file=828377#file828377line21> > > > > nit. Can we remov

Review Request 30196: Patch for KAFKA-1886

2015-01-22 Thread Aditya Auradkar
cala/kafka/consumer/SimpleConsumer.scala cbef84ac76e62768981f74e71d451f2bda995275 core/src/test/scala/unit/kafka/integration/PrimitiveApiTest.scala a5386a03b62956bc440b40783247c8cdf7432315 Diff: https://reviews.apache.org/r/30196/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 30196: Patch for KAFKA-1886

2015-01-22 Thread Aditya Auradkar
iff: https://reviews.apache.org/r/30196/diff/ Testing (updated) --- Added an integration test to PrimitiveAPITest.scala. Thanks, Aditya Auradkar

Re: Review Request 30196: Patch for KAFKA-1886

2015-01-26 Thread Aditya Auradkar
s.apache.org/r/30196/#review69579 --- On Jan. 22, 2015, 10:35 p.m., Aditya Auradkar wrote: > > --- > This is an automatically generated e-mail. To reply, v

Review Request 30527: Patch for KAFKA-1886

2015-02-02 Thread Aditya Auradkar
imitiveApiTest.scala a5386a03b62956bc440b40783247c8cdf7432315 Diff: https://reviews.apache.org/r/30527/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 30196: Patch for KAFKA-1886

2015-02-02 Thread Aditya Auradkar
cbef84ac76e62768981f74e71d451f2bda995275 core/src/test/scala/unit/kafka/integration/PrimitiveApiTest.scala aeb7a19acaefabcc161c2ee6144a56d9a8999a81 Diff: https://reviews.apache.org/r/30196/diff/ Testing --- Added an integration test to PrimitiveAPITest.scala. Thanks, Aditya Auradkar

Re: Review Request 30196: Patch for KAFKA-1886

2015-02-02 Thread Aditya Auradkar
> On Jan. 26, 2015, 1:28 a.m., Neha Narkhede wrote: > > core/src/test/scala/unit/kafka/integration/PrimitiveApiTest.scala, line 235 > > <https://reviews.apache.org/r/30196/diff/1/?file=831148#file831148line235> > > > > what is the purpose of this sleep? &g

Review Request 30570: Patch for KAFKA-1914

2015-02-03 Thread Aditya Auradkar
/ReplicaManager.scala fb948b9ab28c516e81dab14dcbe211dcd99842b6 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala ccf5e2e36260b2484181b81d1b06e81de972674b Diff: https://reviews.apache.org/r/30570/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-03 Thread Aditya Auradkar
added asserts to the SimpleFetchTest to count the number of fetch requests. I'm going to file an additional jira to add unit tests for all the BrokerTopicMetrics updated via ReplicaManager Thanks, Aditya Auradkar

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-09 Thread Aditya Auradkar
- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30570/#review71646 ------- On Feb. 3, 2015, 7:13 p.m., Aditya Auradkar wrote: > > --- > This is an aut

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-09 Thread Aditya Auradkar
e and commit-offset > > requests. And in general I think we might better record the request-level > > metrics on KafkaApis layer intead of the ReplicaManager layer. There are > > some previous work on isolating the request-level information in KafkaApis > > layer. >

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-09 Thread Aditya Auradkar
nerated e-mail. To reply, visit: https://reviews.apache.org/r/30570/#review71660 ------- On Feb. 3, 2015, 7:13 p.m., Aditya Auradkar wrote: > > --- > This is

Review Request 30848: Patch for KAFKA-1943

2015-02-10 Thread Aditya Auradkar
fb948b9ab28c516e81dab14dcbe211dcd99842b6 Diff: https://reviews.apache.org/r/30848/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-17 Thread Aditya Auradkar
/ Testing --- I've added asserts to the SimpleFetchTest to count the number of fetch requests. I'm going to file an additional jira to add unit tests for all the BrokerTopicMetrics updated via ReplicaManager Thanks, Aditya Auradkar

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-17 Thread Aditya Auradkar
TopicsStats()? Good catch. Fixed - Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30570/#review72817 --- On Feb. 17, 2015,

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-17 Thread Aditya Auradkar
s is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30570/#review72845 ------- On Feb. 17, 2015, 11:46 p.m., Aditya Auradkar wrote: > > --- > Thi

Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
Test.scala 525c835803b048c952667567cd205b4b06824391 Diff: https://reviews.apache.org/r/31168/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
fka/server/SimpleFetchTest.scala 525c835803b048c952667567cd205b4b06824391 Diff: https://reviews.apache.org/r/31168/diff/ Testing (updated) --- All unit tests now pass. Thanks, Aditya Auradkar

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Aditya Auradkar
fka/server/SimpleFetchTest.scala 525c835803b048c952667567cd205b4b06824391 Diff: https://reviews.apache.org/r/31168/diff/ Testing (updated) --- All unit tests now pass. BUILD SUCCESSFUL Total time: 9 mins 35.296 secs Thanks, Aditya Auradkar

Re: Review Request 30196: Patch for KAFKA-1886

2015-02-23 Thread Aditya Auradkar
place to put it, if it makes sense to keep it at all? - Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30196/#review71556 ------- On Feb. 2, 2015, 9:57 p.m., Aditya Auradkar wrote: > > -

RE: Announcing the Confluent Platform built on Apache Kafka

2015-02-25 Thread Aditya Auradkar
Congrats! From: Andrew Otto [ao...@wikimedia.org] Sent: Wednesday, February 25, 2015 12:06 PM To: us...@kafka.apache.org Cc: dev@kafka.apache.org Subject: Re: Announcing the Confluent Platform built on Apache Kafka Wow, .deb packages. I love you. > On F

Review Request 31449: Patch for KAFKA-1986

2015-02-25 Thread Aditya Auradkar
/browse/KAFKA-1986 Repository: kafka Description --- Patch for KAFKA-1986 Diffs - core/src/main/scala/kafka/server/ReplicaManager.scala 586cf4caa95f58d5b2e6c7429732d25d2b3635c8 Diff: https://reviews.apache.org/r/31449/diff/ Testing --- Thanks, Aditya Auradkar

[KIP-DISCUSSION] KIP-13 Quotas

2015-03-04 Thread Aditya Auradkar
Posted a KIP for quotas in kafka. https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas Appreciate any feedback. Aditya

RE: [DISCUSS] KIP-6 - New reassignment partition logic for re-balancing

2015-03-05 Thread Aditya Auradkar
Thanks for the writeup and RB Joe/Dmitry. Is it possible to describe the proposed partition reassignment algorithm in more detail on the KIP? In fact, it would be really easy to understand if we had some concrete examples comparing partition assignment with the old algorithm and the new. Adity

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-09 Thread Aditya Auradkar
propriate > time > > and has no per-request logic whatsoever. > > > > 7. We need to think through and state the exact algorithm for how we will > > assign delays to requests for a use case that is over its quota. That is > > closely tied to how we calculate the metric used. Here would be a bad > > approach we should not use: > > a. measure in a 30 second window. > > b. when we have hit the cap in that window, delay for the remainder of > the > > 30 seconds > > As you can imagine with this bad algorithm you might then use all server > > resources for 5 seconds, then suddenly assign a 25 second delay to the > next > > request from that client, then the window would reset and this would > > repeat. > > The quota package is already doing a good job of the windowed metrics, > but > > we'll want to integrate the backoff calculation with that algorithm > > (assuming that is what we are using). > > > > Cheers, > > > > -Jay > > > > On Wed, Mar 4, 2015 at 3:51 PM, Aditya Auradkar < > > aaurad...@linkedin.com.invalid> wrote: > > > > > Posted a KIP for quotas in kafka. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas > > > > > > Appreciate any feedback. > > > > > > Aditya > > > > > >

Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-11 Thread Aditya Auradkar
c124c8df5b5079e5ffbd0c4ea359562a66aaf317 core/src/test/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-12 Thread Aditya Auradkar
> > > > > > client properly backing off. In cases where they don't we may > > > actually > > > > > make > > > > > > things worse. Given the diversity of clients I think this is > > probably > > > > not > > &g

[DISCUSS] KIP 16 - Replica lag tuning

2015-03-12 Thread Aditya Auradkar
I wrote a KIP for this after some discussion on KAFKA-1546. https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning The RB is here: https://reviews.apache.org/r/31967/ Thanks, Aditya

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-12 Thread Aditya Auradkar
/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-12 Thread Aditya Auradkar
c124c8df5b5079e5ffbd0c4ea359562a66aaf317 core/src/test/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya

RE: [DISCUSS] KIP 16 - Replica lag tuning

2015-03-12 Thread Aditya Auradkar
http://www.stealth.ly - - - - - - - - - - - - - - - - - On Thu, Mar 12, 2015 at 4:36 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > I wrote a KIP for this after some discussion on KAFKA-1546. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning > >

[VOTE] KIP-16: Replica Lag Tuning

2015-03-13 Thread Aditya Auradkar
Details in the KIP, Jira and RB. https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning https://issues.apache.org/jira/browse/KAFKA-1546 https://reviews.apache.org/r/31967/ Aditya

Re: Review Request 32063: Patch for KAFKA-2019

2015-03-13 Thread Aditya Auradkar
? - Aditya Auradkar On March 14, 2015, 12:13 a.m., Joseph Holsten wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache

RE: [VOTE] KIP-16: Replica Lag Tuning

2015-03-15 Thread Aditya Auradkar
n the approach. Much better than before. Thanks, Jun On Fri, Mar 13, 2015 at 9:54 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Details in the KIP, Jira and RB. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP+16+:+Automated+Replica+Lag+Tuning >

RE: [VOTE] KIP-16: Replica Lag Tuning

2015-03-15 Thread Aditya Auradkar
Rather, 6 binding +1's. Aditya From: Aditya Auradkar Sent: Sunday, March 15, 2015 10:39 PM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-16: Replica Lag Tuning Thanks. We have 5 binding +1s. I'll update the KIP with your comments and mark a

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Aditya Auradkar
which will start the clock. But your suggestion makes it cleaner, so I'll change. - Aditya Auradkar On March 12, 2015, 8:42 p.m., Aditya Auradkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: >

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Aditya Auradkar
tps://reviews.apache.org/r/31967/#comment124158> Interesting point. I thought that it would be enough to simply check the lag value. But yes, this will cause the HW to be inconsistent. - Aditya Auradkar On March 12, 2015, 8:42 p.m., Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Aditya Auradkar
/test/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-16 Thread Aditya Auradkar
erver/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thanks, Aditya Auradkar

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-16 Thread Aditya Auradkar
n On Wed, Mar 4, 2015 at 3:51 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Posted a KIP for quotas in kafka. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas > > Appreciate any feedback. > > Aditya >

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-17 Thread Aditya Auradkar
tps://reviews.apache.org/r/31967/#comment124449> Renamed. We can use the raw offset directly, but in general the code is passing the LogOffsetMetadata objects and it made sense to remain consistent. - Aditya Auradkar On March 16, 2015, 6:32 p.m., Aditya Auradkar

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-17 Thread Aditya Auradkar
- Thanks, Aditya Auradkar

RE: [DISCUSS] KIP-5 - Broker Configuration Management

2015-03-18 Thread Aditya Auradkar
Hi Andrii, Thanks for the writeup. IMO, we should be able to support dynamically changing certain configs. For example: the Quota proposal relies on such a mechanism for changing quotas on the fly. Aditya From: Andrii Biletskyi [andrii.bilets...@stealt

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Aditya Auradkar
; >>> > > >> > about the response is super dangerous. The error handling is > > >>> one of > > >>> > > the > > >>> > > >> > hardest parts of the client (Guozhang chime in here). &

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Aditya Auradkar
gt; On Mon, Mar 16, 2015 at 9:42 PM, Jay Kreps < > > jay.kr...@gmail.com > > > > > > > >>> > wrote: > > > >>> > > >> > Hey Jun, > > > >>> > > >> > > > > >>> > > >&g

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-19 Thread Aditya Auradkar
t in KAFKA-1930). I agree that this KIP doesn't need to block on the migration of the metrics package. Thanks, Jun On Wed, Mar 18, 2015 at 4:02 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Hey everyone, > > Thanks for the great discussion. There are currently a

Metrics package discussion

2015-03-25 Thread Aditya Auradkar
Hey everyone, Picking up this discussion after yesterdays KIP hangout. For anyone who did not join the meeting, we have 2 different metrics packages being used by the clients (custom package) and the server (codahale). We are discussing whether to migrate the server to the new package. What in

Re: Review Request 31967: Patch for KAFKA-1546

2015-03-25 Thread Aditya Auradkar
1896d07e6e core/src/test/scala/unit/kafka/server/LogRecoveryTest.scala 92d6b2c672f74cdd526f2e98da8f7fb3696a88e3 core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala efb457334bd87e4c5b0dd2c66ae1995993cd0bc1 Diff: https://reviews.apache.org/r/31967/diff/ Testing --- Thank

Re: [ANNOUNCE] New Kafka Committer Ewen Cheslack-Postava

2015-12-08 Thread Aditya Auradkar
Congrats Ewen! On Tue, Dec 8, 2015 at 11:51 AM, Guozhang Wang wrote: > Congrats Ewen! Welcome onboard. > > Guozhang > > On Tue, Dec 8, 2015 at 11:42 AM, Liquan Pei wrote: > > > Congrats, Ewen! > > > > On Tue, Dec 8, 2015 at 11:37 AM, Neha Narkhede > wrote: > > > > > I am pleased to announce th

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

2016-01-04 Thread Aditya Auradkar
Hey Becket/Anna - I have a few comments about the KIP. 1. (Minor) Can we rename the KIP? It's currently "Add CreateTime and LogAppendTime etc..". This is actually the title of the now rejected Option 1. 2. (Minor) Can we rename the proposed option? It isn't really "option 4" anymore. 3. I'm not c

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

2016-01-05 Thread Aditya Auradkar
gt; >> > wrote: > >> > >> > >> > >>> Few suggestions on improving the KIP > >> > >>> > >> > >>> *If some brokers have rack, and some do not, the algorithm will > >> thrown > >> > an > >

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

2016-01-05 Thread Aditya Auradkar
onfigurable now. > > > > 61. In this design, it's possible for a log segment to have an empty time > > index. In the worse case, we may have to scan more than the active > segment > > to recover the latest timestamp. > > > > Thanks, > > > > J

Re: [VOTE] KIP-32 Add CreateTime and LogAppendTime to Kafka message.

2016-01-08 Thread Aditya Auradkar
y make sure the version of Record(Message) and > > > > > > MemoryRecords(MessageSet) matches. > > > > > > > > > > > > Currently new clients uses classes in o.a.k.common.record, and > the > > > > broker > > > > > > and old clients uses c

Re: [VOTE] KIP-36 - Rack aware replica assignment

2016-01-19 Thread Aditya Auradkar
+1 (non-binding) On Tue, Jan 19, 2016 at 8:28 AM, Grant Henke wrote: > +1 (non-binding) > > On Tue, Jan 19, 2016 at 7:46 AM, Ismael Juma wrote: > > > +1 (non-binding) > > > > On Tue, Jan 19, 2016 at 6:12 AM, Guozhang Wang > wrote: > > > > > +1 (binding) > > > > > > On Mon, Jan 18, 2016 at 2:43

[DISCUSS] KIP-46: Self Healing

2016-02-02 Thread Aditya Auradkar
Hey everyone, I just created a kip to discuss automated replica reassignment when we lose a broker in the cluster. https://cwiki.apache.org/confluence/display/KAFKA/KIP-46%3A+Self+Healing+Kafka Any feedback is welcome. Thanks, Aditya

Re: [DISCUSS] KIP-46: Self Healing

2016-02-09 Thread Aditya Auradkar
ment in a running > cluster. Once we stabilize these changes and feel confident that they work, > we can push the policy into the controller and have it automatically be > triggered based on different events. > > Thanks, > Neha > > On Tue, Feb 2, 2016 at 6:13 PM, Aditya Aurad

Re: [DISCUSS] KIP-46: Self Healing

2016-02-16 Thread Aditya Auradkar
the capability to move large > amounts of data to the controller, after we are very sure that the default > policy is well tested and the replica throttling works. If so, then it is > just a matter of placing the trigger in the controller vs in the tool. But > I'm skeptical

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-08 Thread Aditya Auradkar
Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/#review86428 --- On May 12, 2015, 9:42 p.m., Aditya Auradkar wrote: > >

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-08 Thread Aditya Auradkar
/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-09 Thread Aditya Auradkar
scala 5717165f2344823fabe8f7cfafae4bb8af2d949a core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-09 Thread Aditya Auradkar
/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-09 Thread Aditya Auradkar
/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-09 Thread Aditya Auradkar
scala 5717165f2344823fabe8f7cfafae4bb8af2d949a core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-09 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/#review86428 ------- On June 9, 2015, 5:10 p.m., Aditya Auradkar wrote: > > ---

RE: [DISCUSS] KIP-25 System test improvements

2015-06-11 Thread Aditya Auradkar
Hi Geoffrey, Thanks for the writeup. Couple of questions: - Is it possible to configure suites using ducktape? For example: assume all the tests in system_tests have been migrated to ducktape. Can I run a subset of all tests grouped by functional areas i.e. replication, broker failure etc. - Du

RE: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

2015-06-11 Thread Aditya Auradkar
h TMR will > still > return isr. Taking this into account I think yes - it would be good to fix > ISR issue, > although I didn't consider it to be a critical one (isr was part of TMR > from the very > beginning and almost no code relies on this piece of request). > > T

RE: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

2015-06-11 Thread Aditya Auradkar
Andrii, Do we need a new voting thread for this KIP? The last round of votes had 3 binding +1's but there's been a fair amount of discussion since then. Aditya ____ From: Aditya Auradkar Sent: Thursday, June 11, 2015 10:32 AM To: dev@kafka.apache.o

Re: Review Request 33614: Patch for KAFKA-2132

2015-06-11 Thread Aditya Auradkar
14/#comment140028> perhaps you can use the ternary operator here? - Aditya Auradkar On April 30, 2015, 10:53 p.m., Ashish Singh wrote: > > --- > This is an automatically generated e-

RE: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

2015-06-12 Thread Aditya Auradkar
changes. Joel On Thu, Jun 11, 2015 at 06:36:36PM +, Aditya Auradkar wrote: > Andrii, > > Do we need a new voting thread for this KIP? The last round of votes had 3 > binding +1's but there's been a fair amount of discussion since then. > > Aditya > > __

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-12 Thread Aditya Auradkar
Jun Rao. Bugs: KAFKA-2084 https://issues.apache.org/jira/browse/KAFKA-2084 Repository: kafka Description (updated) --- Signed-off-by: Aditya Auradkar Addressing Joel's comments Minor imports changes Added testcase to verify that replication traffic is not throttled Tmp c

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-12 Thread Aditya Auradkar
it/kafka/server/ThrottledRequestExpirationTest.scala PRE-CREATION Diff: https://reviews.apache.org/r/33049/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 35437: Patch for KAFKA-2202

2015-06-14 Thread Aditya Auradkar
formance.scala <https://reviews.apache.org/r/35437/#comment140291> can probably do this inline case _: ConsumerEx => consumerTimeout.set(true) - Aditya Auradkar On June 14, 2015, 11:27 a.m., Manikumar Reddy O wrote: > > --- &g

RE: Guozhang Wang elected to Kafka PMC

2015-06-16 Thread Aditya Auradkar
Congrats Guozhang! From: Ashish Singh [asi...@cloudera.com] Sent: Monday, June 15, 2015 10:53 PM To: dev@kafka.apache.org Cc: Jun Rao Subject: Re: Guozhang Wang elected to Kafka PMC Congrats Guozhang! On Mon, Jun 15, 2015 at 10:20 PM, Sriharsha Chintalapa

Review Request 35734: Patch for KAFKA-2293

2015-06-22 Thread Aditya Auradkar
/browse/KAFKA-2293 Repository: kafka Description --- Fix for 2293 Diffs - core/src/main/scala/kafka/cluster/Partition.scala 6cb647711191aee8d36e9ff15bdc2af4f1c95457 Diff: https://reviews.apache.org/r/35734/diff/ Testing --- Thanks, Aditya Auradkar

RE: [VOTE] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-06-23 Thread Aditya Auradkar
Hey Ashish, Last hangout, I think we discussed adding a "num-iterations" parameter to the script. Do you plan to support that? Aditya From: Neha Narkhede [n...@confluent.io] Sent: Tuesday, June 23, 2015 11:24 AM To: dev@kafka.apache.org Cc: Jun Rao Subje

RE: [VOTE] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-06-23 Thread Aditya Auradkar
with notes on the parameter. Will update the patch. On Tue, Jun 23, 2015 at 11:51 AM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > Hey Ashish, > > Last hangout, I think we discussed adding a "num-iterations" parameter to > the script. Do you plan

RE: Help Us Nominate Apache Kafka for a 2015 Bossie (Best of OSS) Award - Due June 30th

2015-06-28 Thread Aditya Auradkar
Sent :) From: Gwen Shapira [gshap...@cloudera.com] Sent: Friday, June 26, 2015 11:53 AM To: us...@kafka.apache.org Cc: dev@kafka.apache.org Subject: Re: Help Us Nominate Apache Kafka for a 2015 Bossie (Best of OSS) Award - Due June 30th Sent! Thanks for le

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-29 Thread Aditya Auradkar
Jun Rao. Bugs: KAFKA-2084 https://issues.apache.org/jira/browse/KAFKA-2084 Repository: kafka Description (updated) --- Signed-off-by: Aditya Auradkar Addressing Joel's comments Minor imports changes Added testcase to verify that replication traffic is not throttled Tmp c

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-29 Thread Aditya Auradkar
https://reviews.apache.org/r/33049/#review88180 --- On June 30, 2015, 12:53 a.m., Aditya Auradkar wrote: > > --- > This is an automatically generated e-ma

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-29 Thread Aditya Auradkar
cala/unit/kafka/server/ThrottledRequestExpirationTest.scala PRE-CREATION Diff: https://reviews.apache.org/r/33049/diff/ Testing --- Thanks, Aditya Auradkar

Re: Review Request 33049: Patch for KAFKA-2084

2015-06-29 Thread Aditya Auradkar
ottle_time because that requires 2136 to be committed. - Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/#review88180 -------

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-30 Thread Aditya Auradkar
/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-30 Thread Aditya Auradkar
/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79 Diff: https://reviews.apache.org/r/33378/diff/ Testing --- New tests added Thanks, Aditya Auradkar

Re: Review Request 33378: Patch for KAFKA-2136

2015-06-30 Thread Aditya Auradkar
ely after this is committed? - Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/#review89429 ------- On July 1,

Re: Review Request 34554: Patch for KAFKA-2205

2015-07-01 Thread Aditya Auradkar
ta changes can be done within processConfigChanges. - Aditya --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34554/#review89623 --- On

Re: Review Request 34554: Patch for KAFKA-2205

2015-07-01 Thread Aditya Auradkar
/diff/ Testing --- 1. Added new testcases for new code. 2. Verified that both topic and client configs can be changed dynamically by starting a local cluster Thanks, Aditya Auradkar

Re: Review Request 34554: Patch for KAFKA-2205

2015-07-01 Thread Aditya Auradkar
erver/DynamicConfigChangeTest.scala 7877f6ca1845c2edbf96d4a9783a07a552db8f07 Diff: https://reviews.apache.org/r/34554/diff/ Testing --- 1. Added new testcases for new code. 2. Verified that both topic and client configs can be changed dynamically by starting a local cluster Thanks, Aditya Auradkar

RE: [ANNOUNCE] New Committer

2015-07-06 Thread Aditya Auradkar
Congratulations Gwen! Aditya From: Ashish Singh [asi...@cloudera.com] Sent: Monday, July 06, 2015 6:16 PM To: dev@kafka.apache.org Subject: Re: [ANNOUNCE] New Committer Congrats Gwen! On Monday, July 6, 2015, Joe Stein wrote: > I am pleased to announce

<    1   2   3   4   5   >