[jira] [Updated] (KAFKA-1446) Consumer metrics for rebalance

[Neha Narkhede (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neha Narkhede updated KAFKA-1446:
-

Labels: newbie usability  (was: )

> Consumer metrics for rebalance
> --
>
> Key: KAFKA-1446
> URL: https://issues.apache.org/jira/browse/KAFKA-1446
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Clark Haskins
>  Labels: newbie, usability
>
> The Kafka consumer should have metrics around the number of seconds spent in 
> rebalance over the last minute as well as the number of rebalances started 
> during the previous minute.
> The other important thing about these metrics is that they should only be 
> updated once per minute for example the rebalance time should not increase 
> from second to second during a rebalance.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Review Request 21244: Patch for KAFKA-1442

[Neha Narkhede]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21244/
---

Review request for kafka.


Bugs: KAFKA-1442
https://issues.apache.org/jira/browse/KAFKA-1442


Repository: kafka


Description
---

test


Diffs
-

  README.md d757b9748c7b3e3b824670c15173feb9a3847acf 
  kafka-patch-review.py dc6664d22a0b18f96d7d3513a6d1fe739d45c000 

Diff: https://reviews.apache.org/r/21244/diff/


Testing
---


Thanks,

Neha Narkhede

Review Request 21239: Patch for KAFKA-1431

[Sriharsha Chintalapani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21239/
---

Review request for kafka.


Bugs: KAFKA-1431
https://issues.apache.org/jira/browse/KAFKA-1431


Repository: kafka


Description
---

KAFKA-1431. ConsoleConsumer - Option to clean zk consumer path.


Diffs
-

  core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 
0f62819be0562f62c0f778bd20ead053f01a6f2f 

Diff: https://reviews.apache.org/r/21239/diff/


Testing
---


Thanks,

Sriharsha Chintalapani

Review Request 21173: Fix KAFKA-1432: follow-up on using new producer on system test MMs

[Guozhang Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21173/
---

Review request for kafka.


Bugs: KAFKA-1432
https://issues.apache.org/jira/browse/KAFKA-1432


Repository: kafka


Description
---

Add --new.producer in system tests


Diffs
-

  system_test/utils/kafka_system_test_utils.py 
6917ea1d0c79b993b3208510a920484e9323aa5c 

Diff: https://reviews.apache.org/r/21173/diff/


Testing
---


Thanks,

Guozhang Wang

[jira] [Commented] (KAFKA-1445) New Producer should send all partitions that have non-empty batches when on of them is ready

[Guozhang Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994313#comment-13994313
 ] 

Guozhang Wang commented on KAFKA-1445:
--

Created reviewboard https://reviews.apache.org/r/21304/
 against branch origin/trunk

> New Producer should send all partitions that have non-empty batches when on 
> of them is ready
> 
>
> Key: KAFKA-1445
> URL: https://issues.apache.org/jira/browse/KAFKA-1445
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
> Fix For: 0.9.0
>
> Attachments: KAFKA-1445.patch
>
>
> One difference between the new producer and the old producer is that on the 
> new producer the linger time is per partition, instead of global. Therefore, 
> when the traffic is low, the sender will likely expire partitions one-by-one 
> and send lots of small request containing only a few partitions with a few 
> data, resulting largely increased request rate.
> One solution of it would be to let senders select all partitions that have 
> non-empty batches when on of them is ready.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21174: Fix KAFKA-1396

[Guozhang Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21174/
---

(Updated May 7, 2014, 9:56 p.m.)


Review request for kafka.


Bugs: KAFKA-1396
https://issues.apache.org/jira/browse/KAFKA-1396


Repository: kafka


Description (updated)
---

1. Reduce bouncing test iterations from 5 to 2.
2. Double check metadata is propagated after the bouncing iterations and before 
fetching data.


Diffs
-

  core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
a993e8c7531abca6b82d1718ac57ba6a295c1fc7 
  core/src/test/scala/integration/kafka/api/ProducerSendTest.scala 
af11a4983599a20a33b6f1fc05738a59abfede63 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
034f36130a1d0669dda5ed53e9cee328806c811e 

Diff: https://reviews.apache.org/r/21174/diff/


Testing
---


Thanks,

Guozhang Wang

[jira] [Commented] (KAFKA-1308) Publish jar of test utilities to Maven

[Yuta Okamoto (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994618#comment-13994618
 ] 

Yuta Okamoto commented on KAFKA-1308:
-

+1

We need _kafka*-test.jar_ for testing for our third-party client, 
_[finagle-kafka|https://github.com/okapies/finagle-kafka/pull/1]_, without 
adding the jar to the repository.


> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
> Attachments: KAFKA-1308.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1442) RBTools post-review is deprecated

[Neha Narkhede (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neha Narkhede updated KAFKA-1442:
-

Attachment: KAFKA-1442_2014-05-08_16:01:09.patch

> RBTools post-review is deprecated
> -
>
> Key: KAFKA-1442
> URL: https://issues.apache.org/jira/browse/KAFKA-1442
> Project: Kafka
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>Priority: Minor
> Attachments: KAFKA-1442.patch, KAFKA-1442.patch, 
> KAFKA-1442_2014-05-08_16:01:09.patch
>
>
> kafka-patch-review.py uses RBTools post-review for sending patch reviews. 
> post-review command is deprecated and replaced by rbt post in their latest 
> release. 
> kafka-patch-review.py updates the jira ticket with attaching patch even if 
> the RBTools are not found.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21174: Fix KAFKA-1396

[Guozhang Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21174/
---

(Updated May 7, 2014, 9:17 p.m.)


Review request for kafka.


Bugs: KAFKA-1396
https://issues.apache.org/jira/browse/KAFKA-1396


Repository: kafka


Description (updated)
---

1. Reduce bouncing test iterations from 5 to 2.
2. Double check metadata is propagated after the bouncing iterations and before 
fetching data.


Diffs
-

  core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
a993e8c7531abca6b82d1718ac57ba6a295c1fc7 

Diff: https://reviews.apache.org/r/21174/diff/


Testing
---


Thanks,

Guozhang Wang

[jira] [Updated] (KAFKA-1308) Publish jar of test utilities to Maven

[Joe Stein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Stein updated KAFKA-1308:
-

Fix Version/s: 0.8.2

> Publish jar of test utilities to Maven
> --
>
> Key: KAFKA-1308
> URL: https://issues.apache.org/jira/browse/KAFKA-1308
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.8.1
>Reporter: Martin Kleppmann
> Fix For: 0.8.2
>
> Attachments: KAFKA-1308.patch
>
>
> For projects that use Kafka, and want to write tests that exercise Kafka (in 
> our case, Samza), it's useful to have access to Kafka's test utility classes 
> such as kafka.zk.EmbeddedZookeeper and kafka.utils.TestUtils. We can use 
> {{./gradlew testJar}} to build jar files that contain those classes, but as 
> far as I know, these are currently not made available in a binary release.
> At the moment, we have to check those kafka*-test.jar files into the Samza 
> repository. To avoid that, would it be possible to publish those jars of 
> tests to Maven, so that they fit into the normal dependency management?
> Or perhaps, if publishing the tests themselves is not appropriate, we could 
> move the test utilities into a separate module that is published, and make 
> the tests depend on that module?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (KAFKA-1444) kafka.javaapi.TopicMetadata and PartitionMetadata doesn't forward the toString method

[Simon Cooper (JIRA)]

Simon Cooper created KAFKA-1444:
---

 Summary: kafka.javaapi.TopicMetadata and PartitionMetadata doesn't 
forward the toString method
 Key: KAFKA-1444
 URL: https://issues.apache.org/jira/browse/KAFKA-1444
 Project: Kafka
  Issue Type: Bug
Reporter: Simon Cooper
Priority: Minor


the kafka.javaapi.PartitionMetadata and TopicMetadata classes don't forward the 
toString method to the underlying kafka.api.PartitionMetadata/TopicMetadata 
classes along with the other methods. This means toString on these classes 
doesn't work properly.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21239: Patch for KAFKA-1431

[Jun Rao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21239/#review42664
---



core/src/main/scala/kafka/consumer/ConsoleConsumer.scala


Could we change the description to sth like "if specified, the consumer 
path in zookeeper is deleted when starting up".


- Jun Rao


On May 8, 2014, 9:58 p.m., Sriharsha Chintalapani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21239/
> ---
> 
> (Updated May 8, 2014, 9:58 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1431
> https://issues.apache.org/jira/browse/KAFKA-1431
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1431. ConsoleConsumer - Option to clean zk consumer path.
> 
> 
> Diffs
> -
> 
>   core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 
> 0f62819be0562f62c0f778bd20ead053f01a6f2f 
> 
> Diff: https://reviews.apache.org/r/21239/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sriharsha Chintalapani
> 
>

[jira] [Closed] (KAFKA-1273) Brokers should make sure replica.fetch.max.bytes >= message.max.bytes

[Neha Narkhede (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neha Narkhede closed KAFKA-1273.



> Brokers should make sure replica.fetch.max.bytes >= message.max.bytes
> -
>
> Key: KAFKA-1273
> URL: https://issues.apache.org/jira/browse/KAFKA-1273
> Project: Kafka
>  Issue Type: Bug
>  Components: replication
>Affects Versions: 0.8.0
>Reporter: Dong Zhong
>Assignee: Sriharsha Chintalapani
>  Labels: newbie
>
> If message.max.bytes is larger than replica.fetch.max.bytes，followers can't 
> fetch data from the leader and will incur endless retry. And this may cause 
> high network traffic between followers and leaders.
> Brokers should make sure replica.fetch.max.bytes >= message.max.bytes by 
> adding a sanity check, or throw an exception.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Issue Comment Deleted] (KAFKA-1442) RBTools post-review is deprecated

[Neha Narkhede (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neha Narkhede updated KAFKA-1442:
-

Comment: was deleted

(was: Updated reviewboard https://reviews.apache.org/r/21244/
 against branch trunk)

> RBTools post-review is deprecated
> -
>
> Key: KAFKA-1442
> URL: https://issues.apache.org/jira/browse/KAFKA-1442
> Project: Kafka
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>Priority: Minor
> Attachments: KAFKA-1442.patch, KAFKA-1442.patch, 
> KAFKA-1442_2014-05-08_16:01:09.patch
>
>
> kafka-patch-review.py uses RBTools post-review for sending patch reviews. 
> post-review command is deprecated and replaced by rbt post in their latest 
> release. 
> kafka-patch-review.py updates the jira ticket with attaching patch even if 
> the RBTools are not found.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Review Request 21174: Fix KAFKA-1396

[Jun Rao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21174/#review42665
---



core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala


Should we do assertTrue(scheduler.failed == false) again after shutting 
down the scheduler?



core/src/test/scala/unit/kafka/utils/TestUtils.scala


Since this method should always return a valid leader (or an exception will 
be thrown), should we just return Int, instead of Option[Int]?


- Jun Rao


On May 9, 2014, 5:35 p.m., Guozhang Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21174/
> ---
> 
> (Updated May 9, 2014, 5:35 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1396
> https://issues.apache.org/jira/browse/KAFKA-1396
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> Address Jun's comments round two
> 
> 
> Diffs
> -
> 
>   core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
> a993e8c7531abca6b82d1718ac57ba6a295c1fc7 
>   core/src/test/scala/integration/kafka/api/ProducerSendTest.scala 
> af11a4983599a20a33b6f1fc05738a59abfede63 
>   core/src/test/scala/unit/kafka/utils/TestUtils.scala 
> 034f36130a1d0669dda5ed53e9cee328806c811e 
> 
> Diff: https://reviews.apache.org/r/21174/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Guozhang Wang
> 
>

Re: Review Request 21174: Fix KAFKA-1396

[Guozhang Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21174/
---

(Updated May 8, 2014, 10:57 p.m.)


Review request for kafka.


Bugs: KAFKA-1396
https://issues.apache.org/jira/browse/KAFKA-1396


Repository: kafka


Description (updated)
---

Address Jun's comments


Diffs (updated)
-

  core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
a993e8c7531abca6b82d1718ac57ba6a295c1fc7 
  core/src/test/scala/integration/kafka/api/ProducerSendTest.scala 
af11a4983599a20a33b6f1fc05738a59abfede63 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
034f36130a1d0669dda5ed53e9cee328806c811e 

Diff: https://reviews.apache.org/r/21174/diff/


Testing
---


Thanks,

Guozhang Wang

[jira] [Updated] (KAFKA-1444) kafka.javaapi.TopicMetadata and PartitionMetadata doesn't forward the toString method

[Jun Rao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jun Rao updated KAFKA-1444:
---

Labels: newbie  (was: )

> kafka.javaapi.TopicMetadata and PartitionMetadata doesn't forward the 
> toString method
> -
>
> Key: KAFKA-1444
> URL: https://issues.apache.org/jira/browse/KAFKA-1444
> Project: Kafka
>  Issue Type: Bug
>Reporter: Simon Cooper
>Priority: Minor
>  Labels: newbie
>
> the kafka.javaapi.PartitionMetadata and TopicMetadata classes don't forward 
> the toString method to the underlying 
> kafka.api.PartitionMetadata/TopicMetadata classes along with the other 
> methods. This means toString on these classes doesn't work properly.



--
This message was sent by Atlassian JIRA
(v6.2#6252)