[jira] [Work started] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2332 started by Dong Lin. --- > Add quota metrics to old producer and consumer >

[jira] [Updated] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2332: Attachment: KAFKA-2332.patch > Add quota metrics to old producer and consumer >

Review Request 36722: Patch for KAFKA-2332

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36722/ --- Review request for kafka. Bugs: KAFKA-2332 https://issues.apache.org/jira/b

[jira] [Updated] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2332: Status: Patch Available (was: In Progress) > Add quota metrics to old producer and consumer > -

[jira] [Updated] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2332: Attachment: KAFKA-2332.patch > Add quota metrics to old producer and consumer >

[jira] [Commented] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638348#comment-14638348 ] Dong Lin commented on KAFKA-2332: - Created reviewboard https://reviews.apache.org/r/36722/

Re: Review Request 36681: Patch for KAFKA-2275

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36681/#review92730 --- clients/src/test/java/org/apache/kafka/clients/consumer/internals/F

[jira] [Assigned] (KAFKA-1893) Allow regex subscriptions in the new consumer

[ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh reassigned KAFKA-1893: - Assignee: Ashish K Singh (was: Jason Gustafson) > Allow regex subscriptions in the new c

[jira] [Commented] (KAFKA-1893) Allow regex subscriptions in the new consumer

[ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638366#comment-14638366 ] Ashish K Singh commented on KAFKA-1893: --- Thanks [~hachikuji]! Will submit a patch so

Re: Review Request 36664: Patch for KAFKA-2353

> On July 22, 2015, 10:40 a.m., Ismael Juma wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 467 > > > > > > As far as I can see `ClosedChannelException`, `IllegalStateException` > > and `Illega

Re: Review Request 36664: Patch for KAFKA-2353

> On July 22, 2015, 10:40 a.m., Ismael Juma wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 467 > > > > > > As far as I can see `ClosedChannelException`, `IllegalStateException` > > and `Illega

Re: Review Request 36664: Patch for KAFKA-2353

> On July 21, 2015, 11:15 p.m., Gwen Shapira wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 465 > > > > > > Turns out that catching Throwable is a really bad idea: > > https://www.sumologic.co

[jira] [Commented] (KAFKA-2352) Possible memory leak in MirrorMaker and/or new Producer

[ https://issues.apache.org/jira/browse/KAFKA-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638656#comment-14638656 ] Kostya Golikov commented on KAFKA-2352: --- Indeed, the problem was caused by upstream

[jira] [Resolved] (KAFKA-2352) Possible memory leak in MirrorMaker and/or new Producer

[ https://issues.apache.org/jira/browse/KAFKA-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostya Golikov resolved KAFKA-2352. --- Resolution: Not A Problem > Possible memory leak in MirrorMaker and/or new Producer >

[jira] [Created] (KAFKA-2357) Update zookeeper.connect description in Kafka documentation

Yuto Sasaki created KAFKA-2357: -- Summary: Update zookeeper.connect description in Kafka documentation Key: KAFKA-2357 URL: https://issues.apache.org/jira/browse/KAFKA-2357 Project: Kafka Issue

Re: Review Request 36670: Patch for KAFKA-2355

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36670/#review92743 --- Ship it! Ship It! - Grant Henke On July 23, 2015, 1:23 a.m., Edw

Re: Review Request 36670: Patch for KAFKA-2355

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36670/#review92744 --- Ship it! Ship It! - Ismael Juma On July 23, 2015, 1:23 a.m., Edw

Re: Review Request 36670: Patch for KAFKA-2355

> On July 23, 2015, 12:37 a.m., Edward Ribeiro wrote: > > core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala, line 269 > > > > > > I disagree because we are issuing two `deleteTopic` operations, the > > firs

Re: Review Request 36681: Patch for KAFKA-2275

> On July 23, 2015, 7:19 a.m., Ashish Singh wrote: > > core/src/test/scala/integration/kafka/api/ConsumerTest.scala, line 198 > > > > > > I don't think it is an issue. Hope it's OK to drop the issue. Thanks > > for

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638906#comment-14638906 ] Ismael Juma commented on KAFKA-2355: [~eribeiro], change looks good. I was unable to a

Re: Review Request 33620: Patch for KAFKA-1690

> On July 22, 2015, 3 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, > > lines 231-234 > > > > > > Still some questions on this. > > > > 1. When hand

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639011#comment-14639011 ] Edward Ribeiro commented on KAFKA-2355: --- Hi [~ijuma], no problem, mate. If it looks

[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly

[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638991#comment-14638991 ] Ismael Juma commented on KAFKA-2351: [~mgharat], did you see the discussion around exc

[jira] [Commented] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639013#comment-14639013 ] Ismael Juma commented on KAFKA-2332: Is the intent to keep adding features to the exis

[jira] [Updated] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2355: Assignee: Edward Ribeiro > Add an unit test to validate the deletion of a partition marked as delete

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639020#comment-14639020 ] Ismael Juma commented on KAFKA-2355: I am not a committer, so someone else must merge

[GitHub] kafka pull request: KAFKA-2344; kafka-merge-pr improvements

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/90 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[jira] [Commented] (KAFKA-2344) kafka-merge-pr improvements

[ https://issues.apache.org/jira/browse/KAFKA-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639035#comment-14639035 ] ASF GitHub Bot commented on KAFKA-2344: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-2344) kafka-merge-pr improvements

[ https://issues.apache.org/jira/browse/KAFKA-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2344: Resolution: Fixed Status: Resolved (was: Patch Available) > kafka-merge-pr improvements > -

Re: Review Request 32841: Patch for KAFKA-2041

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32841/#review92753 --- log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLo

Re: Review Request 36681: Patch for KAFKA-2275

> On July 23, 2015, 7:19 a.m., Ashish Singh wrote: > > clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java, > > line 173 > > > > > > getAllTopics is a blocking call and we need to send

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639079#comment-14639079 ] Gwen Shapira commented on KAFKA-2355: - +1 and pushed to trunk. Thanks for the contribu

[jira] [Updated] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2355: Resolution: Fixed Status: Resolved (was: Patch Available) > Add an unit test to validate th

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639080#comment-14639080 ] Jay Kreps commented on KAFKA-2350: -- [~becket_qin] I think there are three proposals: 1. A

[jira] [Comment Edited] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639080#comment-14639080 ] Jay Kreps edited comment on KAFKA-2350 at 7/23/15 4:21 PM: --- [~be

[jira] [Comment Edited] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639080#comment-14639080 ] Jay Kreps edited comment on KAFKA-2350 at 7/23/15 4:22 PM: --- [~be

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639085#comment-14639085 ] Gwen Shapira commented on KAFKA-2350: - [~becket_qin] - when I imagined how the API wou

[jira] [Commented] (KAFKA-2355) Add an unit test to validate the deletion of a partition marked as deleted

[ https://issues.apache.org/jira/browse/KAFKA-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639095#comment-14639095 ] Edward Ribeiro commented on KAFKA-2355: --- Oh, thanks [~gwenshap]! Sorry for any incon

[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly

[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639097#comment-14639097 ] Mayuresh Gharat commented on KAFKA-2351: Hi [~ijuma], thanks a lot for pointing me

[jira] [Updated] (KAFKA-2275) Add a ListTopics() API to the new consumer

[ https://issues.apache.org/jira/browse/KAFKA-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-2275: -- Attachment: KAFKA-2275_2015-07-23_09:34:41.patch > Add a ListTopics() API to the new consumer >

[jira] [Commented] (KAFKA-2275) Add a ListTopics() API to the new consumer

[ https://issues.apache.org/jira/browse/KAFKA-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639098#comment-14639098 ] Ashish K Singh commented on KAFKA-2275: --- Updated reviewboard https://reviews.apache.

Re: Review Request 36681: Patch for KAFKA-2275

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36681/ --- (Updated July 23, 2015, 4:34 p.m.) Review request for kafka. Bugs: KAFKA-2275

Re: Review Request 36681: Patch for KAFKA-2275

> On July 23, 2015, 7:19 a.m., Ashish Singh wrote: > > clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java, > > line 173 > > > > > > getAllTopics is a blocking call and we need to send

Build failed in Jenkins: KafkaPreCommit #161

See Changes: [cshapi] KAFKA-2344; kafka-merge-pr improvements [cshapi] KAFKA-2355; Add an unit test to validate the deletion of a partition marked as deleted; patched by Edward Ribeiro, reviewed by Ashish Singh, Ismael Juma and Grant

Re: Review Request 36681: Patch for KAFKA-2275

> On July 23, 2015, 7:19 a.m., Ashish Singh wrote: > > clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java, > > line 173 > > > > > > getAllTopics is a blocking call and we need to send

Re: Round Robin Among Consumers

J A, It looks to me that in your case you actually want to scale the topic, right? Otherwise wouldn't a single consumer be enough? Jiangjie (Becket) Qin On Wed, Jul 22, 2015 at 7:39 PM, J A wrote: > Why have partition at all, if I don't need to scale topic. Coupling topic > scalability with co

[jira] [Commented] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639113#comment-14639113 ] Dong Lin commented on KAFKA-2332: - [~ijuma] I think the intent is to support existing cons

[jira] [Updated] (KAFKA-2357) Update zookeeper.connect description in Kafka documentation

[ https://issues.apache.org/jira/browse/KAFKA-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2357: --- Attachment: KAFKA-2357-1.patch > Update zookeeper.connect description in Kafka documentation > ---

[jira] [Updated] (KAFKA-2357) Update zookeeper.connect description in Kafka documentation

[ https://issues.apache.org/jira/browse/KAFKA-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2357: --- Assignee: David Jacot Fix Version/s: 0.8.3 0.8.1.2 Status: Patch

Re: Build failed in Jenkins: KafkaPreCommit #161

On Thu, Jul 23, 2015 at 5:39 PM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > > BUILD FAILED > This build job seems to be failing almost every time: https://builds.apache.org/job/KafkaPreCommit/ Also, it seems to be running at a similar time as: https://builds.apache.org/job/Kafk

[jira] [Commented] (KAFKA-2332) Add quota metrics to old producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639116#comment-14639116 ] Ismael Juma commented on KAFKA-2332: OK, thanks. > Add quota metrics to old producer

[jira] [Commented] (KAFKA-2357) Update zookeeper.connect description in Kafka documentation

[ https://issues.apache.org/jira/browse/KAFKA-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639117#comment-14639117 ] David Jacot commented on KAFKA-2357: Fixed documentation for 0.8.2 and 0.8.3. > Updat

Re: Review Request 36722: Patch for KAFKA-2332

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36722/#review92765 --- LGTM. It seems the Quota patch hasn't been checked in. Let's hold ba

Re: Review Request 36664: Patch for KAFKA-2353

> On July 22, 2015, 10:40 a.m., Ismael Juma wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 401 > > > > > > Is it intentional to ignore `java.lang.Error` too? > > Jiangjie Qin wrote: > I th

Re: Review Request 36664: Patch for KAFKA-2353

> On July 22, 2015, 10:40 a.m., Ismael Juma wrote: > > core/src/main/scala/kafka/network/SocketServer.scala, line 401 > > > > > > Is it intentional to ignore `java.lang.Error` too? > > Jiangjie Qin wrote: > I th

[jira] [Comment Edited] (KAFKA-2357) Update zookeeper.connect description in Kafka documentation

[ https://issues.apache.org/jira/browse/KAFKA-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639117#comment-14639117 ] David Jacot edited comment on KAFKA-2357 at 7/23/15 5:09 PM: -

Build failed in Jenkins: Kafka-trunk #553

See Changes: [cshapi] KAFKA-2355; Add an unit test to validate the deletion of a partition marked as deleted; patched by Edward Ribeiro, reviewed by Ashish Singh, Ismael Juma and Grant Henke -- [..

Re: Round Robin Among Consumers

As crazy as it sounds, there is an actual use-case there. Writing to HDFS can be very slow, so if you do a batch dump from a topic to HDFS, you may want more consumers reading from the topic than for "normal" streaming use-cases. We ended up modifying Camus to split a partition between multiple ma

Re: Build failed in Jenkins: KafkaPreCommit #161

Isamel, thats true. It will be nice to see this fixed, however as we are now moving to Github PRs should we start using travis CI? On Thu, Jul 23, 2015 at 9:51 AM, Ismael Juma wrote: > On Thu, Jul 23, 2015 at 5:39 PM, Apache Jenkins Server < > jenk...@builds.apache.org> wrote: > > > > BUILD FAIL

Re: Build failed in Jenkins: KafkaPreCommit #161

On Thu, Jul 23, 2015 at 6:39 PM, Ashish Singh wrote: > Isamel, thats true. It will be nice to see this fixed, however as we are > now moving to Github PRs should we start using travis CI? > We are currently using Jenkins for PR builds: https://builds.apache.org/job/kafka-trunk-git-pr/ It would

Re: Review Request 36722: Patch for KAFKA-2332

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36722/#review92778 --- core/src/main/scala/kafka/consumer/SimpleConsumer.scala (line 138)

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639307#comment-14639307 ] Jiangjie Qin commented on KAFKA-2350: - Ah, I see your concern. Originally I was thinki

[jira] [Assigned] (KAFKA-2066) Replace FetchRequest / FetchResponse with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reassigned KAFKA-2066: --- Assignee: Dong Lin > Replace FetchRequest / FetchResponse with their > org.apache.kafka.common.reque

[GitHub] kafka pull request: Kafka-1682: Adding zookeeper authentication. Z...

GitHub user Parth-Brahmbhatt opened a pull request: https://github.com/apache/kafka/pull/93 Kafka-1682: Adding zookeeper authentication. Zookeeper acls should be set so only broker can modify zk entries with the exception of /consumers. You can merge this pull request into a Git r

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639424#comment-14639424 ] Jason Gustafson commented on KAFKA-2350: [~becket_qin] It would help if you would

[jira] [Work started] (KAFKA-1695) Authenticate connection to Zookeeper

[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-1695 started by Parth Brahmbhatt. --- > Authenticate connection to Zookeeper > --

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639460#comment-14639460 ] Yasuhiro Matsuda commented on KAFKA-2350: - Suppose we are using auto assignment. {

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639457#comment-14639457 ] Dong Lin commented on KAFKA-1690: - [~harsha_ch] Thanks for the patch. I would like to test

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639475#comment-14639475 ] Sriharsha Chintalapani commented on KAFKA-1690: --- [~lindong] I am working on

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639477#comment-14639477 ] Dong Lin commented on KAFKA-1690: - [~harsha_ch] Cool. It will great if the patch can be av

Re: Round Robin Among Consumers

Ah, I see. Thanks for the use case, Gwen. I guess in that case it seems the time to use low level consumer. Jiangjie (Becket) Qin On Thu, Jul 23, 2015 at 9:52 AM, Gwen Shapira wrote: > As crazy as it sounds, there is an actual use-case there. > > Writing to HDFS can be very slow, so if you do a

Re: Round Robin Among Consumers

Agree. On Thu, Jul 23, 2015 at 2:43 PM, Jiangjie Qin wrote: > Ah, I see. Thanks for the use case, Gwen. I guess in that case it seems the > time to use low level consumer. > > Jiangjie (Becket) Qin > > On Thu, Jul 23, 2015 at 9:52 AM, Gwen Shapira wrote: > >> As crazy as it sounds, there is an a

[GitHub] kafka pull request: Added to .gitignore Kafka server logs director...

GitHub user sslavic opened a pull request: https://github.com/apache/kafka/pull/94 Added to .gitignore Kafka server logs directory When running Kafka server from sources, logs directory gets created in root of repository, and kafka server logs end up there. Currently that directory

[jira] [Commented] (KAFKA-2356) Support retrieving partitions of ConsumerRecords

[ https://issues.apache.org/jira/browse/KAFKA-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639594#comment-14639594 ] ASF GitHub Bot commented on KAFKA-2356: --- GitHub user sslavic opened a pull request:

[GitHub] kafka pull request: KAFKA-2356 Added support for retrieving partit...

GitHub user sslavic opened a pull request: https://github.com/apache/kafka/pull/95 KAFKA-2356 Added support for retrieving partitions of ConsumerRecords You can merge this pull request into a Git repository by running: $ git pull https://github.com/sslavic/kafka feature/KAFKA-

consumer property - zookeeper.sync.time.ms

Hi, Can someone please throw some light on the consumer property zookeeper.sync.time.ms? What are the implications of decreasing it to lower than 2 secs ? I read the description from documentation but could not understand it completely. Thanks, Kris

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639657#comment-14639657 ] Guozhang Wang commented on KAFKA-2350: -- Personally I would prefer not mixing the topi

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639665#comment-14639665 ] Jiangjie Qin commented on KAFKA-2350: - Hmm, good point. When rebalance come into place

Re: Review Request 36664: Patch for KAFKA-2353

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36664/#review92848 --- Ship it! Ship It! - Guozhang Wang On July 23, 2015, 12:51 a.m.,

[jira] [Commented] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639716#comment-14639716 ] Guozhang Wang commented on KAFKA-2353: -- Committed to trunk, thanks. > SocketServer.P

[jira] [Updated] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2353: - Fix Version/s: 0.8.3 > SocketServer.Processor should catch exception and close the socket properly

[jira] [Updated] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2353: - Resolution: Fixed Status: Resolved (was: Patch Available) > SocketServer.Processor should

[jira] [Commented] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639720#comment-14639720 ] Guozhang Wang commented on KAFKA-2353: -- I made a mistake on not changing the author..

[jira] [Commented] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639733#comment-14639733 ] Gwen Shapira commented on KAFKA-2353: - Perhaps: git commit --amend --author="User Name

Kafka Indentation

Hey everyone, I was wondering if it is possible to standardize indentation across clients and core. As an example, all the java code uses 4 spaces and the scala code 2. As we increasingly share code between clients and core, I think consistency would be super useful. Thoughts? Aditya

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639746#comment-14639746 ] Jiangjie Qin commented on KAFKA-2350: - [~guozhang], my understanding is that pause/unp

[jira] [Commented] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639748#comment-14639748 ] Guozhang Wang commented on KAFKA-2353: -- That seems not work once it is already pushed

Build failed in Jenkins: KafkaPreCommit #162

See Changes: [wangguoz] KAFKA-2353; SocketServer catch exception and close connection properly; reviewed by Gwen Shapira, Ismael Juma and Guozhang Wang -- [...truncated 1349 lines...] kafka.log.Br

Re: Kafka Indentation

For my part I actually prefer it how it is. The scala code is a different language, has different naming and capitalization conventions, etc. I actually like that it has a different look and feel so I don't get confused. In general, I think 4 spaces is standard for java and 2 spaces is pretty stand

[DISCUSS] KIP-28 - Add a transform client for data processing

Hi all, I just posted KIP-28: Add a transform client for data processing . The wiki page does not yet have the full design / implementation details, and this email is to kick-off the conversati

Jenkins build is back to normal : Kafka-trunk #554

See

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639796#comment-14639796 ] Jason Gustafson commented on KAFKA-2350: I agree that even if we use pause/unpause

[jira] [Commented] (KAFKA-2353) SocketServer.Processor should catch exception and close the socket properly in configureNewConnections.

[ https://issues.apache.org/jira/browse/KAFKA-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639854#comment-14639854 ] Jiangjie Qin commented on KAFKA-2353: - [~guozhang] I'm fine with that : ). Thanks a lo

Re: Round Robin Among Consumers

Gwen, I'm curious about this use case. Given the Kafka -> HDFS flow, it obviously relates to Copycat. More generally, this could be a problem even when streaming data if your processing takes too long such that your consumer simply can't keep up with the rate at which messages are produced. The "

Re: [VOTE] Switch to GitHub pull requests for new contributions

+1 (non-binding) One other thing that might be worth mentioning in "The Review Process" section is the hub tool: https://hub.github.com/ I only ever use one feature, but it's immensely useful if you're reviewing PRs on Github. You can just copy the URL into a command to checkout the PR branch: hu

Re: Review Request 36652: Patch for KAFKA-2351

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36652/ --- (Updated July 24, 2015, 4:36 a.m.) Review request for kafka. Bugs: KAFKA-2351

[jira] [Updated] (KAFKA-2351) Brokers are having a problem shutting down correctly

[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-2351: --- Attachment: KAFKA-2351_2015-07-23_21:36:52.patch > Brokers are having a problem shutting down

[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly

[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639910#comment-14639910 ] Mayuresh Gharat commented on KAFKA-2351: Updated reviewboard https://reviews.apach

[jira] [Updated] (KAFKA-2356) Support retrieving partitions of ConsumerRecords

[ https://issues.apache.org/jira/browse/KAFKA-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2356: - Reviewer: Guozhang Wang > Support retrieving partitions of ConsumerRecords > -

[jira] [Commented] (KAFKA-2356) Support retrieving partitions of ConsumerRecords

[ https://issues.apache.org/jira/browse/KAFKA-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639924#comment-14639924 ] Ewen Cheslack-Postava commented on KAFKA-2356: -- [~guozhang] Assigned you as r

  1   2   >