[jira] [Comment Edited] (KAFKA-1070) Auto-assign node id

[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085808#comment-14085808 ] Joe Stein edited comment on KAFKA-1070 at 8/5/14 5:08 AM: -- This s

[jira] [Comment Edited] (KAFKA-1070) Auto-assign node id

[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085808#comment-14085808 ] Joe Stein edited comment on KAFKA-1070 at 8/5/14 5:06 AM: -- This s

[jira] [Commented] (KAFKA-1070) Auto-assign node id

[ https://issues.apache.org/jira/browse/KAFKA-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085808#comment-14085808 ] Joe Stein commented on KAFKA-1070: -- This sounds like a very cool and VERY useful feature.

[jira] [Comment Edited] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085792#comment-14085792 ] Joe Stein edited comment on KAFKA-1387 at 8/5/14 4:41 AM: -- Here i

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085792#comment-14085792 ] Joe Stein commented on KAFKA-1387: -- Here is another way to reproduce this issue. I have

[jira] [Resolved] (KAFKA-1571) MetadataeTest hangs

[ https://issues.apache.org/jira/browse/KAFKA-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-1571. Resolution: Fixed Fix Version/s: 0.8.2 Thanks for the review. Committed to trunk. > MetadataeTest h

Re: Review Request 23895: Patch for KAFKA-1419

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23895/#review49559 --- Look good overall. We need to change the default scala version in

Re: Newer Zookeeper?

I found an already open ticket in regards to this https://issues.apache.org/jira/browse/KAFKA-1485 It also references a conflict with storm upgrading and testing and some other conflicts too. /*** Joe Stein Founder, Principal Consultant Big Data Open Sou

[jira] [Updated] (KAFKA-1485) Upgrade to Zookeeper 3.4.x

[ https://issues.apache.org/jira/browse/KAFKA-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1485: - Labels: newbie (was: ) > Upgrade to Zookeeper 3.4.x > -- > > Key

[jira] [Updated] (KAFKA-1485) Upgrade to Zookeeper 3.4.x

[ https://issues.apache.org/jira/browse/KAFKA-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1485: - Fix Version/s: 0.8.2 > Upgrade to Zookeeper 3.4.x > -- > > Key: K

Re: [New Feature Request] Ability to Inject Queue Implementation Async Mode

Is it possible there is another solution to the problem? I think if you could better describe the problem(s) you are facing and how you are architected some then you may get responses from others that perhaps have faced the same problem with similar architectures ... or maybe folks can chime in wit

Re: Newer Zookeeper?

Thanks Todd, very good to know and learn! - Joestein On Mon, Aug 4, 2014 at 9:44 PM, Todd Palino wrote: > We¹ve started running our test cluster against a Zookeeper 3.4.6 ensemble. > So far, we¹ve had no problems with it that were specific to ZK (since > we¹re using it for testing trunk version

Re: Uniform Distribution of Messages for Topic Across Partitions Without Effecting Performance

Bhavesh, take a look at https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified ? Maybe the root cause issue is something else? Even if producers produce more or less than what they are producing you should be able to

Re: Newer Zookeeper?

We¹ve started running our test cluster against a Zookeeper 3.4.6 ensemble. So far, we¹ve had no problems with it that were specific to ZK (since we¹re using it for testing trunk version of Kafka, as well as mirror maker, we have plenty of problems with it. Just none that are ZK). We¹re probably goi

Uniform Distribution of Messages for Topic Across Partitions Without Effecting Performance

How to achieve uniform distribution of non-keyed messages per topic across all partitions? We have tried to do this uniform distribution across partition using custom partitioning from each producer instance using round robing ( count(messages) % number of partition for topic). This strategy resul

[New Feature Request] Ability to Inject Queue Implementation Async Mode

Kafka Version: 0.8.x 1) Ability to define which messages get drooped (least recently instead of most recent in queue) 2) Try Unbounded Queue to find out the Upper Limit without drooping any messages for application (use case Stress test) 3) Priority Blocking Queue ( meaning a single Producer can

Re: Review Request 24287: Patch for KAFKA-1571

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24287/#review49547 --- Ship it! Ship It! - Guozhang Wang On Aug. 5, 2014, 12:39 a.m., J

[jira] [Updated] (KAFKA-1571) MetadataeTest hangs

[ https://issues.apache.org/jira/browse/KAFKA-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1571: --- Attachment: KAFKA-1571.patch > MetadataeTest hangs > --- > > Key: KAFKA-1571

[jira] [Commented] (KAFKA-1571) MetadataeTest hangs

[ https://issues.apache.org/jira/browse/KAFKA-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085576#comment-14085576 ] Jun Rao commented on KAFKA-1571: Created reviewboard https://reviews.apache.org/r/24287/

Review Request 24287: Patch for KAFKA-1571

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24287/ --- Review request for kafka. Bugs: KAFKA-1571 https://issues.apache.org/jira/b

[jira] [Created] (KAFKA-1571) MetadataeTest hangs

Jun Rao created KAFKA-1571: -- Summary: MetadataeTest hangs Key: KAFKA-1571 URL: https://issues.apache.org/jira/browse/KAFKA-1571 Project: Kafka Issue Type: Bug Components: core Affects

[jira] [Updated] (KAFKA-1569) Create tool to test correctness of transactions end-to-end

[ https://issues.apache.org/jira/browse/KAFKA-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1569: - Attachment: KAFKA-1569.patch > Create tool to test correctness of transactions en

[jira] [Commented] (KAFKA-1569) Create tool to test correctness of transactions end-to-end

[ https://issues.apache.org/jira/browse/KAFKA-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085385#comment-14085385 ] Raul Castro Fernandez commented on KAFKA-1569: -- Created reviewboard https://r

Re: Issues getting IntelliJ set up

Yep, it looks like you win the game. Managed to get a test running by doing this. Thanks a bunch, Gwen! On Mon, Aug 4, 2014 at 1:57 PM, Gwen Shapira wrote: > I think I found it :) > > Go to "project settings" > Pick your module (I picked "core") > "Paths" tab > And change the paths to: > /Users

Review Request 24268: Patch for KAFKA-1569

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24268/ --- Review request for kafka. Bugs: KAFKA-1569 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1541) Add transactional request definitions to clients package

[ https://issues.apache.org/jira/browse/KAFKA-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085369#comment-14085369 ] Raul Castro Fernandez commented on KAFKA-1541: -- Created reviewboard https://r

[jira] [Updated] (KAFKA-1541) Add transactional request definitions to clients package

[ https://issues.apache.org/jira/browse/KAFKA-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1541: - Attachment: KAFKA-1541.patch > Add transactional request definitions to clients

Review Request 24267: Patch for KAFKA-1541

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24267/ --- Review request for kafka. Bugs: KAFKA-1541 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-1524) Implement transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085352#comment-14085352 ] Raul Castro Fernandez commented on KAFKA-1524: -- Created reviewboard https://r

[jira] [Updated] (KAFKA-1524) Implement transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1524: - Attachment: KAFKA-1524.patch > Implement transactional producer > ---

Review Request 24265: Patch for KAFKA-1524

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24265/ --- Review request for kafka. Bugs: KAFKA-1524 https://issues.apache.org/jira/b

Re: Issues getting IntelliJ set up

I think I found it :) Go to "project settings" Pick your module (I picked "core") "Paths" tab And change the paths to: /Users/Natty/apache/kafka-new/core/build/classes/main and /Users/Natty/apache/kafka-new/core/build/classes/test Don't use the project inherited paths, because we have a build dir

[jira] [Updated] (KAFKA-1570) sbt assembly-package-dependency fails with errors

[ https://issues.apache.org/jira/browse/KAFKA-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuri Nagarin updated KAFKA-1570: Component/s: (was: website) build Description: xnag@xnag

Re: Newer Zookeeper?

If Kafka installations are missing something(s) by not having or using the latest Zookeeper from a feature or stability perspective that would be something to understand maybe you could help with that Gwen? I know one of the implementations used this Hadoop version http://docs.hortonworks.com/HDPD

[jira] [Commented] (KAFKA-1569) Create tool to test correctness of transactions end-to-end

[ https://issues.apache.org/jira/browse/KAFKA-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085193#comment-14085193 ] Raul Castro Fernandez commented on KAFKA-1569: -- Created reviewboard https://r

[jira] [Updated] (KAFKA-1569) Create tool to test correctness of transactions end-to-end

[ https://issues.apache.org/jira/browse/KAFKA-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1569: - Attachment: KAFKA-1569.patch > Create tool to test correctness of transactions en

[jira] [Created] (KAFKA-1570) sbt assembly-package-dependency fails with errors

Xuri Nagarin created KAFKA-1570: --- Summary: sbt assembly-package-dependency fails with errors Key: KAFKA-1570 URL: https://issues.apache.org/jira/browse/KAFKA-1570 Project: Kafka Issue Type: Bug

Review Request 24255: Patch for KAFKA-1569

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24255/ --- Review request for kafka. Bugs: KAFKA-1569 https://issues.apache.org/jira/b

Re: Issues getting IntelliJ set up

I did. I actually tried this from a completely clean repo (cloned a new repo from github, changed gradle.properties, ran `gradlew idea`, then imported into IntelliJ) On Mon, Aug 4, 2014 at 12:18 PM, Timothy Chen wrote: > Hi Johnathan, > > Did you update your scala version before you run gradle

[jira] [Updated] (KAFKA-1569) Create tool to test correctness of transactions end-to-end

[ https://issues.apache.org/jira/browse/KAFKA-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1569: - Summary: Create tool to test correctness of transactions end-to-end (was: Create

[jira] [Updated] (KAFKA-1569) Create tool to test end-to-end correctness when using transactions

[ https://issues.apache.org/jira/browse/KAFKA-1569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1569: - Summary: Create tool to test end-to-end correctness when using transactions (was

[jira] [Commented] (KAFKA-1541) Add transactional request definitions to clients package

[ https://issues.apache.org/jira/browse/KAFKA-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085171#comment-14085171 ] Raul Castro Fernandez commented on KAFKA-1541: -- Created reviewboard https://r

[jira] [Updated] (KAFKA-1541) Add transactional request definitions to clients package

[ https://issues.apache.org/jira/browse/KAFKA-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1541: - Attachment: KAFKA-1541.patch > Add transactional request definitions to clients

Review Request 24253: Patch for KAFKA-1541

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24253/ --- Review request for kafka. Bugs: KAFKA-1541 https://issues.apache.org/jira/b

Re: Issues getting IntelliJ set up

Hi Johnathan, Did you update your scala version before you run gradle idea? Also try cleaning up all the artifacts and try it again, as perhaps your intellij is not picking up the right version and from the right build folder. Tim On Mon, Aug 4, 2014 at 12:09 PM, Jonathan Natkins wrote: > Hi,

Issues getting IntelliJ set up

Hi, I've been having some issues getting IntelliJ set up...I followed all the instructions on the wiki, and I've successfully imported the project, and run the jar Gradle target successfully. However, when I try to run a test in the IDE, I get a number of errors: /Users/Natty/apache/kafka-new/cor

[jira] Subscription: outstanding kafka patches

Issue Subscription Filter: outstanding kafka patches (117 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-1567 Metric memory leaking after closing the clients https://issues.apache.org/jira/browse/KAFKA-1567 KAFKA-1561 Data Loss

[jira] [Updated] (KAFKA-1524) Implement transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Castro Fernandez updated KAFKA-1524: - Attachment: KAFKA-1524.patch > Implement transactional producer > ---

[jira] [Commented] (KAFKA-1524) Implement transactional producer

[ https://issues.apache.org/jira/browse/KAFKA-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085063#comment-14085063 ] Raul Castro Fernandez commented on KAFKA-1524: -- Created reviewboard https://r

Review Request 24245: Patch for KAFKA-1524

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24245/ --- Review request for kafka. Bugs: KAFKA-1524 https://issues.apache.org/jira/b

[jira] [Created] (KAFKA-1569) Creat tool to test end-to-end correctness in transactional mode

Raul Castro Fernandez created KAFKA-1569: Summary: Creat tool to test end-to-end correctness in transactional mode Key: KAFKA-1569 URL: https://issues.apache.org/jira/browse/KAFKA-1569 Project

Re: Newer Zookeeper?

Also, specific Zookeeper 3.4.X version where loss of quorum occurred will help. 3.4.5 fixed some pretty serious issues around hanging. Gwen On Mon, Aug 4, 2014 at 9:29 AM, Gwen Shapira wrote: > Thanks for the heads-up, Joe. > > We've been shipping Zookeeper 3.4.X for over two years now (since >

Re: Newer Zookeeper?

Thanks for the heads-up, Joe. We've been shipping Zookeeper 3.4.X for over two years now (since CDH4.0) and have many production customers. I'll check if there are any known issues with breaking quorum. In any case I will take your comments into account and see if I can arrange for extra testing.

Build failed in Jenkins: Kafka-trunk #238

See Changes: [junrao] kafka-1562; kafka-topics.sh alter add partitions resets cleanup.policy; patched by Jonathan Natkins; reviewed by Jun Rao -- [...truncated 671 lines...] there were 12 feature war

[jira] [Commented] (KAFKA-1566) Kafka environment configuration (kafka-env.sh)

[ https://issues.apache.org/jira/browse/KAFKA-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084788#comment-14084788 ] Cosmin Lehene commented on KAFKA-1566: -- Somehow related: KAFKA_LOG_DIR would be bette

[jira] [Comment Edited] (KAFKA-1499) Broker-side compression configuration

[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084777#comment-14084777 ] Manikumar Reddy edited comment on KAFKA-1499 at 8/4/14 3:43 PM:

[jira] [Assigned] (KAFKA-1499) Broker-side compression configuration

[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy reassigned KAFKA-1499: -- Assignee: Manikumar Reddy > Broker-side compression configuration > ---

[jira] [Commented] (KAFKA-1566) Kafka environment configuration (kafka-env.sh)

[ https://issues.apache.org/jira/browse/KAFKA-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084778#comment-14084778 ] Cosmin Lehene commented on KAFKA-1566: -- The convention is to have it in conf/ We can

[jira] [Commented] (KAFKA-1499) Broker-side compression configuration

[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084777#comment-14084777 ] Manikumar Reddy commented on KAFKA-1499: I would like to propose new server proper

Re: Newer Zookeeper?

I have heard issues from installations running 3.4.X that I have not heard from installations running 3.3.X (i.e. zk breaking quorum and cluster going down). In none of these cases did I have an opportunity to isolate and reproduce and confirm the issue happening and caused by 3.4.X. Moving to 3.3

[jira] [Commented] (KAFKA-1419) cross build for scala 2.11

[ https://issues.apache.org/jira/browse/KAFKA-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084715#comment-14084715 ] Ivan Lyutov commented on KAFKA-1419: Updated reviewboard https://reviews.apache.org/r/

[jira] [Updated] (KAFKA-1419) cross build for scala 2.11

[ https://issues.apache.org/jira/browse/KAFKA-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Lyutov updated KAFKA-1419: --- Attachment: KAFKA-1419_2014-08-04_14:43:26.patch > cross build for scala 2.11 > -

Re: Review Request 23895: Patch for KAFKA-1419

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23895/ --- (Updated Aug. 4, 2014, 2:43 p.m.) Review request for kafka. Bugs: KAFKA-1419

Re: Review Request 23895: Patch for KAFKA-1419

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23895/#review49454 --- build.gradle

[jira] [Updated] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

[ https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1562: --- Resolution: Fixed Fix Version/s: 0.8.2 Assignee: Jonathan Natkins Status: Resolved

[jira] [Commented] (KAFKA-1019) kafka-preferred-replica-election.sh will fail without clear error message if /brokers/topics/[topic]/partitions does not exist

[ https://issues.apache.org/jira/browse/KAFKA-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084641#comment-14084641 ] Mickael Hemri commented on KAFKA-1019: -- I tried with zookeeper 3.3.6 and we have the