[jira] [Commented] (KAFKA-1171) Gradle build for Kafka

[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881635#comment-13881635 ] Joe Stein commented on KAFKA-1171: -- yup, will take a look. > Gradle build for Kafka >

[jira] [Commented] (KAFKA-1171) Gradle build for Kafka

[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881633#comment-13881633 ] Jun Rao commented on KAFKA-1171: Joe Stein, Do you think you can take a look at maven pub

[jira] [Updated] (KAFKA-1171) Gradle build for Kafka

[ https://issues.apache.org/jira/browse/KAFKA-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1171: --- Attachment: kafka-1171_v6.patch I made a pass of this. Attach the latest patch v6, which includes: 1. added a

[jira] [Closed] (KAFKA-1055) BrokerTopicStats is updated before checking for MessageSizeTooLarge

[ https://issues.apache.org/jira/browse/KAFKA-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy closed KAFKA-1055. - > BrokerTopicStats is updated before checking for MessageSizeTooLarge > -

[jira] [Resolved] (KAFKA-1055) BrokerTopicStats is updated before checking for MessageSizeTooLarge

[ https://issues.apache.org/jira/browse/KAFKA-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-1055. --- Resolution: Fixed Committed to trunk. > BrokerTopicStats is updated before checking for MessageSizeT

Re: Review Request 17055: Patch for KAFKA-1055

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17055/#review32772 --- Ship it! - Joel Koshy On Jan. 21, 2014, 9:29 p.m., Guozhang Wang

Re: producer rewrite

Same here: +1 on checking in as is. I should be able to go through the patch on Monday. On Fri, Jan 24, 2014 at 01:48:14PM -0800, Neha Narkhede wrote: > I plan to review the code over the weekend and I think it is easier to > address the review comments async. +1 for checking it in as is for now.

Re: New Producer Public API

Clark and all, I thought a little bit about the serialization question. Here are the options I see and the pros and cons I can think of. I'd love to hear people's preferences if you have a strong one. One important consideration is that however the producer works will also need to be how the new

Re: New Producer Public API

Yeah I'll fix that name. Hmm, yeah, I agree that often you want to be able delay network connectivity until you have started everything up. But at the same time I kind of loath special init() methods because you always forget to call them and get one round of error every time. I wonder if in those

Re: New Producer Public API

Jay, Thanks for the explanation. I didn't realize that the broker list was for bootstrapping and was not required to be a complete list of all brokers (although I see now that it's clearly stated in the text description of the parameter). Nonetheless, does it still make sense to make the config

Re: New Producer Public API

Roger, These are good questions. 1. The producer since 0.8 is actually zookeeper free, so this is not new to this client it is true for the current client as well. Our experience was that direct zookeeper connections from zillions of producers wasn't a good idea for a number of reasons. Our inten

[jira] [Commented] (KAFKA-1229) Reload broker config without a restart

[ https://issues.apache.org/jira/browse/KAFKA-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881541#comment-13881541 ] Guozhang Wang commented on KAFKA-1229: -- Hello Carlo, Which broker config do you have

Re: New Producer Public API

A couple comments: 1) Why does the config use a broker list instead of discovering the brokers in ZooKeeper? It doesn't match the HighLevelConsumer API. 2) It looks like broker connections are created on demand. I'm wondering if sometimes you might want to flush out config or network connectivi

[jira] [Created] (KAFKA-1229) Reload broker config without a restart

Carlo Cabanilla created KAFKA-1229: -- Summary: Reload broker config without a restart Key: KAFKA-1229 URL: https://issues.apache.org/jira/browse/KAFKA-1229 Project: Kafka Issue Type: Wish

Re: [jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

Working on the above. Since trunk currently allows Topic configs, and altering topics, does it make sense to you to make this a topic config so that it is kept in sync between topic alterations and partition adds? For compatibility: I can make the zookeeper state parsing backwards / forwards comp

Re: producer rewrite

I plan to review the code over the weekend and I think it is easier to address the review comments async. +1 for checking it in as is for now. Thanks, Neha On Fri, Jan 24, 2014 at 1:36 PM, Jay Kreps wrote: > So folks there are some comments on the RB, I take it from this discussion > people ar

Re: producer rewrite

So folks there are some comments on the RB, I take it from this discussion people are cool with me just checking in what I have and addressing the comments asynchronously? If no objection I will do that. -Jay On Thu, Jan 23, 2014 at 12:56 PM, Jay Kreps wrote: > Cool, I've uploaded a patch and

Re: producer rewrite

The JIRA has the patch and the RB is for comments. -Jay On Fri, Jan 24, 2014 at 9:00 AM, S Ahmed wrote: > Thanks. This is the page correct? > https://reviews.apache.org/r/17263/diff/raw/ > > I was hoping I could download the code and run it locally not just view it. > > > On Thu, Jan 23, 2014

Re: Simple Auth and Auth

No it’s not based on sasl - will upload to a jira ticket and let you know Cheers Joe On 24 Jan 2014, at 16:27, Jun Rao wrote: > Joe, > > Thanks for looking into this. We do plan to support security. However, > timing-wise, this probably has to be done after the client rewrites, which > change

code layout for new clients

Hey All, Another topic worth discussing is how to layout code for the new producer and consumer as well as the common code they will share. There are really three questions: 1. Which top-level sub-modules/directories should we have (currently everything is under core, but presumably we want to spl

New Producer Public API

As mentioned in a previous email we are working on a re-implementation of the producer. I would like to use this email thread to discuss the details of the public API and the configuration. I would love for us to be incredibly picky about this public api now so it is as good as possible and we don'

[jira] [Commented] (KAFKA-1212) System test exception handling does not stop background producer threads

[ https://issues.apache.org/jira/browse/KAFKA-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881358#comment-13881358 ] Guozhang Wang commented on KAFKA-1212: -- Updated reviewboard https://reviews.apache.or

[jira] [Updated] (KAFKA-1212) System test exception handling does not stop background producer threads

[ https://issues.apache.org/jira/browse/KAFKA-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1212: - Attachment: KAFKA-1212_2014-01-24_11:43:53.patch > System test exception handling does not stop b

Re: Review Request 17006: Keep track of local pid and kill it on stopping all entities as discussed with John

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17006/ --- (Updated Jan. 24, 2014, 7:44 p.m.) Review request for kafka. Bugs: KAFKA-1212

[jira] [Commented] (KAFKA-1028) per topic configuration of preference for consistency over availability

[ https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881326#comment-13881326 ] Andrew Olson commented on KAFKA-1028: - Right, it's a broker config property (default b

[jira] [Commented] (KAFKA-1028) per topic configuration of preference for consistency over availability

[ https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881297#comment-13881297 ] Jason Rosenberg commented on KAFKA-1028: Thanks for the explanation [~noslowerdna]

[jira] [Commented] (KAFKA-1028) per topic configuration of preference for consistency over availability

[ https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881262#comment-13881262 ] Andrew Olson commented on KAFKA-1028: - Yes, it is a more robust solution than simply r

[jira] [Commented] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881211#comment-13881211 ] Jun Rao commented on KAFKA-1228: Thanks for the patch. We probably should do the same on t

[jira] [Resolved] (KAFKA-1214) Support arguments to zookeeper-shell.sh script

[ https://issues.apache.org/jira/browse/KAFKA-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-1214. Resolution: Fixed Fix Version/s: 0.8.1 Assignee: Philip K. Warren Thanks for the patch. +1.

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881193#comment-13881193 ] Jun Rao commented on KAFKA-1215: Thanks for the patch. Looks good over all. Some comments.

[jira] [Commented] (KAFKA-1028) per topic configuration of preference for consistency over availability

[ https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881170#comment-13881170 ] Jason Rosenberg commented on KAFKA-1028: Can you clarify, is this different than u

Re: Review Request 17006: Keep track of local pid and kill it on stopping all entities as discussed with John

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17006/#review32722 --- system_test/utils/kafka_system_test_utils.py

Re: producer rewrite

Thanks. This is the page correct? https://reviews.apache.org/r/17263/diff/raw/ I was hoping I could download the code and run it locally not just view it. On Thu, Jan 23, 2014 at 8:02 PM, Neha Narkhede wrote: > There is a link to the reviewboard url on the > JIRA

[jira] [Updated] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmy Yulrizka updated KAFKA-1228: - Attachment: KAFKA-1228-2014-01-24.patch Catch all exception and close socket connection > Socket

[jira] [Updated] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmy Yulrizka updated KAFKA-1228: - Component/s: (was: network) consumer > Socket Leak on ReplicaFetcherThread >

[jira] [Issue Comment Deleted] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmy Yulrizka updated KAFKA-1228: - Comment: was deleted (was: Catch all exception and close socket connection) > Socket Leak on Rep

Re: Simple Auth and Auth

Joe, Thanks for looking into this. We do plan to support security. However, timing-wise, this probably has to be done after the client rewrites, which changes how the client uses sockets and potentially adds new request types. Our current estimate is that the producer rewrite will take probably ab

[jira] [Updated] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmy Yulrizka updated KAFKA-1228: - Status: Patch Available (was: Open) Catch all exception and close the socket > Socket Leak on R

[jira] [Updated] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmy Yulrizka updated KAFKA-1228: - Status: Patch Available (was: Open) Close socket connection on any Exception > Socket Leak on R

[jira] [Updated] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmy Yulrizka updated KAFKA-1228: - Status: Open (was: Patch Available) > Socket Leak on ReplicaFetcherThread >

[jira] [Created] (KAFKA-1228) Socket Leak on ReplicaFetcherThread

Ahmy Yulrizka created KAFKA-1228: Summary: Socket Leak on ReplicaFetcherThread Key: KAFKA-1228 URL: https://issues.apache.org/jira/browse/KAFKA-1228 Project: Kafka Issue Type: Bug C

Simple Auth and Auth

Hi All I’ve been looking at the wiki proposal to add Auth and Auth to kafka https://cwiki.apache.org/confluence/display/KAFKA/Security In the meantime I’ve had a recent immediate requirement to implement something similar - my solution is detailed below - fairly quick and dirty but achieved t

[jira] [Commented] (KAFKA-1028) per topic configuration of preference for consistency over availability

[ https://issues.apache.org/jira/browse/KAFKA-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881062#comment-13881062 ] Andrew Olson commented on KAFKA-1028: - I am currently working on implementing this enh