Re: [DISCUSS] KIP-11- Authorization design for kafka security

2015-03-18 Thread Parth Brahmbhatt
Hi Michael, Thanks for taking the time to review. Currently I did not plan on adding “Deny” but I guess it can’t hurt except for adding more constructs would probably make the acls more complex. When a topic is created with no acls provided , I was planning to add a default ACL which would allow

Re: Review Request 31742: Patch for KAFKA-527

2015-03-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31742/#review76985 --- Thanks for the patch. Could we test out the patch with StressTestLo

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Steven Wu
> the quota is against all of those instances added up across all their topics ah. I miss this point earlier. so you intend to enforce quota at client-id level cross all topics. I thought quota is at per-topic level. when we have sudden surge of traffic, it is usually just a topic (not all topics

Re: [VOTE] KIP-7 Security - IP Filtering

2015-03-18 Thread Jeff Holoman
One other thought. Does the timing of the implementation (or lack thereof) affect the proposal? It seems like the question you are asking is an implementation detail in terms of when the work would be done. If there isn't really support for the KIP that's ok, just wanting to make sure we are segmen

Re: [VOTE] KIP-7 Security - IP Filtering

2015-03-18 Thread Jeff Holoman
Hey Jun thanks for the comment. Is the plan to re-factor the SocketServer implementation significantly? The current check is just in the acceptor. Does this change with the refactor? Thanks Jeff On Wed, Mar 18, 2015 at 7:25 PM, Jun Rao wrote: > The proposal sounds reasonable. Timing wise,

[jira] [Commented] (KAFKA-2031) Executing scripts that invoke kafka-run-class.sh results in 'permission denied to create log dir' warning.

2015-03-18 Thread Manikandan Narayanaswamy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368285#comment-14368285 ] Manikandan Narayanaswamy commented on KAFKA-2031: - Great! Thanks for resol

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Gwen Shapira
We can't rip them out completely, unfortunately - the SimpleConsumer uses them. So we'll need conversion at some point. I'll try to make the conversion point "just before hitting a public API that we can't modify", and hopefully it won't look too arbitrary. On Wed, Mar 18, 2015 at 5:24 PM, Jay

[jira] [Resolved] (KAFKA-2031) Executing scripts that invoke kafka-run-class.sh results in 'permission denied to create log dir' warning.

2015-03-18 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani resolved KAFKA-2031. --- Resolution: Duplicate > Executing scripts that invoke kafka-run-class.sh resul

[jira] [Commented] (KAFKA-2031) Executing scripts that invoke kafka-run-class.sh results in 'permission denied to create log dir' warning.

2015-03-18 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368239#comment-14368239 ] Sriharsha Chintalapani commented on KAFKA-2031: --- [~mnarayan] we've a JIRA to

[jira] [Commented] (KAFKA-2030) Creating a topic cause Unsupported major.minor version 52.0

2015-03-18 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368224#comment-14368224 ] Ewen Cheslack-Postava commented on KAFKA-2030: -- It looks like the homebrew bo

[jira] [Resolved] (KAFKA-2030) Creating a topic cause Unsupported major.minor version 52.0

2015-03-18 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2030. -- Resolution: Invalid > Creating a topic cause Unsupported major.minor version 52.

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Jay Kreps
I think either approach is okay in the short term. However our goal should be to eventually get rid of that duplicate code, so if you are up for just ripping and cutting that may get us there sooner. -Jay On Wed, Mar 18, 2015 at 5:19 PM, Gwen Shapira wrote: > Thanks! > > Another clarification:

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Gwen Shapira
Thanks! Another clarification: The Common request/responses use slightly different infrastructure objects: Node instead of Broker, TopicPartition instead of TopicAndPartition and few more. I can write utilities to convert Node to Broker to minimize the scope of the change. Or I can start replacin

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Jay Kreps
I think that is a good summary. 1. I'd favor delay over error--I think we have a design where delay will basically mimic the same behavior you would get if you had a lower capacity Kafka cluster all to yourself, which from my point of view is ideal. I'm aesthetically opposed to delay+error. 2. I a

[jira] [Created] (KAFKA-2031) Executing scripts that invoke kafka-run-class.sh results in 'permission denied to create log dir' warning.

2015-03-18 Thread Manikandan Narayanaswamy (JIRA)
Manikandan Narayanaswamy created KAFKA-2031: --- Summary: Executing scripts that invoke kafka-run-class.sh results in 'permission denied to create log dir' warning. Key: KAFKA-2031 URL: https://issues.apach

Re: [VOTE] KIP-7 Security - IP Filtering

2015-03-18 Thread Jun Rao
The proposal sounds reasonable. Timing wise, since we plan to refactor the network layer code in the broker, perhaps this can wait until KAFKA-1928 is done? Thanks, Jun On Tue, Mar 17, 2015 at 6:56 AM, Jeff Holoman wrote: > bump > > On Tue, Mar 3, 2015 at 8:12 PM, Jeff Holoman > wrote: > > >

[jira] [Assigned] (KAFKA-1684) Implement TLS/SSL authentication

2015-03-18 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned KAFKA-1684: - Assignee: Sriharsha Chintalapani (was: Ivan Lyutov) > Implement TLS/SSL

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Joel Koshy
+1 as well. I think it helps to keep the rerouting approach orthogonal to this KIP. On Wed, Mar 18, 2015 at 03:40:48PM -0700, Jay Kreps wrote: > I'm +1 on Jun's suggestion as long as it can work for all the requests. > > On Wed, Mar 18, 2015 at 3:35 PM, Jun Rao wrote: > > > Andrii, > > > > I th

[jira] [Created] (KAFKA-2030) Creating a topic cause Unsupported major.minor version 52.0

2015-03-18 Thread Jeff Fogarty (JIRA)
Jeff Fogarty created KAFKA-2030: --- Summary: Creating a topic cause Unsupported major.minor version 52.0 Key: KAFKA-2030 URL: https://issues.apache.org/jira/browse/KAFKA-2030 Project: Kafka Issu

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Aditya Auradkar
Hey everyone, Thanks for the great discussion. There are currently a few points on this KIP that need addressing and I want to make sure we are on the same page about those. 1. Append and delay response vs delay and return error - I think we've discussed the pros and cons of each approach but h

Build failed in Jenkins: KafkaPreCommit #40

2015-03-18 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-1997; Hopefully last follow-up fix to get messageHandlerArgs right -- [...truncated 3501 lines...] at org.apache.zookeeper.server.NIOServerCnxnFactory.configu

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Jay Kreps
I'm +1 on Jun's suggestion as long as it can work for all the requests. On Wed, Mar 18, 2015 at 3:35 PM, Jun Rao wrote: > Andrii, > > I think we agreed on the following. > > (a) Admin requests can be sent to and handled by any broker. > (b) Admin requests are processed asynchronously, at least f

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Jun Rao
Andrii, I think we agreed on the following. (a) Admin requests can be sent to and handled by any broker. (b) Admin requests are processed asynchronously, at least for now. That is, when the client gets a response, it just means that the request is initiated, but not necessarily completed. Then, i

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Jay Kreps
Great. For (3) yeah I think we should just think through the end-to-end pattern for these versioned requests since it seems like we will have a number of them. The serialization code used as you described gets us to the right Struct which the user would then wrap in something like ProduceRequest.

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Joel Koshy
Yes that is what I was alluding to when I said that if we finally do request rerouting in Kafka then the field would add little to no value. I wasn't sure if we agreed that we _will_ do rerouting or whether we agreed to evaluate it (KAFKA-1912). Andrii can you update the KIP with this? Thanks, Jo

Build failed in Jenkins: Kafka-trunk #427

2015-03-18 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-1997; Hopefully last follow-up fix to get messageHandlerArgs right -- [...truncated 1921 lines...] at kafka.integration.PrimitiveApiTest.setUp(PrimitiveApiTest.s

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Jun Rao
3. The way that things are working right now is that there is a single request object for all versions. The layout of the request object always corresponds to the latest version. Under normal version evolution, the request object should be able to be constructed the binary of any version. For examp

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Andrii Biletskyi
Jun, I might be wrong but didn't we agree we will let any broker from the cluster handle *long-running* admin requests (at this time preferredReplica and reassignPartitions), via zk admin path. Thus CreateTopics etc should be sent only to the controller. Thanks, Andrii Biletskyi On Wed, Mar 18,

Re: Review Request 32193: Patch for KAFKA-1997

2015-03-18 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32193/#review76955 --- Ship it! Ship It! - Guozhang Wang On March 18, 2015, 7:47 p.m.,

Re: Review Request 31568: Patch for KAFKA-1989

2015-03-18 Thread Guozhang Wang
> On March 17, 2015, 8:56 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/server/DelayedOperation.scala, lines 187-192 > > > > > > TBD > > Guozhang Wang wrote: > Replace the TBD here: we can let Timer.add

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Jun Rao
Joel, Andril, I think we agreed that those admin requests can be issued to any broker. Because of that, there doesn't seem to be a strong need to know the controller. So, perhaps we can proceed by not making any change to the format of TMR right now. When we start using create topic request in the

Re: [DISCUSSION] KIP-15 close(timeout) for producer

2015-03-18 Thread Jay Kreps
Personally I'm in favor of (1) just to reduce the number of different APIs. People will find the difference between abort and close subtle and confusing and the only instance where you want it is this somewhat unusual case you guys are pursuing, right? -Jay On Wed, Mar 18, 2015 at 2:13 PM, Jiangj

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Gwen Shapira
See inline responses: On Wed, Mar 18, 2015 at 2:26 PM, Jay Kreps wrote: > Hey Gwen, > > This makes sense to me. > > A couple of thoughts, mostly confirming what you said I think: > >1. Ideally we would move completely over to the new style of request >definition for server-side processing

Re: [DISCUSS] KIP-5 - Broker Configuration Management

2015-03-18 Thread Jay Kreps
The other question to figure out is the hierarchy of overrides. Currently some configs can be overridden at the topic level using a fairly ad hoc mechanism. I think Aditya's quota proposal requires having defaults and overrides at the client.id level. You could imagine similar requirements at the u

[jira] [Commented] (KAFKA-1912) Create a simple request re-routing facility

2015-03-18 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367966#comment-14367966 ] Jay Kreps commented on KAFKA-1912: -- 1. I think this can be a single thread. The number of

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Jay Kreps
Hey Gwen, This makes sense to me. A couple of thoughts, mostly confirming what you said I think: 1. Ideally we would move completely over to the new style of request definition for server-side processing, even for the internal requests. This way all requests would have the same header/b

Re: [DISCUSS] KIP-11- Authorization design for kafka security

2015-03-18 Thread Michael Herstine
Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in c

Re: [DISCUSSION] KIP-15 close(timeout) for producer

2015-03-18 Thread Jiangjie Qin
It looks we have another option and are now deciding between the following two interfaces: 1. Close() + close(timeout) - timeout could be either positive or zero. - only close(0) can be called from sender thread 2. Close() + abort() + close(timeout) - timeout can either be positive or zero

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Jay Kreps
Hey Steven, The current proposal is actually to enforce quotas at the client/application level, NOT the topic level. So if you have a service with a few dozen instances the quota is against all of those instances added up across all their topics. So actually the effect would be the same either way

[jira] [Commented] (KAFKA-1481) Stop using dashes AND underscores as separators in MBean names

2015-03-18 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367878#comment-14367878 ] Guozhang Wang commented on KAFKA-1481: -- We hit an issue related to this ticket, which

[jira] [Commented] (KAFKA-1997) Refactor Mirror Maker

2015-03-18 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367761#comment-14367761 ] Jiangjie Qin commented on KAFKA-1997: - Updated reviewboard https://reviews.apache.org/

[jira] [Updated] (KAFKA-1997) Refactor Mirror Maker

2015-03-18 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1997: Attachment: KAFKA-1997_2015-03-18_12:47:32.patch > Refactor Mirror Maker > - > >

Re: Review Request 32193: Patch for KAFKA-1997

2015-03-18 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32193/ --- (Updated March 18, 2015, 7:47 p.m.) Review request for kafka. Bugs: KAFKA-199

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Andrii Biletskyi
Joel, I'm totally behind your arguments concerning adding irrelevant stuff to TopicMetadataRequest. And also about having a bloated request. Personally I'd go with a separate ClusterMetadataRequest (CMR), actually this was our initial proposal. But since the second part of the request - brokers i

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Jun Rao
Yes, that looks reasonable. Then, in KafkaApi, when we get into the handling of a specific request, we can construct the specific request object from struct. Thanks, Jun On Wed, Mar 18, 2015 at 11:11 AM, Gwen Shapira wrote: > Hi Jun, > > I was taking a slightly different approach. Let me know

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Gwen Shapira
OK, Andrii clarified that we need KAFKA-1927 before KIP-4 for the infrastructure for using the common request/response classes in core. Jun, when you got a moment, please confirm if the approach I'm taking is acceptable, or if you see issues that I'm missing. On Wed, Mar 18, 2015 at 11:33 AM, G

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Gwen Shapira
Got it. Thanks for clarifying! On Wed, Mar 18, 2015 at 11:54 AM, Andrii Biletskyi wrote: > Gwen, > > Yes, looks like KAFKA-1927 will leave TopicMetadataRequest/Response. > But I believe, KIP is still tightly related with KAFKA-1927 since we are > not only > going to update TopicMetadataRequest th

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Joel Koshy
(Thanks Andrii for the summary) For (1) yes we will circle back on that shortly after syncing up in person. I think it is close to getting committed although development for KAFKA-1927 can probably begin without it. There is one more item we covered at the hangout. i.e., whether we want to add th

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Andrii Biletskyi
Gwen, Yes, looks like KAFKA-1927 will leave TopicMetadataRequest/Response. But I believe, KIP is still tightly related with KAFKA-1927 since we are not only going to update TopicMetadataRequest there but we will introduce a bunch of new requests too. And it probably makes sense to do those correct

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

2015-03-18 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367645#comment-14367645 ] Parth Brahmbhatt commented on KAFKA-1688: - [~junrao][~jkreps] Can you guys comment

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Gwen Shapira
On Wed, Mar 18, 2015 at 9:34 AM, Jun Rao wrote: > Andri, > > Thanks for the summary. > > 1. I just realized that in order to start working on KAFKA-1927, we will > need to merge the changes to OffsetCommitRequest (from 0.8.2) to trunk. > This is planned to be done as part of KAFKA-1634. So, we wil

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Gwen Shapira
Looking at the SimpleConsumer, we need to leave in place: ConsumerMetadataResponse FetchRequest / FetchResponse OffsetFetchRequest / OffsetFetchResponse OffsetCommitRequest / OffsetCommitResponse OffsetRequest / OffsetResponse TopicMetadata TopicMetadataRequest / TopicMetadataResponse Specificall

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Gwen Shapira
Hi Jun, I was taking a slightly different approach. Let me know if it makes sense to you: 1. Get the bytes from network (kinda unavoidable...) 2. Modify RequestChannel.Request to contain header and body (instead of a single object) 3. Create the head and body from bytes as follow: val header:

Re: Review Request 32193: Patch for KAFKA-1997

2015-03-18 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32193/#review76910 --- core/src/main/scala/kafka/tools/MirrorMaker.scala

Re: [DISCUSS] KIP-5 - Broker Configuration Management

2015-03-18 Thread Andrii Biletskyi
Aditya, Yes, you are right. We agreed to think how global config can be done w/o restarting the broker, consumers/producers. I haven't yet updated the KIP accordingly though. In short, the problem is that KafkaConfig is "spread out" over different components, like SocketServer, OffsetManager etc.

Re: Review Request 31568: Patch for KAFKA-1989

2015-03-18 Thread Yasuhiro Matsuda
> On March 18, 2015, 4:14 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/server/DelayedOperation.scala, line 373 > > > > > > Try purging on each watcher list in each doWork will likely increase > > the CPU a

Re: Review Request 31568: Patch for KAFKA-1989

2015-03-18 Thread Yasuhiro Matsuda
> On March 17, 2015, 8:56 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/server/DelayedOperation.scala, line 116 > > > > > > We need to make tickMs and wheelSize configurable. > > Yasuhiro Matsuda wrote: >

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-18 Thread Aditya Auradkar
Is it possible these 3 options during the next KIP hangout? Aditya From: Steven Wu [stevenz...@gmail.com] Sent: Tuesday, March 17, 2015 10:08 AM To: dev@kafka.apache.org Subject: Re: [KIP-DISCUSSION] KIP-13 Quotas Jay, let's say an app produces to 10 di

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Jun Rao
Hi, Gwen, I was thinking that we will be doing the following in KAFKA-1927. 1. Get the bytes from network. 2. Use a new generic approach to convert bytes into request objects. 2.1 Read the fixed request header (using the util in client). 2.2 Based on the request id in the header, deserialize the

RE: [DISCUSS] KIP-5 - Broker Configuration Management

2015-03-18 Thread Aditya Auradkar
Hi Andrii, Thanks for the writeup. IMO, we should be able to support dynamically changing certain configs. For example: the Quota proposal relies on such a mechanism for changing quotas on the fly. Aditya From: Andrii Biletskyi [andrii.bilets...@stealt

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

2015-03-18 Thread Jun Rao
Andri, Thanks for the summary. 1. I just realized that in order to start working on KAFKA-1927, we will need to merge the changes to OffsetCommitRequest (from 0.8.2) to trunk. This is planned to be done as part of KAFKA-1634. So, we will need Guozhang and Joel's help to wrap this up. 2. Thinking

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Andrii Biletskyi
Yes, I was talking about TopicMetadataRequest.scala (TMR), which is MetadataRequest in java definitions. None of the mentioned above messages (StopReplica etc) are required in KIP-4. Sorry, if I misled someone. Thanks, Andrii Biletskyi On Wed, Mar 18, 2015 at 5:42 PM, Gwen Shapira wrote: > Sor

[jira] [Comment Edited] (KAFKA-1927) Replace requests in kafka.api with requests in org.apache.kafka.common.requests

2015-03-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367396#comment-14367396 ] Jun Rao edited comment on KAFKA-1927 at 3/18/15 4:18 PM: - In order

[jira] [Commented] (KAFKA-1927) Replace requests in kafka.api with requests in org.apache.kafka.common.requests

2015-03-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367396#comment-14367396 ] Jun Rao commented on KAFKA-1927: In order to work on this jira, we will need to merge KAFK

[jira] [Commented] (KAFKA-1634) Improve semantics of timestamp in OffsetCommitRequests and update documentation

2015-03-18 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367394#comment-14367394 ] Jun Rao commented on KAFKA-1634: [~guozhang], [~jjkoshy], in order to work on KAFKA-1927,

Re: Review Request 31568: Patch for KAFKA-1989

2015-03-18 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31568/#review76899 --- core/src/main/scala/kafka/server/DelayedOperation.scala

[jira] [Commented] (KAFKA-1716) hang during shutdown of ZookeeperConsumerConnector

2015-03-18 Thread Ashwin Jayaprakash (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367361#comment-14367361 ] Ashwin Jayaprakash commented on KAFKA-1716: --- Sorry for the delay, I will get bac

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Gwen Shapira
Sorry, Andrii, I'm a bit confused. The following request/response pairs are currently not implemented in the client: StopReplica, LeaderAndIsr, UpdateMetadata, ControlledShutdown. Does KIP-4 require any of these? Also, what's TMR? Gwen On Wed, Mar 18, 2015 at 4:07 AM, Andrii Biletskyi wrote:

[jira] [Commented] (KAFKA-1912) Create a simple request re-routing facility

2015-03-18 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367331#comment-14367331 ] Andrii Biletskyi commented on KAFKA-1912: - [~jkreps] thanks for adding the code.

Re: Review Request 32199: Patch for KAFKA-1858

2015-03-18 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32199/#review76896 --- This will definitely make the test more stable. I am just thinking d

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-18 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367275#comment-14367275 ] Jiangjie Qin commented on KAFKA-2029: - +1 on this in general. Current design has seve

[jira] [Commented] (KAFKA-2019) RoundRobinAssignor clusters by consumer

2015-03-18 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367189#comment-14367189 ] Jiangjie Qin commented on KAFKA-2019: - Yeah, you are right. I somehow thought the part

[jira] [Commented] (KAFKA-1759) transient unit test failure in PartitionAssignorTest

2015-03-18 Thread Rajini Sivaram (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367063#comment-14367063 ] Rajini Sivaram commented on KAFKA-1759: --- This has been fixed under [KAFKA-1823] > t

[jira] [Commented] (KAFKA-1858) Make ServerShutdownTest a bit less flaky

2015-03-18 Thread Adrian Preston (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366985#comment-14366985 ] Adrian Preston commented on KAFKA-1858: --- As per the description in the bug report -

[jira] [Commented] (KAFKA-1858) Make ServerShutdownTest a bit less flaky

2015-03-18 Thread Adrian Preston (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366975#comment-14366975 ] Adrian Preston commented on KAFKA-1858: --- Created reviewboard https://reviews.apache.

[jira] [Updated] (KAFKA-1858) Make ServerShutdownTest a bit less flaky

2015-03-18 Thread Adrian Preston (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Preston updated KAFKA-1858: -- Attachment: KAFKA-1858.patch > Make ServerShutdownTest a bit less flaky > --

Review Request 32199: Patch for KAFKA-1858

2015-03-18 Thread Adrian Preston
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32199/ --- Review request for kafka. Bugs: KAFKA-1858 https://issues.apache.org/jira/b

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-18 Thread Andrii Biletskyi
Gwen, Thanks for bringing this up! Regarding UpdateMetadata in KIP-4 - no it shouldn't be used in Admin CLI, its internal server message. We will probably use TMR there (depends how generic re-routing facility goes) but TMR is already used in NetworkClient, so I believe there are no doubts about i

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-18 Thread Dmitry Bugaychenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Description: Controlled shutdown as implemented currently can cause numerous problems:

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-18 Thread Dmitry Bugaychenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Description: Controlled shutdown as implemented currently can cause numerous problems:

[jira] [Created] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-18 Thread Dmitry Bugaychenko (JIRA)
Dmitry Bugaychenko created KAFKA-2029: - Summary: Improving controlled shutdown for rolling updates Key: KAFKA-2029 URL: https://issues.apache.org/jira/browse/KAFKA-2029 Project: Kafka Iss