[jira] [Commented] (KAFKA-1974) NPE in SelectorTest on trunk

[ https://issues.apache.org/jira/browse/KAFKA-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372557#comment-14372557 ] Jungtaek Lim commented on KAFKA-1974: - I agree with [~aauradkar], but I'm curious that

Re: [VOTE] KIP-7 Security - IP Filtering

I'd like to add that HDFS has had ACLs + RBAC + global IP white/black list for years now. We did not notice any customers confusing the features. I've seen customers use each feature for different purposes. Actually, the only system I am aware of that integrated IP access controls together with

Re: Review Request 31816: Fix for KAFKA-527

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31816/#review77328 --- Thanks for the patch. A few comments below. core/src/main/scala/ka

Re: [DISCUSSION] KIP-15 close(timeout) for producer

That's true. However, in general, timeout is not going to be enforced precisely due to other things like GCs. Thanks, Jun On Fri, Mar 20, 2015 at 4:39 PM, Guozhang Wang wrote: > With this approach close(timeout > 0) will not actually close by the > elapsed timeout right? This semantics mismatc

Re: [DISCUSSION] KIP-15 close(timeout) for producer

With this approach close(timeout > 0) will not actually close by the elapsed timeout right? This semantics mismatch is a bit concerning to me.. Guozhang On Fri, Mar 20, 2015 at 2:37 PM, Jun Rao wrote: > Yes, that probably is simpler. Basically, close() and close(timeout) will > always wait for

Re: [VOTE] KIP-7 Security - IP Filtering

Parth, I think it's important to understand the timing of both the initial JIRA and the KIP, it helps put my comments in proper context. The initial JIRA for this was created back in December, so the timeline for 1688/KIP-11 was pretty unclear. KIP-7 came out when we started doing KIPs, back in J

[jira] [Commented] (KAFKA-2037) Can't leave comments in Confluence

[ https://issues.apache.org/jira/browse/KAFKA-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372262#comment-14372262 ] Greg Temchenko commented on KAFKA-2037: --- Should the permission be granted to all new

[jira] [Resolved] (KAFKA-2037) Can't leave comments in Confluence

[ https://issues.apache.org/jira/browse/KAFKA-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-2037. Resolution: Fixed Just granted you the permission to Kafka wiki. > Can't leave comments in Confluence > ---

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

Andrii, A few points. 1. Create/Alter can typically complete quickly. So, it's possible to make the request block until it's completed. However, currently, doing this at the broker is a bit involved. To make Create block, we will need to add some callbacks in KafkaController. This is possible. Ho

[jira] [Created] (KAFKA-2037) Can't leave comments in Confluence

Greg Temchenko created KAFKA-2037: - Summary: Can't leave comments in Confluence Key: KAFKA-2037 URL: https://issues.apache.org/jira/browse/KAFKA-2037 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-7 Security - IP Filtering

I am guessing in your last reply you meant KIP-11. And yes, I think KIP-11 subsumed KIP-7 so if we can finish KIP-11 we should not need KIP=7 but I will let Jeff confirm that, Thanks Parth On 3/20/15, 2:32 PM, "Jun Rao" wrote: >Right, if this KIP is subsumed by KIP-7, perhaps we just need to w

[jira] [Commented] (KAFKA-2036) Consumer and broker have different networks

[ https://issues.apache.org/jira/browse/KAFKA-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372199#comment-14372199 ] Arsenii Krasikov commented on KAFKA-2036: - simple solution with exhaustive search

[jira] [Updated] (KAFKA-2036) Consumer and broker have different networks

[ https://issues.apache.org/jira/browse/KAFKA-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arsenii Krasikov updated KAFKA-2036: Attachment: patch > Consumer and broker have different networks > --

KIP discussion Mar 24 at 11am PST

Hi, Everyone, We plan to have a KIP discussion on Google hangout on Mar 24 at 11am PST. If you are interested in participating and have not already received a calendar invitation, please let me know. The following is the agenda. KIP-4 (admin commands): 10 mins * status of KAFKA-1927 (refactoring

[jira] [Created] (KAFKA-2036) Consumer and broker have different networks

Arsenii Krasikov created KAFKA-2036: --- Summary: Consumer and broker have different networks Key: KAFKA-2036 URL: https://issues.apache.org/jira/browse/KAFKA-2036 Project: Kafka Issue Type: B

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

Jun, I see your point. But wouldn't that lead to a "fat" client implementations? Suppose someone would like to implement client for Admin Wire protocol. Not only people will have to code quite complicated logic like "send describe request to each broker" (again state machin?) but it will also mean

Re: [DISCUSSION] KIP-15 close(timeout) for producer

Yes, that probably is simpler. Basically, close() and close(timeout) will always wait for the sender thread to complete. Close(0) will just initiate the shutdown of the sender thread, but not waiting for the thread to complete. Thanks, Jun On Fri, Mar 20, 2015 at 11:29 AM, Jiangjie Qin wrote:

Re: [VOTE] KIP-7 Security - IP Filtering

Right, if this KIP is subsumed by KIP-7, perhaps we just need to wait until KIP-7 is done? If we add the small change now, we will have to worry about migrating existing users and deprecating some configs when KIP-7 is done. Thanks, Jun On Fri, Mar 20, 2015 at 10:36 AM, Parth Brahmbhatt < pbrahm

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

For 1), 2) and 3), blocking would probably mean that the new metadata is propagated to every broker. To achieve that, the client can keep issuing the describe topic request to every broker until it sees the new metadata in the response. Thanks, Jun On Fri, Mar 20, 2015 at 12:16 PM, Andrii Bilets

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372058#comment-14372058 ] Allen Wang commented on KAFKA-1215: --- What's the status of this JIRA? I have two questio

[jira] [Commented] (KAFKA-1983) TestEndToEndLatency can be unreliable after hard kill

[ https://issues.apache.org/jira/browse/KAFKA-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371981#comment-14371981 ] Grayson Chao commented on KAFKA-1983: - I'm having some trouble reproducing this issue.

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

Hm, actually the ticket you linked, Guozhang, brings as back to the problem what should be considered a post-condition for each of the admin commands. In my understanding: 1) CreateTopic - broker created /brokers/topics/ (Not the controller picked up changes from zk and broadcasted LeaderAndIsr an

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371844#comment-14371844 ] Dmitry Bugaychenko commented on KAFKA-2029: --- You right about timeout - it was a

Re: [DISCUSSION] KIP-15 close(timeout) for producer

That’s a very good point. Currently if timeout > 0, it will wait up to timeout for the sender thread to complete. After that it will do a force close but not block. Maybe we should make it to be: If timeout > 0, wait up to timeout for the sender thread to complete. After that, make a force close b

[jira] [Commented] (KAFKA-1901) Move Kafka version to be generated in code by build (instead of in manifest)

[ https://issues.apache.org/jira/browse/KAFKA-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371784#comment-14371784 ] Jason Rosenberg commented on KAFKA-1901: This is really a pretty severe error. An

[jira] [Created] (KAFKA-2035) Add a topic config cache.

Parth Brahmbhatt created KAFKA-2035: --- Summary: Add a topic config cache. Key: KAFKA-2035 URL: https://issues.apache.org/jira/browse/KAFKA-2035 Project: Kafka Issue Type: Task Re

Re: [VOTE] KIP-7 Security - IP Filtering

I am not entirely sure what you mean by integrating KIP-7 work with KAFKA-1688. Wouldn¹t the work done as part of KIP-7 become obsolete once KAFKA-1688 is done? Multiple ways of controlling these authorization just seems extra configuration that will confuse admins/users. If timing is the only is

Re: [VOTE] KIP-7 Security - IP Filtering

I can confirm that KAFKA-1688 will cover this use case. Please go over https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+In terface and let me know if you think there is a different use case being covered by KIP-7. Thanks Parth On 3/20/15, 9:26 AM, "Jun Rao" wrote: >Yes,

Re: [VOTE] KIP-7 Security - IP Filtering

Hey Jun, The intent was for the same functionality to be utilized when 1688 is done, as mentioned in the KIP: "The broader security initiative will add more robust controls for these types of environments, and this proposal could be integrated with that work at the appropriat

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

I think while loop is fine for supporting blocking, just that we need to add back off to avoid bombarding brokers with DescribeTopic requests. Also I have linked KAFKA-1125 to your proposal, and when KAFKA-1694 is done this ticket can also be clos

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

Great. I want to elaborate this a bit more, to see we are on the same page concerning the client code. So with all topic commands being async a client (AdminClient in our case or any other other client people would like to implement) to support a blocking operation (which seems to be a natural use

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371573#comment-14371573 ] Gwen Shapira commented on KAFKA-1688: - Regarding the "abstraction for requests that in

Re: [VOTE] KIP-7 Security - IP Filtering

Yes, we can discuss the implementation separately. As for the proposal itself, have you looked at KAFKA-1688? Could this just be a special case for authorization and be included there? Thanks, Jun On Wed, Mar 18, 2015 at 6:26 PM, Jeff Holoman wrote: > One other thought. Does the timing of the

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations

Andrii, I think you are right. It seems that only ReassignPartitions needs a separate verification request. Thanks, Jun On Thu, Mar 19, 2015 at 9:22 AM, Andrii Biletskyi < andrii.bilets...@stealth.ly> wrote: > Guys, > I like this idea too. Let's stick with that. I'll update KIP accordingly. >

Re: [DISCUSSION] KIP-15 close(timeout) for producer

Got it. Just to clarify, does close(timeout) always wait for the sender thread to complete? Thanks, Jun On Fri, Mar 20, 2015 at 7:41 AM, Jiangjie Qin wrote: > I think the we agreed that we are going to log an error and block. By > doing this we can make sure the error log to be checked by user

Re: [KIP-DISCUSSION] KIP-13 Quotas

+1 on Jun's suggestion of maintaining one set/style of metrics at broker. In Netflix, we have to convert the yammer metrics to servo metrics at broker. it will be painful to know some metrics are in a different style and get to be handled differently. On Fri, Mar 20, 2015 at 8:17 AM, Jun Rao wrot

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371544#comment-14371544 ] Jun Rao commented on KAFKA-1688: [~lparth], having a TopicConfigCache to cache all topic c

Re: [DISCUSS] KIP-5 - Broker Configuration Management

Hi Jay, Could you elaborate a bit more regarding 2) above, what is the motivation behind this? On Wed, Mar 18, 2015 at 2:46 PM, Jay Kreps wrote: > The other question to figure out is the hierarchy of overrides. Currently > some configs can be overridden at the topic level using a fairly ad hoc

[jira] [Created] (KAFKA-2034) sourceCompatibility not set in Kafka build.gradle

Derek Bassett created KAFKA-2034: Summary: sourceCompatibility not set in Kafka build.gradle Key: KAFKA-2034 URL: https://issues.apache.org/jira/browse/KAFKA-2034 Project: Kafka Issue Type: B

[jira] [Commented] (KAFKA-1989) New purgatory design

[ https://issues.apache.org/jira/browse/KAFKA-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371503#comment-14371503 ] Yasuhiro Matsuda commented on KAFKA-1989: - Updated reviewboard https://reviews.apa

[jira] [Updated] (KAFKA-1989) New purgatory design

[ https://issues.apache.org/jira/browse/KAFKA-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yasuhiro Matsuda updated KAFKA-1989: Attachment: KAFKA-1989_2015-03-20_08:44:57.patch > New purgatory design > --

Re: Review Request 31568: Patch for KAFKA-1989

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31568/ --- (Updated March 20, 2015, 3:45 p.m.) Review request for kafka. Bugs: KAFKA-198

Re: [KIP-DISCUSSION] KIP-13 Quotas

Not so sure. People who use quota will definitely want to monitor the new metrics at the client id level. Then they will need to deal with those metrics differently from the rest of the metrics. It would be better if we can hide this complexity from the users. Thanks, Jun On Thu, Mar 19, 2015 at

Re: [DISCUSSION] KIP-15 close(timeout) for producer

I think the we agreed that we are going to log an error and block. By doing this we can make sure the error log to be checked by user in all cases. If we silently replace close() to close(0) in sender thread, in some cases such as send error during a normal close(), user might not notice something

[jira] [Commented] (KAFKA-2024) Cleaner can generate unindexable log segments

[ https://issues.apache.org/jira/browse/KAFKA-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371177#comment-14371177 ] Rajini Sivaram commented on KAFKA-2024: --- The attached patch stops grouping of segmen

[jira] [Commented] (KAFKA-2024) Cleaner can generate unindexable log segments

[ https://issues.apache.org/jira/browse/KAFKA-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371171#comment-14371171 ] Rajini Sivaram commented on KAFKA-2024: --- Created reviewboard https://reviews.apache.

[jira] [Updated] (KAFKA-2024) Cleaner can generate unindexable log segments

[ https://issues.apache.org/jira/browse/KAFKA-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram updated KAFKA-2024: -- Attachment: KAFKA-2024.patch > Cleaner can generate unindexable log segments > -

Review Request 32300: Fix segment grouping in LogCleaner so that relative offsets fit in 4 bytes

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32300/ --- Review request for kafka. Bugs: KAFKA-2024 https://issues.apache.org/jira/b

[GitHub] kafka pull request: ConsumerConfig now expects "bootstrap.servers"...

Github user pyr closed the pull request at: https://github.com/apache/kafka/pull/51 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled b

Review Request 32290: Patch for KAFKA-2033

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32290/ --- Review request for kafka. Bugs: KAFKA-2033 https://issues.apache.org/jira/b

[jira] [Commented] (KAFKA-2033) Small typo in documentation

[ https://issues.apache.org/jira/browse/KAFKA-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14370917#comment-14370917 ] Pierre-Yves Ritschard commented on KAFKA-2033: -- Created reviewboard https://r

[jira] [Updated] (KAFKA-2033) Small typo in documentation

[ https://issues.apache.org/jira/browse/KAFKA-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Yves Ritschard updated KAFKA-2033: - Assignee: Pierre-Yves Ritschard (was: Neha Narkhede) Status: Patch Available

[jira] [Created] (KAFKA-2033) Small typo in documentation

Pierre-Yves Ritschard created KAFKA-2033: Summary: Small typo in documentation Key: KAFKA-2033 URL: https://issues.apache.org/jira/browse/KAFKA-2033 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-2033) Small typo in documentation

[ https://issues.apache.org/jira/browse/KAFKA-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Yves Ritschard updated KAFKA-2033: - Attachment: KAFKA-2033.patch > Small typo in documentation > -

[GitHub] kafka pull request: ConsumerConfig now expects "bootstrap.servers"...

GitHub user pyr opened a pull request: https://github.com/apache/kafka/pull/51 ConsumerConfig now expects "bootstrap.servers" in config. Small typo to have the documentation in line with `KafkaProducer.java`. Kudos on the awesome documentation work! You can merge this pull reque