[jira] [Commented] (KAFKA-1610) Local modifications to collections generated from mapValues will be lost

2014-09-30 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154426#comment-14154426 ] Mayuresh Gharat commented on KAFKA-1610: Updated reviewboard https://reviews.apach

[jira] [Updated] (KAFKA-1610) Local modifications to collections generated from mapValues will be lost

2014-09-30 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-1610: --- Attachment: KAFKA-1610_2014-09-30_23:21:46.patch > Local modifications to collections generate

Re: Review Request 25136: Patch for KAFKA-1610

2014-09-30 Thread Mayuresh Gharat
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25136/ --- (Updated Oct. 1, 2014, 6:21 a.m.) Review request for kafka. Bugs: KAFKA-1610

[jira] [Commented] (KAFKA-1499) Broker-side compression configuration

2014-09-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154336#comment-14154336 ] Jay Kreps commented on KAFKA-1499: -- Yeah I totally agree. I agree that some heuristic th

Re: Two open issues on Kafka security

2014-09-30 Thread Joe Stein
inline On Tue, Sep 30, 2014 at 11:58 PM, Jay Kreps wrote: > Hey Joe, > > For (1) what are you thinking for the PermissionManager api? > > The way I see it, the first question we have to answer is whether it > is possible to make authentication and authorization independent. What > I mean by that

[jira] [Commented] (KAFKA-1659) Ability to cleanly abort the KafkaProducer

2014-09-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154322#comment-14154322 ] Jay Kreps commented on KAFKA-1659: -- Yeah and that is in some sense a bigger problem becau

Re: Two open issues on Kafka security

2014-09-30 Thread Jay Kreps
Hey Gwen, That makes sense. I think this is one area where having pluggable authorization makes the story a bit more complex since all the management of default permissions or even how to ensure a user does or doesn't have a permission is going to be specific to the authorization model a particul

[jira] [Commented] (KAFKA-1659) Ability to cleanly abort the KafkaProducer

2014-09-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154315#comment-14154315 ] Guozhang Wang commented on KAFKA-1659: -- Right. Once the message is in the accumulator

Re: Two open issues on Kafka security

2014-09-30 Thread Jay Kreps
Hey Joe, For (1) what are you thinking for the PermissionManager api? The way I see it, the first question we have to answer is whether it is possible to make authentication and authorization independent. What I mean by that is whether I can write an authorization library that will work the same

[jira] [Commented] (KAFKA-1659) Ability to cleanly abort the KafkaProducer

2014-09-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154292#comment-14154292 ] Jay Kreps commented on KAFKA-1659: -- Yeah that makes sense. I feel like maybe the real is

Re: Two open issues on Kafka security

2014-09-30 Thread Joe Stein
<< we need to make it easy for secured clusters to pass audits (SOX, PCI and friends) I think this is the MVP for the security features for 0.9 as a guideline for how we should be proceeding. On Tue, Sep 30, 2014 at 7:25 PM, Gwen Shapira wrote: > Re #2: > > I don't object to the "late authentic

[jira] [Commented] (KAFKA-1499) Broker-side compression configuration

2014-09-30 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154209#comment-14154209 ] Joel Koshy commented on KAFKA-1499: --- If we provide a broker-compression-enabled config:

[jira] [Updated] (KAFKA-1555) provide strong consistency with reasonable availability

2014-09-30 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1555: Attachment: KAFKA-1555.5.patch Uploaded latest version. Addressing the duplicates and the acks>1 is

Re: Review Request 25886: KAFKA-1555: provide strong consistency with reasonable availability

2014-09-30 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25886/ --- (Updated Oct. 1, 2014, 1:19 a.m.) Review request for kafka. Changes ---

[jira] [Closed] (KAFKA-1658) Overriding #of partitions per topic, does not take effect

2014-09-30 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein closed KAFKA-1658. You can also email on the mailing list for these type of questions https://kafka.apache.org/contact.html for th

[jira] [Resolved] (KAFKA-1658) Overriding #of partitions per topic, does not take effect

2014-09-30 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1658. -- Resolution: Won't Fix Hi, please take a look at https://kafka.apache.org/documentation.html#basic_ops_a

[jira] [Commented] (KAFKA-1658) Overriding #of partitions per topic, does not take effect

2014-09-30 Thread aarti gupta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154059#comment-14154059 ] aarti gupta commented on KAFKA-1658: Is there a way (programatic or otherwise) to set

[jira] [Commented] (KAFKA-1659) Ability to cleanly abort the KafkaProducer

2014-09-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154049#comment-14154049 ] Guozhang Wang commented on KAFKA-1659: -- Hey [~jkreps], from Andrew's question email o

Re: Two open issues on Kafka security

2014-09-30 Thread Gwen Shapira
Re #2: I don't object to the "late authentication" approach, but we need to make it easy for secured clusters to pass audits (SOX, PCI and friends). So, we need to be able to configure a cluster as "secured" and with this config switch "nobody" user to zero privileges. I liked the multi-port appro

[jira] [Commented] (KAFKA-1659) Ability to cleanly abort the KafkaProducer

2014-09-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153999#comment-14153999 ] Jay Kreps commented on KAFKA-1659: -- This makes sense, I think what you are asking for is

Re: Two open issues on Kafka security

2014-09-30 Thread Joe Stein
1) We need to support the most flexibility we can and make this transparent to kafka (to use Gwen's term). Any specific implementation is going to make it not work with some solution stopping people from using Kafka. That is a reality because everyone just does it slightly differently enough. If

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2014-09-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153954#comment-14153954 ] Jay Kreps commented on KAFKA-1660: -- Aren't this and KAFKA-1659 the same? > Ability to ca

Re: Two open issues on Kafka security

2014-09-30 Thread Gwen Shapira
Re #1: Since the auth_to_local is a kerberos config, its up to the admin to decide how he likes the user names and set it up properly (or leave empty) and make sure the ACLs match. Simplified names may be needed if the authorization system integrates with LDAP to get groups or something fancy like

Two open issues on Kafka security

2014-09-30 Thread Jay Kreps
Current proposal is here: https://cwiki.apache.org/confluence/display/KAFKA/Security Here are the two open questions I am aware of: 1. We want to separate authentication and authorization. This means permissions will be assigned to some user-like subject/entity/person string that is independent

[jira] [Resolved] (KAFKA-520) ConsumerIterator implemented by KafkaStream doesn't follow Java practices

2014-09-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-520. - Resolution: Duplicate This is being fixed in the new consumer implementation. > ConsumerIterator implement

[jira] [Commented] (KAFKA-520) ConsumerIterator implemented by KafkaStream doesn't follow Java practices

2014-09-30 Thread Jeff Wartes (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153791#comment-14153791 ] Jeff Wartes commented on KAFKA-520: --- This is still true two years later. Something that

[jira] [Commented] (KAFKA-1558) AdminUtils.deleteTopic does not work

2014-09-30 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153663#comment-14153663 ] Sriharsha Chintalapani commented on KAFKA-1558: --- [~nehanarkhede] I was tryin

Re: [DISCUSS] 0.8.1.2 Release

2014-09-30 Thread Jonathan Weeks
I was one asking for 0.8.1.2 a few weeks back, when 0.8.2 was at least 6-8 weeks out. If we truly believe that 0.8.2 will go “golden” and stable in 2-3 weeks, I, for one, don’t need a 0.8.1.2, but it depends on the confidence in shipping 0.8.2 soonish. YMMV, -Jonathan On Sep 30, 2014, at 12

RE: [DISCUSS] 0.8.1.2 Release

2014-09-30 Thread Seshadri, Balaji
In DISH we are having issues in 0.8-beta version used in PROD, it's crashing every 2 days and becoming a blocker for us. It would be great if we get 0.8.2 or 0.8.1.2 whichever is faster as we can't wait for 3 weeks as our new Order Management system is going to sit on top of Kafka. -Origin

Re: [DISCUSS] 0.8.1.2 Release

2014-09-30 Thread Neha Narkhede
Can we discuss the need for 0.8.1.2? I'm wondering if it's related to the timeline of 0.8.2 in any way? For instance, if we can get 0.8.2 out in the next 2-3 weeks, do we still need to get 0.8.1.2 out or can people just upgrade to 0.8.2? On Tue, Sep 30, 2014 at 9:53 AM, Joe Stein wrote: > Hi, I

[jira] [Commented] (KAFKA-1558) AdminUtils.deleteTopic does not work

2014-09-30 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153631#comment-14153631 ] Neha Narkhede commented on KAFKA-1558: -- [~sriharsha] Thanks for running more tests. I

Re: Review Request 25136: Patch for KAFKA-1610

2014-09-30 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25136/#review54990 --- Thanks for the updated patch. This looks fine - however, wouldn't it

[jira] [Commented] (KAFKA-1499) Broker-side compression configuration

2014-09-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153407#comment-14153407 ] Jay Kreps commented on KAFKA-1499: -- Here is the problem with compaction. Currently the lo

[jira] [Commented] (KAFKA-1558) AdminUtils.deleteTopic does not work

2014-09-30 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153404#comment-14153404 ] Sriharsha Chintalapani commented on KAFKA-1558: --- [~guozhang] yes testing aga

Re: Review Request 25886: KAFKA-1555: provide strong consistency with reasonable availability

2014-09-30 Thread Gwen Shapira
> On Sept. 30, 2014, 4:55 p.m., Gwen Shapira wrote: > > I didn't get a chance to go over the entire review in detail. I did notice > > that we now have separate objects for MFromConfig and MToConfig. What's the > > reasoning behind that? Ignore. commenting on wrong RB. - Gwen -

Re: Review Request 25886: KAFKA-1555: provide strong consistency with reasonable availability

2014-09-30 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25886/#review54987 --- I didn't get a chance to go over the entire review in detail. I did

[DISCUSS] 0.8.1.2 Release

2014-09-30 Thread Joe Stein
Hi, I wanted to kick off a specific discussion on a 0.8.1.2 release. Here are the JIRAs I would like to propose to back port a patch (if not already done so) and apply them to the 0.8.1 branch for a 0.8.1.2 release https://issues.apache.org/jira/browse/KAFKA-1502 (source jar is empty) https://iss

[jira] [Commented] (KAFKA-1558) AdminUtils.deleteTopic does not work

2014-09-30 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153393#comment-14153393 ] Guozhang Wang commented on KAFKA-1558: -- Just wondering if it is caused by KAFKA-1578.

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

2014-09-30 Thread James Lent (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153370#comment-14153370 ] James Lent commented on KAFKA-1387: --- I have messed things up. I tried to use the Submit

[jira] [Updated] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

2014-09-30 Thread James Lent (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Lent updated KAFKA-1387: -- Attachment: kafka-1387.patch > Kafka getting stuck creating ephemeral node it has already created when t

[jira] [Commented] (KAFKA-1558) AdminUtils.deleteTopic does not work

2014-09-30 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153343#comment-14153343 ] Sriharsha Chintalapani commented on KAFKA-1558: --- [~nehanarkhede] [~junrao] T

[jira] [Updated] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

2014-09-30 Thread James Lent (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Lent updated KAFKA-1387: -- Labels: newbie patch (was: ) Affects Version/s: 0.8.1.1 Status: Patch Avai

[jira] [Resolved] (KAFKA-1547) maven sources jar still empty

2014-09-30 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein resolved KAFKA-1547. -- Resolution: Duplicate taken care in https://issues.apache.org/jira/browse/KAFKA-1502 > maven sources ja

Re: [DISCUSS] 0.8.2 release branch, "unofficial" release candidates(s), 0.8.1.2 release

2014-09-30 Thread Joe Stein
Thanks Jun! I created 0.8.3 in JIRA too just now. - Joestein On Tue, Sep 30, 2014 at 11:15 AM, Jun Rao wrote: > I just created the 0.8.2 branch and bumped up the version in trunk to > 0.8.3. > > Thanks, > > Jun > > On Mon, Sep 29, 2014 at 11:43 AM, Sriram Subramanian < > srsubraman...@linkedin.

Re: [DISCUSS] 0.8.2 release branch, "unofficial" release candidates(s), 0.8.1.2 release

2014-09-30 Thread Jun Rao
I just created the 0.8.2 branch and bumped up the version in trunk to 0.8.3. Thanks, Jun On Mon, Sep 29, 2014 at 11:43 AM, Sriram Subramanian < srsubraman...@linkedin.com.invalid> wrote: > +1 on 0.8.3-SNAPSHOT. > > On 9/29/14 11:40 AM, "Neha Narkhede" wrote: > > >2) change trunk to be 0.9-SNAP

[jira] [Created] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2014-09-30 Thread Andrew Stein (JIRA)
Andrew Stein created KAFKA-1660: --- Summary: Ability to call close() with a timeout on the Java Kafka Producer. Key: KAFKA-1660 URL: https://issues.apache.org/jira/browse/KAFKA-1660 Project: Kafka

[jira] [Created] (KAFKA-1659) Ability to cleanly abort the KafkaProducer

2014-09-30 Thread Andrew Stein (JIRA)
Andrew Stein created KAFKA-1659: --- Summary: Ability to cleanly abort the KafkaProducer Key: KAFKA-1659 URL: https://issues.apache.org/jira/browse/KAFKA-1659 Project: Kafka Issue Type: Improvemen