[jira] [Commented] (KAFKA-4276) REST configuration not visible in connector properties config files

2017-02-03 Thread Akhilesh Naidu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851200#comment-15851200 ] Akhilesh Naidu commented on KAFKA-4276: --- Hi Gwen, I would like to work on this issue

Re: Trying to understand design decision about producer ack and min.insync.replicas

2017-02-03 Thread Ismael Juma
I'd be in favour too. Ismael On 3 Feb 2017 7:33 am, "Ewen Cheslack-Postava" wrote: > On Thu, Feb 2, 2017 at 11:21 PM, James Cheng wrote: > > > Ewen, > > > > Ah right, that's a good point. > > > > My initial reaction to your examples was that "well, those should be in > > separate topics", but

[jira] [Comment Edited] (KAFKA-4276) REST configuration not visible in connector properties config files

2017-02-03 Thread Akhilesh Naidu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851200#comment-15851200 ] Akhilesh Naidu edited comment on KAFKA-4276 at 2/3/17 8:04 AM: -

[jira] [Assigned] (KAFKA-4276) REST configuration not visible in connector properties config files

2017-02-03 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava reassigned KAFKA-4276: Assignee: Akhilesh Naidu > REST configuration not visible in connector prop

[jira] [Commented] (KAFKA-4276) REST configuration not visible in connector properties config files

2017-02-03 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851233#comment-15851233 ] Ewen Cheslack-Postava commented on KAFKA-4276: -- [~akhilesh_naidu] -- I set yo

[jira] [Created] (KAFKA-4728) KafkaConsumer#commitSync should clone its input

2017-02-03 Thread JIRA
Jan Lukavský created KAFKA-4728: --- Summary: KafkaConsumer#commitSync should clone its input Key: KAFKA-4728 URL: https://issues.apache.org/jira/browse/KAFKA-4728 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-4728) KafkaConsumer#commitSync should clone its input

2017-02-03 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lukavský updated KAFKA-4728: Attachment: KAFKA-4728.patch Attaching patch for this. > KafkaConsumer#commitSync should clone its

[jira] [Updated] (KAFKA-4728) KafkaConsumer#commitSync should clone its input

2017-02-03 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lukavský updated KAFKA-4728: Status: Patch Available (was: Open) Patch available for this issue. > KafkaConsumer#commitSync sho

[jira] [Commented] (KAFKA-4728) KafkaConsumer#commitSync should clone its input

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851268#comment-15851268 ] Ismael Juma commented on KAFKA-4728: Can you do a pull request as described here http

[GitHub] kafka pull request #2491: [KAFKA-4728] KafkaConsumer#commitSync should clone...

2017-02-03 Thread je-ik
GitHub user je-ik opened a pull request: https://github.com/apache/kafka/pull/2491 [KAFKA-4728] KafkaConsumer#commitSync should clone its input You can merge this pull request into a Git repository by running: $ git pull https://github.com/je-ik/kafka KAFKA-4728 Alternativel

[jira] [Commented] (KAFKA-4728) KafkaConsumer#commitSync should clone its input

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851276#comment-15851276 ] ASF GitHub Bot commented on KAFKA-4728: --- GitHub user je-ik opened a pull request:

[jira] [Created] (KAFKA-4729) Stores for kstream-kstream join cannot be in-memory

2017-02-03 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-4729: --- Summary: Stores for kstream-kstream join cannot be in-memory Key: KAFKA-4729 URL: https://issues.apache.org/jira/browse/KAFKA-4729 Project: Kafka Issue Type: I

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

2017-02-03 Thread Damian Guy
Hi Matthias, It possibly doesn't make sense to disable it, but then i'm sure someone will come up with a reason they don't want it! I'm happy to change it such that the checkpoint interval must be > 0. Cheers, Damian On Fri, 3 Feb 2017 at 01:29 Matthias J. Sax wrote: > Thanks Damian. > > One m

[jira] [Created] (KAFKA-4730) Streams does not have an in-memory windowed store

2017-02-03 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-4730: --- Summary: Streams does not have an in-memory windowed store Key: KAFKA-4730 URL: https://issues.apache.org/jira/browse/KAFKA-4730 Project: Kafka Issue Type: New

[DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

2017-02-03 Thread Ismael Juma
Hi all, I have posted a KIP for dropping support for Java 7 in Kafka 0.11: https://cwiki.apache.org/confluence/display/KAFKA/KIP-118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 Most people were supportive when we last discussed the topic[1], but there were a few concerns. I believe the following sh

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-03 Thread Rajini Sivaram
I have a few questions on security (sorry, only just catching up on the updates). 1. Will the transaction coordinator check topic ACLs based on the requesting client's credentials? Access to transaction logs, topics being added for transaction etc? 2. If I create a transactional produce request (b

[jira] [Updated] (KAFKA-4422) Drop support for Scala 2.10

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4422: --- Description: Now that Scala 2.12 has been released, we should drop support for Scala 2.10 in the nex

[jira] [Updated] (KAFKA-4422) Drop support for Scala 2.10

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4422: --- Labels: kip (was: ) > Drop support for Scala 2.10 > --- > > K

KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-02-03 Thread Ismael Juma
Hi all, I have posted a KIP for dropping support for Scala 2.10 in Kafka 0.11: https://cwiki.apache.org/confluence/display/KAFKA/KIP-119%3A+Drop+Support+for+Scala+2.10+in+Kafka+0.11 Please take a look. Your feedback is appreciated. Thanks, Ismael

Re: [DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

2017-02-03 Thread Damian Guy
Thanks Ismael. Makes sense to me. On Fri, 3 Feb 2017 at 10:39 Ismael Juma wrote: > Hi all, > > I have posted a KIP for dropping support for Java 7 in Kafka 0.11: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 > > Most people were supporti

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-03 Thread Ismael Juma
Comments inline. On Thu, Feb 2, 2017 at 6:28 PM, Jason Gustafson wrote: > Took me a while to remember why we didn't do this. The timestamp that is > included at the message set level is the max timestamp of all messages in > the message set as is the case in the current message format (I will up

Re: [DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

2017-02-03 Thread Grant Henke
Looks good to me. Thanks for handling the KIP. On Fri, Feb 3, 2017 at 8:49 AM, Damian Guy wrote: > Thanks Ismael. Makes sense to me. > > On Fri, 3 Feb 2017 at 10:39 Ismael Juma wrote: > > > Hi all, > > > > I have posted a KIP for dropping support for Java 7 in Kafka 0.11: > > > > > > https://cw

Re: KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-02-03 Thread Grant Henke
Thanks for proposing this Ismael. This makes sense to me. In this KIP and the java KIP you state: A reasonable policy is to support the 2 most recently released versions so > that we can strike a good balance between supporting older versions, > maintainability and taking advantage of language an

Re: Trying to understand design decision about producer ack and min.insync.replicas

2017-02-03 Thread Grant Henke
I would be in favor of defaulting acks=all. I have found that most people want to start with the stronger/safer guarantees and then adjust them for performance on a case by case basis. This gives them a chance to understand and accept the tradeoffs. A few other defaults I would be in favor of cha

Re: KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-02-03 Thread Ismael Juma
Hi Grant, That's an interesting point. It would be good to hear what others' think of making that the official policy instead of starting a discussion/vote each time. If there is consensus, I am happy to revise the KIPs. Otherwise, we keep them as they are and discuss/vote on this instance only.

[jira] [Updated] (KAFKA-4662) Improve test coverage TopologyBuilder

2017-02-03 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4662: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-4662) Improve test coverage TopologyBuilder

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851710#comment-15851710 ] ASF GitHub Bot commented on KAFKA-4662: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2464: KAFKA-4662: adding test coverage for addSource met...

2017-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2464 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka-site pull request #43: [DO NOT MERGE] manual changes after copy-paste ...

2017-02-03 Thread guozhangwang
GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka-site/pull/43 [DO NOT MERGE] manual changes after copy-paste to 0.10.2 @derrickdoo You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka-

[jira] [Updated] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4725: --- Labels: reliability (was: ) > Kafka broker fails due to OOM when producer exceeds throttling quota fo

[jira] [Updated] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4725: --- Priority: Critical (was: Major) > Kafka broker fails due to OOM when producer exceeds throttling quot

[jira] [Updated] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4725: --- Fix Version/s: 0.10.2.1 > Kafka broker fails due to OOM when producer exceeds throttling quota for >

[jira] [Updated] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4725: --- Fix Version/s: 0.10.3.0 > Kafka broker fails due to OOM when producer exceeds throttling quota for >

[jira] [Commented] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851730#comment-15851730 ] Ismael Juma commented on KAFKA-4725: Nice catch, a contribution via a PR would be welc

[jira] [Updated] (KAFKA-4649) Improve test coverage GlobalStateManagerImpl

2017-02-03 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4649: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-4649) Improve test coverage GlobalStateManagerImpl

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851745#comment-15851745 ] ASF GitHub Bot commented on KAFKA-4649: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2452: KAFKA-4649: Improve test coverage GlobalStateManag...

2017-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2452 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2447: KAFKA-4646: Improve test coverage AbstractProcesso...

2017-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2447 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-4646) Improve test coverage AbstractProcessorContext

2017-02-03 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4646: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-4646) Improve test coverage AbstractProcessorContext

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851758#comment-15851758 ] ASF GitHub Bot commented on KAFKA-4646: --- Github user asfgit closed the pull request

Re: [DISCUSS] KIP-111 : Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-03 Thread Mayuresh Gharat
Hi All, If there are no more concerns, I will like to start vote for this KIP. Thanks, Mayuresh On Wed, Feb 1, 2017 at 8:38 PM, Mayuresh Gharat wrote: > Hi Dong, > > What I meant was "Right now Kafka just extracts the name out of the > Principal that is generated by the PrincipalBuilder. Inst

[jira] [Created] (KAFKA-4731) Add event-based session windows

2017-02-03 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-4731: -- Summary: Add event-based session windows Key: KAFKA-4731 URL: https://issues.apache.org/jira/browse/KAFKA-4731 Project: Kafka Issue Type: New Feature

[jira] [Commented] (KAFKA-4614) Long GC pause harming broker performance which is caused by mmap objects created for OffsetIndex

2017-02-03 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851847#comment-15851847 ] Uwe Schindler commented on KAFKA-4614: -- Just to inform you. Cleaner is no longer avai

Re: [VOTE] 0.10.2.0 RC0

2017-02-03 Thread Tom Crayford
-1 (non-binding). Whilst testing 0.10.1.1, we found that there's a bug with quotas that means clients can cause broker OOMs very easily. If a producer has exceeded the produce quota, the produce request memory is retained inside the delay queue. (https://issues.apache.org/jira/browse/KAFKA-4725).

[GitHub] kafka pull request #2492: HOTFIX: update incorrect docs and broken links

2017-02-03 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2492 HOTFIX: update incorrect docs and broken links You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfixDocs Alternatively you can revie

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-02-03 Thread Colin McCabe
On Thu, Feb 2, 2017, at 17:54, Dong Lin wrote: > Hey Colin, > > Thanks for the KIP. I have a few comments below: > > - I share similar view with Ismael that a Future-based API is better. > PurgeDataFrom() is an example API that user may want to do it > asynchronously even though there is only one

Re: [DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

2017-02-03 Thread Roger Hoover
This is great. Thanks, Ismael. On Fri, Feb 3, 2017 at 7:35 AM, Grant Henke wrote: > Looks good to me. Thanks for handling the KIP. > > On Fri, Feb 3, 2017 at 8:49 AM, Damian Guy wrote: > > > Thanks Ismael. Makes sense to me. > > > > On Fri, 3 Feb 2017 at 10:39 Ismael Juma wrote: > > > > > Hi

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-02-03 Thread Colin McCabe
On Thu, Feb 2, 2017, at 21:45, Becket Qin wrote: > Hi Colin, > > Thanks for the KIP. An admin client is probably a must after we block > direct access to ZK. Some comments and thoughts below: > > 1. Do we have a clear scope for the admin client? It might be worth > thinking about the entire user

[jira] [Commented] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Tim Carey-Smith (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851880#comment-15851880 ] Tim Carey-Smith commented on KAFKA-4725: Hi there, Jeff and I have prototyped a

[jira] [Commented] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851885#comment-15851885 ] Ismael Juma commented on KAFKA-4725: Great. The target should be trunk, we cherry-pick

[jira] [Commented] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread Jeff Chao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851898#comment-15851898 ] Jeff Chao commented on KAFKA-4725: -- Ok, we'll base it off trunk and open up a PR. Thanks.

[jira] [Commented] (KAFKA-4614) Long GC pause harming broker performance which is caused by mmap objects created for OffsetIndex

2017-02-03 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851904#comment-15851904 ] Jun Rao commented on KAFKA-4614: [~thetaphi], thanks for the info. Much appreciated! > Lo

Re: [DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

2017-02-03 Thread Eno Thereska
Makes sense. Eno > On 3 Feb 2017, at 10:38, Ismael Juma wrote: > > Hi all, > > I have posted a KIP for dropping support for Java 7 in Kafka 0.11: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 > > Most people were supportive when we las

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-02-03 Thread Colin McCabe
On Thu, Feb 2, 2017, at 15:02, Ismael Juma wrote: > Hi Colin, > > Thanks for the KIP, great to make progress on this. I have some initial > comments, will post more later: > > 1. We have KafkaProducer that implements the Producer interface and > KafkaConsumer that implements the Consumer interfac

[jira] [Commented] (KAFKA-4614) Long GC pause harming broker performance which is caused by mmap objects created for OffsetIndex

2017-02-03 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851905#comment-15851905 ] Ismael Juma commented on KAFKA-4614: Thanks [~thetaphi]. We are aware of the issue and

Build failed in Jenkins: kafka-trunk-jdk8 #1247

2017-02-03 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-4662: adding test coverage for addSource methods with [wangguoz] KAFKA-4649: Improve test coverage GlobalStateManagerImpl -- [...truncated 15916 lines...] org.apac

[jira] [Commented] (KAFKA-2376) Add Copycat metrics

2017-02-03 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851926#comment-15851926 ] Chris Riccomini commented on KAFKA-2376: One thing we'd like to see here is the ab

[GitHub] kafka pull request #2333: KAFKA-3452 Follow-up: Optimize ByteStore Scenarios

2017-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2333 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3452) Support session windows

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851970#comment-15851970 ] ASF GitHub Bot commented on KAFKA-3452: --- Github user asfgit closed the pull request

Re: [VOTE] KIP-54: Sticky Partition Assignment Strategy

2017-02-03 Thread Vahid S Hashemian
The voting on this KIP concludes with 3 binding and 3 non-binding +1 votes. Thanks to everyone who provided feedback / voted and to Jeff Widman for reviving the voting thread. Binding votes by: * Jason Gustafson * Ewen Cheslack-Postava * Guozhang Wang Non-binding votes by: * Rajini Sivaram * Bi

[VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-03 Thread Mayuresh Gharat
Hi All, It seems that there is no further concern with the KIP-111. At this point we would like to start the voting process. The KIP can be found at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=67638388 Thanks, Mayuresh

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-03 Thread Jason Gustafson
Hey Rajini, Thanks for the questions. Responses below: > 1. Will the transaction coordinator check topic ACLs based on the > requesting client's credentials? Access to transaction logs, topics being > added for transaction etc? Good question. I think it makes sense to check topic Write permiss

[jira] [Created] (KAFKA-4732) Unstable test: KStreamKTableJoinIntegrationTest.shouldCountClicksPerRegion[1]

2017-02-03 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-4732: -- Summary: Unstable test: KStreamKTableJoinIntegrationTest.shouldCountClicksPerRegion[1] Key: KAFKA-4732 URL: https://issues.apache.org/jira/browse/KAFKA-4732 Proje

[GitHub] kafka pull request #2493: KAFKA-4720: add a KStream#peek(ForeachAction...

2017-02-03 Thread stevenschlansker
GitHub user stevenschlansker opened a pull request: https://github.com/apache/kafka/pull/2493 KAFKA-4720: add a KStream#peek(ForeachAction) https://issues.apache.org/jira/browse/KAFKA-4720 Peek is a handy method to have to insert diagnostics that do not affect the stream it

[jira] [Commented] (KAFKA-4720) Add KStream.peek(ForeachAction)

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852102#comment-15852102 ] ASF GitHub Bot commented on KAFKA-4720: --- GitHub user stevenschlansker opened a pull

[GitHub] kafka pull request #2494: HOTFIX: Verifiable producer request timeout needs ...

2017-02-03 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2494 HOTFIX: Verifiable producer request timeout needs conversion to milliseconds You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka

[jira] [Commented] (KAFKA-2376) Add Copycat metrics

2017-02-03 Thread Randall Hauch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852164#comment-15852164 ] Randall Hauch commented on KAFKA-2376: -- I agree with [~criccomini]. It's be great to

[GitHub] kafka-site pull request #44: 0102 fixes

2017-02-03 Thread derrickdoo
GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/44 0102 fixes - Updated include statements for all main pages to look for latest docs in 0102 directory - Removed double navigation include on Introduction page (will fix in main Kafka repo)

[GitHub] kafka-site issue #44: 0102 fixes

2017-02-03 Thread derrickdoo
Github user derrickdoo commented on the issue: https://github.com/apache/kafka-site/pull/44 @guozhangwang can you take a look at this and double check to make sure this fixes all the issues you were seeing. I'll make another PR into the main Kafka repo to make sure we don't run into i

[GitHub] kafka pull request #2494: HOTFIX: Verifiable producer request timeout needs ...

2017-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2494 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Jenkins build is back to normal : kafka-trunk-jdk8 #1248

2017-02-03 Thread Apache Jenkins Server
See

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-03 Thread Rajini Sivaram
Hi Jason, Thank you for the responses. Agree that authorizing transactional.id in the producer requests will be good enough for version 1. And making it tighter in future based on delegation tokens sounds good too. Regards, Rajini On Fri, Feb 3, 2017 at 8:04 PM, Jason Gustafson wrote: > Hey

Re: [VOTE] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-03 Thread Rajini Sivaram
+1 (non-binding) (with additional authorization from Jason's note in the discussion thread) On Fri, Feb 3, 2017 at 1:10 AM, Apurva Mehta wrote: > The wiki has been updated with a section on authorization, as well a > summary of the message format changes. > > On Thu, Feb 2, 2017 at 9:38 AM, Ja

[GitHub] kafka pull request #2495: Adjustments to docs to make them work with latest ...

2017-02-03 Thread derrickdoo
GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka/pull/2495 Adjustments to docs to make them work with latest template - Change introduction.html to be a partial like all the other doc content sections - Fix typo with include statement for the footer

[jira] [Commented] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852189#comment-15852189 ] ASF GitHub Bot commented on KAFKA-4725: --- GitHub user halorgium opened a pull request

[GitHub] kafka pull request #2496: KAFKA-4725: Stop leaking messages in produce reque...

2017-02-03 Thread halorgium
GitHub user halorgium opened a pull request: https://github.com/apache/kafka/pull/2496 KAFKA-4725: Stop leaking messages in produce request body when requests are delayed This change is in response to [KAFKA-4725](https://issues.apache.org/jira/browse/KAFKA-4725). When a

[GitHub] kafka-site issue #44: 0102 fixes

2017-02-03 Thread derrickdoo
Github user derrickdoo commented on the issue: https://github.com/apache/kafka-site/pull/44 Here's the corresponded PR to get these changes permanently into the main Kafka repo for future releases https://github.com/apache/kafka/pull/2495 --- If your project is set up for it, you can

Build failed in Jenkins: kafka-trunk-jdk8 #1249

2017-02-03 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-3452 Follow-up: Optimize ByteStore Scenarios [jason] HOTFIX: Verifiable producer request timeout needs conversion to -- [...truncated 968 lines...] kafka.server.S

[jira] [Updated] (KAFKA-3856) Cleanup Kafka Streams builder API

2017-02-03 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-3856: --- Summary: Cleanup Kafka Streams builder API (was: Move inner classes accessible only functions

[jira] [Updated] (KAFKA-3856) Move inner classes accessible only functions in TopologyBuilder out of public APIs

2017-02-03 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-3856: --- Labels: api kip (was: api needs-kip) > Move inner classes accessible only functions in Topolo

[jira] [Updated] (KAFKA-3856) Move inner classes accessible only functions in TopologyBuilder out of public APIs

2017-02-03 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-3856: --- Description: https://cwiki.apache.org/confluence/display/KAFKA/KIP-120%3A+Cleanup+Kafka+Stream

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

2017-02-03 Thread Jason Gustafson
Hi Tom, I said this in the voting thread, but can the authors include a section > about new ACLs if there are going to be ACLs for TransactionalId. It's > mentioned in the google doc, but I think new ACLs should be in a KIP > directly. We've updated the wiki. Can you take a look and let us know

[jira] [Created] (KAFKA-4733) Improve Streams Reset Tool console output

2017-02-03 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created KAFKA-4733: -- Summary: Improve Streams Reset Tool console output Key: KAFKA-4733 URL: https://issues.apache.org/jira/browse/KAFKA-4733 Project: Kafka Issue Type: Impro

[jira] [Updated] (KAFKA-4720) Add KStream.peek(ForeachAction)

2017-02-03 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4720: --- Labels: needs-kip (was: ) > Add KStream.peek(ForeachAction) > ---

[jira] [Commented] (KAFKA-4734) timeindex on old segments not trimmed to actual size

2017-02-03 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852317#comment-15852317 ] Jun Rao commented on KAFKA-4734: [~becket_qin], do you think you could take a look at this

[jira] [Created] (KAFKA-4734) timeindex on old segments not trimmed to actual size

2017-02-03 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-4734: -- Summary: timeindex on old segments not trimmed to actual size Key: KAFKA-4734 URL: https://issues.apache.org/jira/browse/KAFKA-4734 Project: Kafka Issue Type: Improveme

[DISCUSS] KIP-120: Cleanup Kafka Streams builder API

2017-02-03 Thread Matthias J. Sax
Hi All, I did prepare a KIP to do some cleanup some of Kafka's Streaming API. Please have a look here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-120%3A+Cleanup+Kafka+Streams+builder+API Looking forward to your feedback! -Matthias signature.asc Description: OpenPGP digital signat

[GitHub] kafka-site pull request #43: [DO NOT MERGE] manual changes after copy-paste ...

2017-02-03 Thread guozhangwang
Github user guozhangwang closed the pull request at: https://github.com/apache/kafka-site/pull/43 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[jira] [Assigned] (KAFKA-4733) Improve Streams Reset Tool console output

2017-02-03 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira reassigned KAFKA-4733: --- Assignee: Gwen Shapira > Improve Streams Reset Tool console output >

Re: [VOTE] 0.10.2.0 RC0

2017-02-03 Thread Vahid S Hashemian
Hi Ewen, I tested the quickstart with this release candidate on Ubuntu, Windows (32 and 64 bit) and Mac. Everything looks good, except for one issue on Windows 7 32 bit when running the WordCount demo application from the Kafka Streams step. Running that command gave me this error, and as a res

[GitHub] kafka pull request #2497: [MINOR] fixed some JavaDoc typos

2017-02-03 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2497 [MINOR] fixed some JavaDoc typos You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hofixJavaDocs0102 Alternatively you can review and a

[GitHub] kafka pull request #2498: MINOR: Make Kafka Connect step in quickstart more ...

2017-02-03 Thread vahidhashemian
GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/2498 MINOR: Make Kafka Connect step in quickstart more Windows friendly You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-02-03 Thread Guozhang Wang
Thanks for the proposal Colin. A few comments below: 1. There are a couple of classes that looks new to me but not defined anywhere. For example: NewTopic (topic name and configs?), TopicInfo (is this a wrapper of MetadataResponse.TopicMetadata?), NodeApiVersions, GroupOverview. Could you provide

[jira] [Assigned] (KAFKA-4654) Improve test coverage MemoryLRUCache

2017-02-03 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck reassigned KAFKA-4654: -- Assignee: Bill Bejeck > Improve test coverage MemoryLRUCache >

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

2017-02-03 Thread Dong Lin
Thanks for the reply, Colin. I have some comments inline. In addition, I also have some comments regarding the Future() in response to your latest email. As Ismael mentioned, we have added purgeDataBefore() API in AdminClient. This API returns Future() that allows user to purge data in either syn

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-03 Thread radai
+1 On Fri, Feb 3, 2017 at 11:25 AM, Mayuresh Gharat wrote: > Hi All, > > It seems that there is no further concern with the KIP-111. At this point > we would like to start the voting process. The KIP can be found at > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=67638388 > >

Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

2017-02-03 Thread radai
why is it so important to make those classes final ? On Fri, Feb 3, 2017 at 3:33 PM, Matthias J. Sax wrote: > Hi All, > > I did prepare a KIP to do some cleanup some of Kafka's Streaming API. > > Please have a look here: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 120%3A+Cleanup+K

[GitHub] kafka pull request #2499: KAFKA-4461 Added support to ProcessorTopologyTestD...

2017-02-03 Thread amccague
GitHub user amccague opened a pull request: https://github.com/apache/kafka/pull/2499 KAFKA-4461 Added support to ProcessorTopologyTestDriver for internal topics. This resolves an issue in driving tests using the ProcessorTopologyTestDriver when `groupBy()` is invoked downstream of

[jira] [Commented] (KAFKA-4461) When using ProcessorTopologyTestDriver, the combination of map and .groupByKey does not produce any result

2017-02-03 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852496#comment-15852496 ] ASF GitHub Bot commented on KAFKA-4461: --- GitHub user amccague opened a pull request:

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-02-03 Thread Dong Lin
Hey ALexey, Thanks for all the comments! I have updated the KIP to specify how we enforce quota. I also updated the "The thread model and broker logic for moving replica data between log directories" to make it easier to read. You can find the exact change here

  1   2   >