[GitHub] kafka pull request #2626: MINOR: Fix typo in GlobalKTable javadocs

2017-03-02 Thread miguno
GitHub user miguno opened a pull request: https://github.com/apache/kafka/pull/2626 MINOR: Fix typo in GlobalKTable javadocs You can merge this pull request into a Git repository by running: $ git pull https://github.com/miguno/kafka trunk-globalktable-typos Alternatively you

[jira] [Updated] (KAFKA-4823) Creating Kafka Producer on application running on Java older version

2017-03-02 Thread Aneesh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aneesh updated KAFKA-4823: -- Summary: Creating Kafka Producer on application running on Java older version (was: Creating Kafka Producer on

[jira] [Updated] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4092: --- Fix Version/s: (was: 0.10.2.0) > retention.bytes should not be allowed to be less than segment.byt

[jira] [Commented] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892019#comment-15892019 ] Ismael Juma commented on KAFKA-4092: [~ewencp], yeah, I was thinking about what's the

[jira] [Updated] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4092: --- Description: Right now retention.bytes can be as small as the user wants but it doesn't really get ac

[jira] [Updated] (KAFKA-4092) retention.bytes should not be allowed to be less than segment.bytes

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4092: --- Description: Right now retention.bytes can be as small as the user wants but it doesn't really get ac

[jira] [Created] (KAFKA-4832) kafka producer send message to the wrong IP cannot to stop producer.close()

2017-03-02 Thread Wang Hong (JIRA)
Wang Hong created KAFKA-4832: Summary: kafka producer send message to the wrong IP cannot to stop producer.close() Key: KAFKA-4832 URL: https://issues.apache.org/jira/browse/KAFKA-4832 Project: Kafka

[GitHub] kafka pull request #2627: [WIP]: add a consumer leave group config

2017-03-02 Thread dguy
GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2627 [WIP]: add a consumer leave group config Add a new config to ConsumerConfig, `leave.group.on.close`. When set to `false` then the `AbstractCoordinator` will not send the `LeaveGroupRequest`. Default i

[jira] [Updated] (KAFKA-4832) kafka producer send Async message to the wrong IP cannot to stop producer.close()

2017-03-02 Thread Wang Hong (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Hong updated KAFKA-4832: - Description: 1.When I tried to send msg by Async with wrong IP in loop 1400times 10batches. 2.I use javaap

[jira] [Updated] (KAFKA-4832) kafka producer send Async message to the wrong IP cannot to stop producer.close()

2017-03-02 Thread Wang Hong (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang Hong updated KAFKA-4832: - Description: 1.When I tried to send msg by Async with wrong IP in loop 1400times 10batches. 2.I use javaap

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

2017-03-02 Thread Damian Guy
Hi Florian, Thanks for the KIP. It seems there is some overlap here with what we already have in KafkaStreams.allMetadata(). This currently returns a Collection where each StreamsMetadata instance holds the state stores and partition assignment for every instance of the KafkaStreams application.

Allow to replace Zookeeper with different Coordination Service (etcd/consul)

2017-03-02 Thread Molnár Bálint
Hi, I was wonderring to refactor Kafka core code to be able to use different Coordination Service than Zookeeper. I know I will need to create a KIP for that. I think the first part of this task to refactor the classes which are using the ZkUtil methods to use a zookeeper independent trait instead

[jira] [Updated] (KAFKA-3038) Speeding up partition reassignment after broker failure

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3038: --- Fix Version/s: (was: 0.11.0.0) > Speeding up partition reassignment after broker failure > ---

[jira] [Updated] (KAFKA-2507) Replace ControlledShutdown{Request,Response} with org.apache.kafka.common.requests equivalent

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2507: --- Fix Version/s: (was: 0.11.0.0) 0.12.0.0 > Replace ControlledShutdown{Request,Re

[jira] [Updated] (KAFKA-4094) Fix importance labels for Kafka Server config

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4094: --- Labels: newbie (was: ) > Fix importance labels for Kafka Server config >

[jira] [Updated] (KAFKA-4094) Fix importance labels for Kafka Server config

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4094: --- Fix Version/s: (was: 0.11.0.0) > Fix importance labels for Kafka Server config > -

Re: Consumption on a explicitly (dynamically) created topic has a 5 minute delay

2017-03-02 Thread Rajini Sivaram
This issue is being addressed in KAFKA-4631. See https://issues.apache.org/jira/browse/KAFKA-4631 and the discussion in the PR https://github.com/apache/kafka/pull/2622 for details. Regards, Rajini On Thu, Mar 2, 2017 at 4:35 AM, Jaikiran Pai wrote: > For future reference - I asked this questi

[jira] [Commented] (KAFKA-1368) Upgrade log4j

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892265#comment-15892265 ] Ismael Juma commented on KAFKA-1368: [~ecomar], are you interested in picking this up?

[jira] [Commented] (KAFKA-1368) Upgrade log4j

2017-03-02 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892271#comment-15892271 ] Edoardo Comar commented on KAFKA-1368: -- Yes [~ijuma] we will pick this one up. I work

[jira] [Updated] (KAFKA-4711) Change Default unclean.leader.election.enabled from True to False (KIP-106)

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4711: --- Affects Version/s: (was: 0.11.0.0.0) > Change Default unclean.leader.election.enabled from True to

[jira] [Updated] (KAFKA-4767) KafkaProducer is not joining its IO thread properly

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4767: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > KafkaProducer is not joinin

[jira] [Updated] (KAFKA-4210) Improve Replication Follower Throttling Stability By Checking Quota During Backoff

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4210: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Improve Replication Followe

[jira] [Updated] (KAFKA-4239) Replication Quotas: kafka-configs.sh should allow the throttled rate to be set using broker wildcards

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4239: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Replication Quotas: kafka-c

[jira] [Updated] (KAFKA-4604) Gradle Eclipse plugin creates projects for non project subfolders

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4604: --- Affects Version/s: (was: 0.11.0.0.0) > Gradle Eclipse plugin creates projects for non project subf

[jira] [Updated] (KAFKA-3958) To set the maximum no of consumers which can consume from the particular topic.

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3958: --- Affects Version/s: (was: 0.11.0.0.0) > To set the maximum no of consumers which can consume from t

[jira] [Updated] (KAFKA-3356) Remove ConsumerOffsetChecker, deprecated in 0.9, in 0.11

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3356: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Remove ConsumerOffsetChecke

[jira] [Updated] (KAFKA-3353) Remove deprecated producer configs.

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3353: --- Affects Version/s: (was: 0.11.0.0.0) 0.10.2.0 > Remove deprecated producer

[jira] [Updated] (KAFKA-3353) Remove deprecated producer configs.

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3353: --- Fix Version/s: 0.11.0.0 > Remove deprecated producer configs. > --- >

[jira] [Commented] (KAFKA-1368) Upgrade log4j

2017-03-02 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892282#comment-15892282 ] Ismael Juma commented on KAFKA-1368: Differences in the configuration format: https:/

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

2017-03-02 Thread Ismael Juma
I've updated JIRA versions and the KIP wiki page. Ismael On Thu, Mar 2, 2017 at 12:51 AM, Ismael Juma wrote: > Thanks everyone for the feedback. Since everyone was in favour and we had > covered this particular case in the time-based release plan[1], I went > ahead and updated the wiki page to

Re: groupBy without auto-repartition topics for Kafka Streams

2017-03-02 Thread Tianji Li
Hi Guys, Thanks so much for your quick replies, very appreciated! Thanks Tianji On Wed, Mar 1, 2017 at 2:53 PM, Matthias J. Sax wrote: > It should be: > > groupBy -> always trigger repartitioning > groupByKey -> maybe trigger repartitioning > > And there will not be two repartitioning topics.

[GitHub] kafka pull request #2628: MINOR: Bump version to 0.11.0.0-SNAPSHOT

2017-03-02 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/2628 MINOR: Bump version to 0.11.0.0-SNAPSHOT There won't be a 0.10.3.0. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka bump-version-to-0.1

[jira] [Comment Edited] (KAFKA-4825) Likely Data Loss in ReassignPartitionsTest System Test

2017-03-02 Thread Ben Stopford (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890661#comment-15890661 ] Ben Stopford edited comment on KAFKA-4825 at 3/2/17 2:55 PM: -

Re: Consumption on a explicitly (dynamically) created topic has a 5 minute delay

2017-03-02 Thread Jaikiran Pai
Thank you for pointing me to that JIRA. It indeed is the same issue we discussed in this thread. I'll keep a watch on that JIRA for the code to be merged. -Jaikiran On Thursday 02 March 2017 07:11 PM, Rajini Sivaram wrote: This issue is being addressed in KAFKA-4631. See https://issues.apache

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

2017-03-02 Thread Jay Kreps
I second the concern on with the one producer per task approach. At a high-level it seems to make sense but I think Damian is exactly right that that cuts against the general design of the producer. Many people have high input partition counts and will have high task counts as a result. I think pro

0.10.2.0 Release Retrospective

2017-03-02 Thread Ewen Cheslack-Postava
Hi all, I think it'd be useful to review & refine our release process, especially in light of the new time based releases. I've created a page under the release plan for 0.10.2.0 and added some of my notes: https://cwiki.apache.org/confluence/display/KAFKA/0.10.2.0+Retrospective If others could

Add me to jira/confluence?

2017-03-02 Thread Jeff Widman
Can someone add me to Jira/confluence for Kafka? Don't use this work email, use j...@jeffwidman.com Thanks!

Re: Add me to jira/confluence?

2017-03-02 Thread Guozhang Wang
Done. Guozhang On Thu, Mar 2, 2017 at 9:08 AM, Jeff Widman wrote: > Can someone add me to Jira/confluence for Kafka? > > Don't use this work email, use j...@jeffwidman.com > > Thanks! > -- -- Guozhang

[GitHub] kafka pull request #2470: Kafka-4711 :Change Default unclean.leader.election...

2017-03-02 Thread sharad-develop
Github user sharad-develop closed the pull request at: https://github.com/apache/kafka/pull/2470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #2625: KAFKA:4623- Change default unclean.leader.election...

2017-03-02 Thread sharad-develop
GitHub user sharad-develop reopened a pull request: https://github.com/apache/kafka/pull/2625 KAFKA:4623- Change default unclean.leader.election.enabled from True to False KAFKA:4623- Change default unclean.leader.election.enabled from True to False You can merge this pull request

[GitHub] kafka pull request #2616: KAFKA:4738 - Remove generic type of class ClientSt...

2017-03-02 Thread sharad-develop
GitHub user sharad-develop reopened a pull request: https://github.com/apache/kafka/pull/2616 KAFKA:4738 - Remove generic type of class ClientState Remove generic type of class ClientState and generic T inTaskAssignor. You can merge this pull request into a Git repository by runnin

[jira] [Commented] (KAFKA-4722) Add application.id to StreamThread name

2017-03-02 Thread Sharad (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892648#comment-15892648 ] Sharad commented on KAFKA-4722: --- I have enabled github notifications. Also, I have updated t

Re: [VOTE] KIP-107: Add purgeDataBefore() API in AdminClient

2017-03-02 Thread Jun Rao
Hi, Dong, It seems that delete means removing everything while purge means removing a portion. So, it seems that it's better to be able to distinguish the two? Thanks, Jun On Wed, Mar 1, 2017 at 1:57 PM, Dong Lin wrote: > Hi all, > > I have updated the KIP to include a script that allows user

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-03-02 Thread Jun Rao
Hi, Dong, Thanks for the KIP. A few comments below. 1. For moving data across directories 1.1 I am not sure why we want to use ReplicaFetcherThread to move data around in the leader. ReplicaFetchThread fetches data from socket. For moving data locally, it seems that we want to avoid the socket ov

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

2017-03-02 Thread Jun Rao
Hi, Dong, Ok. We can keep LeaderAndIsrRequest as it is in the wiki. Since we need both KIP-112 and KIP-113 to make a compelling case for JBOD, perhaps we should discuss KIP-113 before voting for both? I left some comments in the other thread. Thanks, Jun On Wed, Mar 1, 2017 at 1:58 PM, Dong Li

Re: [VOTE] KIP-107: Add purgeDataBefore() API in AdminClient

2017-03-02 Thread Dong Lin
Hey Jun, Thanks for this information. I am not aware of this difference between the purge and delete. Given this difference, I will prefer to the existing name of the purge. Ismael, please let me if you are strong about using delete. Thanks, Dong On Thu, Mar 2, 2017 at 9:40 AM, Jun Rao wrote:

[GitHub] kafka pull request #2606: kafka4811: ReplicaFetchThread may fail to create d...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2606 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-4811) ReplicaFetchThread may fail to create due to existing metric

2017-03-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-4811. Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 2606 [https://github.com/apa

[jira] [Created] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
Jiangjie Qin created KAFKA-4833: --- Summary: Transient unit test failure in AbstractCoordinatorTest Key: KAFKA-4833 URL: https://issues.apache.org/jira/browse/KAFKA-4833 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-4833: Affects Version/s: 0.10.2.0 > Transient unit test failure in AbstractCoordinatorTest > -

[jira] [Updated] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-4833: Description: I saw this couple of times when running unit test locally. Haven't looked into this in

Re: [VOTE] KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-03-02 Thread Grant Henke
+1 On Wed, Mar 1, 2017 at 9:44 PM, Gwen Shapira wrote: > +1 > > On Tue, Feb 28, 2017 at 2:40 AM, Ismael Juma wrote: > > > Hi everyone, > > > > Since the few who responded in the discuss thread were in favour and > there > > were no objections, I would like to initiate the voting process for > >

Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Jeff Widman
I thought I saw mention somewhere of changing the default of auto.create.topics.enable to false. I searched, but couldn't find anything in JIRA... am I imagining things? Now that there's API support for creating topics, the version bump to 0.11.0 seems like a good time to re-evaluate whether this

Build failed in Jenkins: kafka-trunk-jdk8 #1316

2017-03-02 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA4811; ReplicaFetchThread may fail to create due to existing metric -- [...truncated 159.64 KB...] kafka.utils.timer.TimerTest > testTaskExpirat

Re: [VOTE] KIP-119: Drop Support for Scala 2.10 in Kafka 0.11

2017-03-02 Thread Becket Qin
Thanks for the clarification, Ismael. In that case, it is reasonable to drop support for Scala 2.10. LinkedIn is probably fine with this change. I did not notice we have recommended Scala version on the download page. +1 on the KIP On Thu, Mar 2, 2017 at 10:46 AM, Grant Henke wrote: > +1 > > O

Re: Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Grant Henke
I think the idea was that once clients have the ability to create topics, we would move "auto topic creation" client side and deprecate and eventually remove the support for server side auto create. This simplifies error handling, authorization, and puts the client in control of details like partit

[jira] [Resolved] (KAFKA-4833) Transient unit test failure in AbstractCoordinatorTest

2017-03-02 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin resolved KAFKA-4833. - Resolution: Invalid Just noticed that the test was failing on a review branch instead of trunk...

Re: Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Jeff Widman
Thanks, that's the ticket I was thinking of. On Thu, Mar 2, 2017 at 11:19 AM, Grant Henke wrote: > I think the idea was that once clients have the ability to create topics, > we would move "auto topic creation" client side and deprecate and > eventually remove the support for server side auto cr

Re: [DISCUSS] KIP-124: Request rate quotas

2017-03-02 Thread Jun Rao
Another way to express an absolute limit is to use request.percentage, but treat it as the percentage used in a single request handling thread. For now, the request handling threads can be just the io threads. In the future, they can cover the network threads as well. This is similar to how top rep

Re: Is there already a KIP or JIRA to change auto.create.topics.enable to default to false?

2017-03-02 Thread Grant Henke
Feel free to create a KIP based on that discussion and drive the jira. I don't think a KIP exists yet. On Thu, Mar 2, 2017 at 1:28 PM, Jeff Widman wrote: > Thanks, that's the ticket I was thinking of. > > On Thu, Mar 2, 2017 at 11:19 AM, Grant Henke wrote: > > > I think the idea was that once c

[jira] [Commented] (KAFKA-4778) OOM on kafka-streams instances with high numbers of unreaped Record classes

2017-03-02 Thread Dave Thomas (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892886#comment-15892886 ] Dave Thomas commented on KAFKA-4778: [~guozhang] Yes on rocksDB - how we use it is mos

Re: [DISCUSS] KIP-124: Request rate quotas

2017-03-02 Thread Rajini Sivaram
Jun, If we use request.percentage as the percentage used in a single I/O thread, the total percentage being allocated will be num.io.threads * 100 for I/O threads and num.network.threads * 100 for network threads. A single quota covering the two as a percentage wouldn't quite work if you want to a

First contribution

2017-03-02 Thread Marc Juchli
Hi all. We are a group of master students, about to start contributing to the apache Kafka project . Are there any suggestions for an open issue for us to knock our heads on and propose a fix within a reasonable amount of time? Thanks! Marc

Re: First contribution

2017-03-02 Thread Guozhang Wang
Hi Marc, You can take a look at this page: https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20labels%20%3D%20newbie%20AND%20status%20%3D%20Open Guozhang On Thu, Mar 2, 2017 at 12:59 PM, Marc Juchli wrote: > Hi all. > > We are a group of master students, about to start

[GitHub] kafka pull request #96: KAFKA-2358 Cluster collection returning methods neve...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/96 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[jira] [Updated] (KAFKA-2358) Cluster collection returning methods should never return null

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2358: - Fix Version/s: 0.10.3.0 > Cluster collection returning methods should never return null >

[jira] [Commented] (KAFKA-2358) Cluster collection returning methods should never return null

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893029#comment-15893029 ] ASF GitHub Bot commented on KAFKA-2358: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-2358) Cluster collection returning methods should never return null

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2358: - Fix Version/s: (was: 0.10.3.0) 0.11.0.0 > Cluster collection returning meth

Build failed in Jenkins: kafka-trunk-jdk7 #1980

2017-03-02 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA4811; ReplicaFetchThread may fail to create due to existing metric -- [...truncated 843.93 KB...] org.apache.kafka.streams.processor.internals.S

[jira] [Resolved] (KAFKA-4652) Improve test coverage KStreamBuilder

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4652. -- Resolution: Fixed Issue resolved by pull request 2597 [https://github.com/apache/kafka/pull/2597

[GitHub] kafka pull request #2597: KAFKA-4652: Added tests for KStreamBuilder

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2597 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4652) Improve test coverage KStreamBuilder

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893041#comment-15893041 ] ASF GitHub Bot commented on KAFKA-4652: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #2626: MINOR: Fix typo in GlobalKTable javadocs

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2626 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #2621: KAFKA-4800: Streams State transition ASCII diagram...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2621 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4800) Streams State transition ASCII diagrams need fixing and polishing

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893051#comment-15893051 ] ASF GitHub Bot commented on KAFKA-4800: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-4800) Streams State transition ASCII diagrams need fixing and polishing

2017-03-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4800. -- Resolution: Fixed Fix Version/s: 0.10.2.1 Issue resolved by pull request 2621 [https://gi

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

2017-03-02 Thread Jorge Esteban Quilcate Otoya
Hi, I have incorporated the latest revisions into the KIP and created a PR to check the implementation details. If there are no more issues, the VOTE thread has already started. Looking forward to your comments. Jorge. El mar., 28 feb. 2017 a las 19:46, Vahid S Hashemian (< vahidhashem...@us.ib

[jira] [Commented] (KAFKA-4828) ProcessorTopologyTestDriver does not work when using .through()

2017-03-02 Thread Hamidreza Afzali (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893105#comment-15893105 ] Hamidreza Afzali commented on KAFKA-4828: - There are two issues: - ProcessorTopolo

[GitHub] kafka pull request #2629: KAFKA-4828: ProcessorTopologyTestDriver does not w...

2017-03-02 Thread hrafzali
GitHub user hrafzali opened a pull request: https://github.com/apache/kafka/pull/2629 KAFKA-4828: ProcessorTopologyTestDriver does not work when using through This resolves the following issues in the ProcessorTopologyTestDriver: - It should not create an internal changelog

[jira] [Commented] (KAFKA-4828) ProcessorTopologyTestDriver does not work when using .through()

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893141#comment-15893141 ] ASF GitHub Bot commented on KAFKA-4828: --- GitHub user hrafzali opened a pull request:

[GitHub] kafka pull request #2481: Kafka 4706 :Unify StreamsKafkaClient instances

2017-03-02 Thread sharad-develop
Github user sharad-develop closed the pull request at: https://github.com/apache/kafka/pull/2481 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #2630: MINOR: improve hanging ResetIntegrationTest

2017-03-02 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2630 MINOR: improve hanging ResetIntegrationTest You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka minor-reset-integration-test Alternativel

Jenkins build is back to normal : kafka-trunk-jdk7 #1981

2017-03-02 Thread Apache Jenkins Server
See

Jenkins build is back to normal : kafka-trunk-jdk8 #1317

2017-03-02 Thread Apache Jenkins Server
See

[jira] [Resolved] (KAFKA-4631) Refresh consumer metadata more frequently for unknown subscribed topics

2017-03-02 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-4631. Resolution: Fixed Issue resolved by pull request 2622 [https://github.com/apache/kafka/pull/

[GitHub] kafka pull request #2622: KAFKA-4631: Request metadata in consumer if topic/...

2017-03-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2622 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

2017-03-02 Thread Hai Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893563#comment-15893563 ] Hai Lin commented on KAFKA-4669: We have the same issue with Kafka 0.8.2.1 to Kafka 0.9 cl

[jira] [Commented] (KAFKA-4631) Refresh consumer metadata more frequently for unknown subscribed topics

2017-03-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893567#comment-15893567 ] ASF GitHub Bot commented on KAFKA-4631: --- Github user asfgit closed the pull request

[jira] [Created] (KAFKA-4834) Kafka cannot delete topic with ReplicaStateMachine went wrong

2017-03-02 Thread Dan (JIRA)
Dan created KAFKA-4834: -- Summary: Kafka cannot delete topic with ReplicaStateMachine went wrong Key: KAFKA-4834 URL: https://issues.apache.org/jira/browse/KAFKA-4834 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #2631: KAFKA:4706 - Unify StreamsKafkaClient instances

2017-03-02 Thread sharad-develop
GitHub user sharad-develop opened a pull request: https://github.com/apache/kafka/pull/2631 KAFKA:4706 - Unify StreamsKafkaClient instances KAFKA:4706 - Unify StreamsKafkaClient instances You can merge this pull request into a Git repository by running: $ git pull https://githu

[jira] [Commented] (KAFKA-4834) Kafka cannot delete topic with ReplicaStateMachine went wrong

2017-03-02 Thread huxi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893619#comment-15893619 ] huxi commented on KAFKA-4834: - How did you delete the topic? Seems the partitions' state is st

[jira] [Assigned] (KAFKA-4631) Refresh consumer metadata more frequently for unknown subscribed topics

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-4631: -- Assignee: Rajini Sivaram (was: Matthias J. Sax) > Refresh consumer metadata more frequ

[jira] [Updated] (KAFKA-4706) Unify StreamsKafkaClient instances

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4706: --- Status: Patch Available (was: Open) https://github.com/apache/kafka/pull/2631 > Unify Stream

[jira] [Updated] (KAFKA-4722) Add application.id to StreamThread name

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4722: --- Status: Patch Available (was: Open) https://github.com/apache/kafka/pull/2617 > Add applicat

[jira] [Updated] (KAFKA-4738) Remove generic type of class ClientState

2017-03-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4738: --- Status: Patch Available (was: Open) > Remove generic type of class ClientState >

[jira] [Commented] (KAFKA-1342) Slow controlled shutdowns can result in stale shutdown requests

2017-03-02 Thread Todd Palino (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893728#comment-15893728 ] Todd Palino commented on KAFKA-1342: [~wushujames], I'm not sure about increasing the

Re: [DISCUSS] KIP-124: Request rate quotas

2017-03-02 Thread Jun Rao
Hi, Rajini, Consider modeling as n * 100% unit. For 2), the question is what's causing the I/O threads to be saturated. It's unlikely that all users' utilization have increased at the same. A more likely case is that a few isolated users' utilization have increased. If so, after increasing the num

[jira] [Commented] (KAFKA-3182) Failure in kafka.network.SocketServerTest.testSocketsCloseOnShutdown

2017-03-02 Thread Armin Braun (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893845#comment-15893845 ] Armin Braun commented on KAFKA-3182: I'll try to fix it this morning, can easily repro

[jira] [Assigned] (KAFKA-3182) Failure in kafka.network.SocketServerTest.testSocketsCloseOnShutdown

2017-03-02 Thread Armin Braun (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Armin Braun reassigned KAFKA-3182: -- Assignee: Armin Braun > Failure in kafka.network.SocketServerTest.testSocketsCloseOnShutdown >

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-03-02 Thread Dong Lin
Hey Jun, Thanks for all the comments! Please see my answer below. I have updated the KIP to address most of the questions and make the KIP easier to understand. Thanks, Dong On Thu, Mar 2, 2017 at 9:35 AM, Jun Rao wrote: > Hi, Dong, > > Thanks for the KIP. A few comments below. > > 1. For movi

  1   2   >