[jira] [Commented] (KAFKA-3769) KStream job spending 60% of time writing metrics

[ https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309288#comment-15309288 ] Jay Kreps commented on KAFKA-3769: -- Nice catch. > KStream job spending 60% of time writi

[jira] [Assigned] (KAFKA-2894) WorkerSinkTask doesn't handle rewinding offsets on rebalance

[ https://issues.apache.org/jira/browse/KAFKA-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liquan Pei reassigned KAFKA-2894: - Assignee: Liquan Pei (was: Ewen Cheslack-Postava) > WorkerSinkTask doesn't handle rewinding offs

[jira] [Commented] (KAFKA-3773) SocketServer inflightResponses collection leaks memory on client disconnect

[ https://issues.apache.org/jira/browse/KAFKA-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308919#comment-15308919 ] ASF GitHub Bot commented on KAFKA-3773: --- Github user hachikuji closed the pull reque

[GitHub] kafka pull request: KAFKA-3773: Remove from inflightResponses on client disc...

Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/1452 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[jira] [Resolved] (KAFKA-3773) SocketServer inflightResponses collection leaks memory on client disconnect

[ https://issues.apache.org/jira/browse/KAFKA-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3773. Resolution: Fixed Reviewer: Ismael Juma > SocketServer inflightResponses collection leaks memo

[jira] [Updated] (KAFKA-3773) SocketServer inflightResponses collection leaks memory on client disconnect

[ https://issues.apache.org/jira/browse/KAFKA-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3773: --- Fix Version/s: 0.9.0.2 > SocketServer inflightResponses collection leaks memory on client disconnect >

Re: kafka consumer group is rebalancing

Hi Sunny, It would be helpful to know the processing logic after the records are returned by poll(). How do you make sure that the polling is happening within session timeout? Did you try to reduce the number of messages returned by setting max.poll.records to a smaller value? Thanks, Liquan On

[GitHub] kafka pull request: KAFKA-3773: Remove from inflightResponses on client disc...

Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/1452 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[jira] [Updated] (KAFKA-3769) KStream job spending 60% of time writing metrics

[ https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3769: - Reviewer: Guozhang Wang > KStream job spending 60% of time writing metrics > -

[jira] [Updated] (KAFKA-3715) Higher granularity streams metrics

[ https://issues.apache.org/jira/browse/KAFKA-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3715: - Reviewer: Eno Thereska > Higher granularity streams metrics > ---

[jira] [Commented] (KAFKA-3773) SocketServer inflightResponses collection leaks memory on client disconnect

[ https://issues.apache.org/jira/browse/KAFKA-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308718#comment-15308718 ] ASF GitHub Bot commented on KAFKA-3773: --- GitHub user hachikuji reopened a pull reque

[jira] [Created] (KAFKA-3773) SocketServer inflightResponses collection leaks memory on client disconnect

Jason Gustafson created KAFKA-3773: -- Summary: SocketServer inflightResponses collection leaks memory on client disconnect Key: KAFKA-3773 URL: https://issues.apache.org/jira/browse/KAFKA-3773 Project

[jira] [Commented] (KAFKA-3773) SocketServer inflightResponses collection leaks memory on client disconnect

[ https://issues.apache.org/jira/browse/KAFKA-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308717#comment-15308717 ] ASF GitHub Bot commented on KAFKA-3773: --- Github user hachikuji closed the pull reque

[GitHub] kafka pull request: KAFKA-3773: Remove from inflightResponses on client disc...

GitHub user hachikuji reopened a pull request: https://github.com/apache/kafka/pull/1452 KAFKA-3773: Remove from inflightResponses on client disconnect to prevent memory leak You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachi

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

Hey Onur, Thanks for the detailed response. I think the problem of controlling rebalance times is the main (known) gap in the proposal as it stands. This burden goes away if you loosen the liveness property by having a > required rebalance time and optional processing time where rebalance > happe

[jira] [Updated] (KAFKA-3748) Add consumer-property to console tools consumer (similar to --producer-property)

[ https://issues.apache.org/jira/browse/KAFKA-3748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated KAFKA-3748: -- Fix Version/s: (was: 0.9.0.1) (was: 0.9.0.0) > Add consumer-p

[jira] [Commented] (KAFKA-3111) java.lang.ArithmeticException: / by zero in ConsumerPerformance

[ https://issues.apache.org/jira/browse/KAFKA-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308547#comment-15308547 ] Vahid Hashemian commented on KAFKA-3111: [~junrao] Could you please take a look at

[jira] [Commented] (KAFKA-3758) KStream job fails to recover after Kafka broker stopped

[ https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308306#comment-15308306 ] Greg Fodor commented on KAFKA-3758: --- No, the kstream job was running across 2 servers, a

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308263#comment-15308263 ] Martin Nowak commented on KAFKA-3764: - I didn't configure that, so it was the default

Build failed in Jenkins: kafka-trunk-jdk8 #664

See Changes: [me] MINOR: Avoid trace logging computation in -- [...truncated 3189 lines...] kafka.log.LogTest > testOpenDeletesObsoleteFiles PASSED kafka.log.LogTest > testSizeBasedLogRoll PASS

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308174#comment-15308174 ] Ismael Juma commented on KAFKA-3764: There could be a bug in the conversion, but we ha

RE: [VOTE] KIP-58 - Make Log Compaction Point Configurable

During the vote it was suggested the name of the property be changed from: log.cleaner.compaction.lag.ms to log.cleaner.compaction.delay.ms Note that this feature concerns controlling the portion of the head of the log (new messages go on the head) that will be left un-compacted (i.e. i

[GitHub] kafka pull request: HOTFIX: remove from inflightResponses on client disconne...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/1452 HOTFIX: remove from inflightResponses on client disconnect to prevent memory leak You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/k

RE: [VOTE] KIP-58 - Make Log Compaction Point Configurable

Thanks everyone for voting. The vote passed with 5 binding +1's (Gwen, Jay, Ewen, Ismael, Jun) and with 9 nb's (Becket, James, Tom, Mnikumar, Ben, Grant, Joel, Eric) The discussion should move to KAFKA-1981

Build failed in Jenkins: kafka-trunk-jdk7 #1328

See Changes: [me] MINOR: Avoid trace logging computation in -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker Ubuntu ubuntu yahoo-not

[jira] [Commented] (KAFKA-3758) KStream job fails to recover after Kafka broker stopped

[ https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308047#comment-15308047 ] Guozhang Wang commented on KAFKA-3758: -- I think there may be an issue about directory

[GitHub] kafka pull request: MINOR: Avoid trace logging computation in `checkEnoughRe...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1449 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307964#comment-15307964 ] Martin Nowak commented on KAFKA-3764: - The client driver still is on the 0.9.0 protoco

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307936#comment-15307936 ] Ismael Juma commented on KAFKA-3764: Yes, it's definitely useful to have the bug repor

[jira] [Commented] (KAFKA-3693) Race condition between highwatermark-checkpoint thread and handleLeaderAndIsrRequest at broker start-up

[ https://issues.apache.org/jira/browse/KAFKA-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307932#comment-15307932 ] Maysam Yabandeh commented on KAFKA-3693: [~junrao] Any thoughts? > Race condition

[jira] [Updated] (KAFKA-3772) MirrorMaker crashes on Corrupted Message

[ https://issues.apache.org/jira/browse/KAFKA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Ranson updated KAFKA-3772: Description: We recently came across an issue where a message on our source kafka cluster became co

[jira] [Created] (KAFKA-3772) MirrorMaker crashes on Corrupted Message

James Ranson created KAFKA-3772: --- Summary: MirrorMaker crashes on Corrupted Message Key: KAFKA-3772 URL: https://issues.apache.org/jira/browse/KAFKA-3772 Project: Kafka Issue Type: Bug

Garbage free Kafka client

Hi. I am committer on the Apache Log4j 2 project. We are working on providing garbage-free logging, as described here: http://logging.apache.org/log4j/2.x/manual/garbagefree.html I work on the Kafka appender in Log4j 2: http://logging.apache.org/log4j/2.x/manual/appenders.html#KafkaAppender https

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307885#comment-15307885 ] Martin Nowak commented on KAFKA-3764: - Well this bug report is mostly about the fact t

[jira] [Commented] (KAFKA-2318) replica manager repeatedly tries to fetch from partitions already moved during controlled shutdown

[ https://issues.apache.org/jira/browse/KAFKA-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307514#comment-15307514 ] Wanli Tian commented on KAFKA-2318: --- same issue > replica manager repeatedly tries to f

[jira] [Commented] (KAFKA-2081) testUncleanLeaderElectionEnabledByTopicOverride transient failure

[ https://issues.apache.org/jira/browse/KAFKA-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307492#comment-15307492 ] Balint Molnar commented on KAFKA-2081: -- [~junrao] I think this one is not happening a

[jira] [Commented] (KAFKA-1573) Transient test failures on LogTest.testCorruptLog

[ https://issues.apache.org/jira/browse/KAFKA-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307481#comment-15307481 ] Balint Molnar commented on KAFKA-1573: -- [~gwenshap] I think this one is not failing a

[jira] [Commented] (KAFKA-1534) transient unit test failure in testBasicPreferredReplicaElection

[ https://issues.apache.org/jira/browse/KAFKA-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307435#comment-15307435 ] Balint Molnar commented on KAFKA-1534: -- [~nehanarkhede] I think this issue is already