Consumer lag issues

2013-10-21 Thread Jason Rosenberg
(I've changed the subject of this thread (was "Preparing for the 0.8 final release")) So, I'm not sure that my issue is exactly the same as that mentioned in the FAQ. Anyway, in looking at the MaxLag values for several consumers (not all consuming the same topics), it looks like there was a stran

Exception management in kafka async producer

2013-10-21 Thread Alan Lavintman
Hi guys, i was wondering what will be the best way to provide delivery guarantee service by using an async producer in kafka 0.7.2. I want my service to be ready to support kafka downtimes for at least 12 hours so i would like to aggregate logs locally into some failover spool and then rerun them w

Re: New Committer

2013-10-21 Thread David Arthur
Thanks, Joe. I'm happy to be on board. -David On 10/17/13 2:21 PM, Joe Stein wrote: I am pleased to announce that David Arthur has been voted to become a committer for Apache Kafka. Welcome, David, and thanks for your ongoing contributions! /*** Joe S

Re: Exception management in kafka async producer

2013-10-21 Thread Neha Narkhede
In 0.7.x and 0.8.x, there is no way for the async producer to notify the client through a callback. However, this is being planned for the upcoming 0.9 release. You can take a look at the proposed API here - https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProd

[jira] [Created] (KAFKA-1095) Kafka does not compile with sbt

2013-10-21 Thread Marcel Lohmann (JIRA)
Marcel Lohmann created KAFKA-1095: - Summary: Kafka does not compile with sbt Key: KAFKA-1095 URL: https://issues.apache.org/jira/browse/KAFKA-1095 Project: Kafka Issue Type: Bug Com

[jira] [Commented] (KAFKA-1095) Kafka does not compile with sbt

2013-10-21 Thread Marcel Lohmann (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800776#comment-13800776 ] Marcel Lohmann commented on KAFKA-1095: --- with `./sbt "++2.9.2 package"` produces man

[jira] [Commented] (KAFKA-1095) Kafka does not compile with sbt

2013-10-21 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800779#comment-13800779 ] Guozhang Wang commented on KAFKA-1095: -- This is due one current jira KAFKA-1042, with

[jira] [Updated] (KAFKA-1042) Fix segment flush logic

2013-10-21 Thread Joris Van Remoortere (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van Remoortere updated KAFKA-1042: Attachment: flush_segment_fix-v3.patch Compilation error fix > Fix segment flush l

[jira] [Comment Edited] (KAFKA-1042) Fix segment flush logic

2013-10-21 Thread Joris Van Remoortere (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800895#comment-13800895 ] Joris Van Remoortere edited comment on KAFKA-1042 at 10/21/13 6:07 PM: -

[jira] [Commented] (KAFKA-1095) Kafka does not compile with sbt

2013-10-21 Thread Joris Van Remoortere (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800900#comment-13800900 ] Joris Van Remoortere commented on KAFKA-1095: - I added a patch (v3) to KAFKA-1

[jira] [Resolved] (KAFKA-1042) Fix segment flush logic

2013-10-21 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-1042. -- Resolution: Fixed > Fix segment flush logic > --- > > Key:

[jira] [Commented] (KAFKA-1042) Fix segment flush logic

2013-10-21 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800910#comment-13800910 ] Neha Narkhede commented on KAFKA-1042: -- Thanks for the follow up patch. Committed it

[jira] [Created] (KAFKA-1096) An old controller coming out of long GC could update its epoch to the latest controller's epoch

2013-10-21 Thread Swapnil Ghike (JIRA)
Swapnil Ghike created KAFKA-1096: Summary: An old controller coming out of long GC could update its epoch to the latest controller's epoch Key: KAFKA-1096 URL: https://issues.apache.org/jira/browse/KAFKA-1096

Re: [jira] [Created] (KAFKA-1095) Kafka does not compile with sbt

2013-10-21 Thread David Arthur
Ideally, compile issues like this are caught by some CI system. I see there are some builds on https://builds.apache.org/ for Kafka which are currently disabled. Are there plans to re-enable these builds? -David On 10/21/13 12:22 PM, Marcel Lohmann (JIRA) wrote: Marcel Lohmann created KAFKA-1

[jira] Subscription: outstanding kafka patches

2013-10-21 Thread jira
Issue Subscription Filter: outstanding kafka patches (66 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-1093 Log.getOffsetsBefore(t, …) does not return the last confirmed offset before t https://issues.apache.org/jira/browse/KAF

[jira] [Updated] (KAFKA-1096) An old controller coming out of long GC could update its epoch to the latest controller's epoch

2013-10-21 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1096: - Fix Version/s: 0.8.1 > An old controller coming out of long GC could update its epoch to the late

Re: Review Request 14730: Patch for KAFKA-1001

2013-10-21 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14730/ --- (Updated Oct. 21, 2013, 8:35 p.m.) Review request for kafka. Bugs: KAFKA-1001

[jira] [Updated] (KAFKA-1001) Handle follower transition in batch

2013-10-21 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1001: - Attachment: KAFKA-1001_2013-10-21_13:35:41.patch > Handle follower transition in batch >

[jira] [Commented] (KAFKA-1001) Handle follower transition in batch

2013-10-21 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801048#comment-13801048 ] Guozhang Wang commented on KAFKA-1001: -- Updated reviewboard https://reviews.apache.or

[jira] [Commented] (KAFKA-156) Messages should not be dropped when brokers are unavailable

2013-10-21 Thread Steve Morin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801089#comment-13801089 ] Steve Morin commented on KAFKA-156: --- Scott Carey - completely agree with your use-case he

[jira] [Created] (KAFKA-1097) Race condition while reassigning partition leads to incorrect ISR information in zookeeper

2013-10-21 Thread Neha Narkhede (JIRA)
Neha Narkhede created KAFKA-1097: Summary: Race condition while reassigning partition leads to incorrect ISR information in zookeeper Key: KAFKA-1097 URL: https://issues.apache.org/jira/browse/KAFKA-1097

[jira] [Updated] (KAFKA-1097) Race condition while reassigning partition leads to incorrect ISR information in zookeeper

2013-10-21 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1097: --- Fix Version/s: 0.8 > Race condition while reassigning partition leads to incorrect ISR information > in zook

Re: preparing for the 0.8 final release

2013-10-21 Thread Jun Rao
Hi, Everyone, At this moment, we have only one remaining jira (KAFKA-1097) that we plan to fix in 0.8. After that, we can cut the final 0.8 release. Thanks, Jun On Mon, Oct 7, 2013 at 5:33 PM, Jun Rao wrote: > Hi, Everyone, > > I made another pass of the remaining jiras that we plan to fix i