[jira] [Comment Edited] (KAFKA-664) Kafka server threads die due to OOME during long running test

[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528733#comment-13528733 ] Neha Narkhede edited comment on KAFKA-664 at 12/11/12 6:45 AM: --

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528733#comment-13528733 ] Neha Narkhede commented on KAFKA-664: - That's correct. > Kafka server

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528723#comment-13528723 ] Jun Rao commented on KAFKA-664: --- Got it. So the issue is that for low volume topics, the fetc

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528713#comment-13528713 ] Neha Narkhede commented on KAFKA-664: - For the current implementation, that is true onl

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528707#comment-13528707 ] Jun Rao commented on KAFKA-664: --- If the problem is due to an expired request not being remove

[jira] [Commented] (KAFKA-374) Move to java CRC32 implementation

[ https://issues.apache.org/jira/browse/KAFKA-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528690#comment-13528690 ] Jun Rao commented on KAFKA-374: --- Should this be a post 0.8 item? > Move to j

[jira] [Commented] (KAFKA-581) provides windows batch script for starting Kafka/Zookeeper

[ https://issues.apache.org/jira/browse/KAFKA-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528688#comment-13528688 ] Jun Rao commented on KAFKA-581: --- Thanks for the patch. Committed the stop scripts to 0.8.

Re: Objections to doing this in 0.8?

That sounds goods to me. Thanks, Jun On Mon, Dec 10, 2012 at 3:05 PM, Jay Kreps wrote: > https://issues.apache.org/jira/browse/KAFKA-667 > > Goal would just be forward compatibility with a more sane naming scheme... > > -Jay >

[jira] [Commented] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528559#comment-13528559 ] Jun Rao commented on KAFKA-646: --- Thanks for patch v4. Looks good overall. A few minor comment

[jira] [Commented] (KAFKA-513) Add state change log to Kafka brokers

[ https://issues.apache.org/jira/browse/KAFKA-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528432#comment-13528432 ] Jay Kreps commented on KAFKA-513: - It would be nice if we updated our log4j.properties as p

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528429#comment-13528429 ] Neha Narkhede commented on KAFKA-664: - Joel, patch v2 works better and I see why v1 did

Re: Objections to doing this in 0.8?

Prefer doing this now than later. Thanks, Neha On Mon, Dec 10, 2012 at 3:05 PM, Jay Kreps wrote: > https://issues.apache.org/jira/browse/KAFKA-667 > > Goal would just be forward compatibility with a more sane naming scheme... > > -Jay

[jira] [Commented] (KAFKA-513) Add state change log to Kafka brokers

[ https://issues.apache.org/jira/browse/KAFKA-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528408#comment-13528408 ] Neha Narkhede commented on KAFKA-513: - I think we cannot release without this functiona

Objections to doing this in 0.8?

https://issues.apache.org/jira/browse/KAFKA-667 Goal would just be forward compatibility with a more sane naming scheme... -Jay

[jira] [Created] (KAFKA-667) Rename .highwatermark file

Jay Kreps created KAFKA-667: --- Summary: Rename .highwatermark file Key: KAFKA-667 URL: https://issues.apache.org/jira/browse/KAFKA-667 Project: Kafka Issue Type: Improvement Affects Versions: 0.

[jira] [Updated] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-646: Attachment: kafka-646-patch-num1-v4.patch Fixed a typo in FetchRequestAndResponseStats mbean creatio

[jira] [Updated] (KAFKA-647) Provide a property in System Test for no. of topics and topics string will be generated automatically

[ https://issues.apache.org/jira/browse/KAFKA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-647: Status: Patch Available (was: Open) > Provide a property in System Test for no. of topics and topics st

[jira] [Updated] (KAFKA-647) Provide a property in System Test for no. of topics and topics string will be generated automatically

[ https://issues.apache.org/jira/browse/KAFKA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-647: Attachment: kafka-647-v1.patch Uploaded kafka-647-v1.patch for the following changes: 1. To let System Test

[jira] [Updated] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-646: Attachment: kafka-646-patch-num1-v3.patch Thanks for reviewing. Patch v3: 1. Oh, that's because cli

Re: Kafka TLP website move

I have updated the ticket with the appropriate details. Thanks all. https://issues.apache.org/jira/browse/INFRA-5586 -Jay On Mon, Dec 10, 2012 at 9:32 AM, Joe Schaefer wrote: > > You need to tell us HOW to create it, > either as a CMS site or using svnpubsub! > >

[jira] [Updated] (KAFKA-664) Kafka server threads die due to OOME during long running test

[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-664: - Attachment: kafka-664-draft-2.patch Can you try this instead? The update should run as requests expire, no

Re: 0.8 Protocol Status

I haven't gone back to check the code, but it feels like every size that's given can be used to verify how to proceed. In the case I was having, 200k made no sense because there wasn't 200k worth of data to write, but the system still flushed the message out to disk, presumably due to using an ear

[jira] [Resolved] (KAFKA-597) Refactor KafkaScheduler

[ https://issues.apache.org/jira/browse/KAFKA-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-597. - Resolution: Fixed Assignee: Jay Kreps > Refactor KafkaScheduler > --- > >

[jira] [Commented] (KAFKA-597) Refactor KafkaScheduler

[ https://issues.apache.org/jira/browse/KAFKA-597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528195#comment-13528195 ] Joel Koshy commented on KAFKA-597: -- +1 - thanks for the patch. > Refactor

[jira] [Updated] (KAFKA-597) Refactor KafkaScheduler

[ https://issues.apache.org/jira/browse/KAFKA-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-597: Attachment: KAFKA-597-v6.patch Attached patch v6, which fixes Joel's comments-- - Formatting issue - Scalado

[jira] [Commented] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528183#comment-13528183 ] Neha Narkhede commented on KAFKA-646: - Patch v2 looks good to me. Few minor questions -

[jira] Subscription: outstanding kafka patches

Issue Subscription Filter: outstanding kafka patches (57 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-664 Kafka server threads die due to OOME during long running test https://issues.apache.org/jira/browse/KAFKA-664 KA

Re: Kafka TLP website move

Our experience with CMS has been good, btw. Consider as an option. You get a very simple mark-down based web site with browser editing if you like it. Works very well. On Mon, Dec 10, 2012 at 10:26 AM, Jay Kreps wrote: > Cool, makes sense. Let's go with SVN and svnpubsub then. The site > subd

Async segment delete patch

Hello fellow log maintainers, I have a patch and would love your feedback: https://issues.apache.org/jira/browse/KAFKA-636 There will be a few more in this series as I finish off the log compaction work. This patch is against trunk, though we may end up needing to backport it if we hit delete-rela

Re: Kafka TLP website move

Cool, makes sense. Let's go with SVN and svnpubsub then. The site subdirectory that we would published is https://svn.apache.org/repos/asf/kafka/site In the future if we switch to git we will just leave the site in svn and continue to use that for site updates. -Jay On Mon, Dec 10, 2012 at 10

[jira] [Resolved] (KAFKA-654) Irrecoverable error while trying to roll a segment that already exists

[ https://issues.apache.org/jira/browse/KAFKA-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-654. - Resolution: Fixed Assignee: Neha Narkhede Thanks for the review, committed patch v1 to 0.8 b

[jira] [Closed] (KAFKA-654) Irrecoverable error while trying to roll a segment that already exists

[ https://issues.apache.org/jira/browse/KAFKA-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-654. --- > Irrecoverable error while trying to roll a segment that already exists > ---

Re: Kafka TLP website move

We have an SVN repo for the web site and a git repo for code. On Mon, Dec 10, 2012 at 10:13 AM, Jay Kreps wrote: > I am confused. The CMS documentation says this: > * * > > *Instead of developing versioning support and a notification scheme into > a database driven CMS, Apache's subversion > in

Re: Kafka TLP website move

I am confused. The CMS documentation says this: * * *Instead of developing versioning support and a notification scheme into a database driven CMS, Apache's subversion infrastructurewas chosen as the central data store for everything. The fact that the web interface to the

[jira] [Commented] (KAFKA-604) Add missing metrics in 0.8

[ https://issues.apache.org/jira/browse/KAFKA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528116#comment-13528116 ] Yang Ye commented on KAFKA-604: --- Sure, I'll do that soon Best, --- Victo

Re: 0.8 Protocol Status

>> Am I correct in believing that the broker doesn't sanity check the message size field against the received data? I'm not sure there is a good way the broker can do that. The only way the broker knows if the message was corrupted is when it is unable to read the header of the next message. When

[jira] [Commented] (KAFKA-604) Add missing metrics in 0.8

[ https://issues.apache.org/jira/browse/KAFKA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528095#comment-13528095 ] Jun Rao commented on KAFKA-604: --- The patch no longer applies. We should probably also add the

Re: Kafka TLP website move

You need to tell us HOW to create it, either as a CMS site or using svnpubsub! > > From: Jay Kreps >To: "infrastruct...@apache.org" >Cc: dev@kafka.apache.org >Sent: Monday, December 10, 2012 12:30 PM >Subject: Re: Kafka TLP website move > > >Ooops, wrong ti

[jira] [Commented] (KAFKA-654) Irrecoverable error while trying to roll a segment that already exists

[ https://issues.apache.org/jira/browse/KAFKA-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528087#comment-13528087 ] Jay Kreps commented on KAFKA-654: - +1 I had fixed this on trunk: http://svn.apache.org/rep

Re: Kafka TLP website move

Ooops, wrong ticket: https://issues.apache.org/jira/browse/INFRA-5586 :-) -Jay On Mon, Dec 10, 2012 at 9:29 AM, Jay Kreps wrote: > Hey guys, > > It's been a few weeks and we are still waiting on getting a top-level > website url for Kafka. Tried just making it myself, but that didn't work:

Kafka TLP website move

Hey guys, It's been a few weeks and we are still waiting on getting a top-level website url for Kafka. Tried just making it myself, but that didn't work: jkreps@minotaur:/www$ mkdir kafka.apache.org mkdir: kafka.apache.org: Permission denied Are we confused? Can anyone help? Here is the tick

Re: 0.8 Protocol Status

Neha, thanks for the tip. Useful util! My problem was simple -- I missed one of the size field changes in the producer, which led to a completely wrong size field. Am I correct in believing that the broker doesn't sanity check the message size field against the received data? In this case, the