[jira] [Updated] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

2012-12-12 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-646: Attachment: kafka-646-patch-num1-v6.patch Made the changes. > Provide aggregate sta

can't commit to Kafka svn

2012-12-12 Thread Jun Rao
Hi, I can't seem to commit to Kafka svn since this morning. The commit is on a fresh checkout. Do you know what the issue is? jrao-mn:0.8_top_temp jrao$ svn commit --username junrao -m "trivial fix to increase offset commit interval to reduce ZK load" Sendingcore/src/main/scala/kafka/cons

[jira] [Created] (KAFKA-670) Cleanup spurious .index files if present in the log directory on initialization

2012-12-12 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-670: --- Summary: Cleanup spurious .index files if present in the log directory on initialization Key: KAFKA-670 URL: https://issues.apache.org/jira/browse/KAFKA-670 Project: Kafka

[jira] [Updated] (KAFKA-670) Cleanup spurious .index files if present in the log directory on initialization

2012-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-670: Priority: Blocker (was: Major) > Cleanup spurious .index files if present in the log directory on > in

[jira] [Commented] (KAFKA-604) Add missing metrics in 0.8

2012-12-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530165#comment-13530165 ] Joel Koshy commented on KAFKA-604: -- The rebased patch does not really comment on/address t

[jira] [Commented] (KAFKA-657) Add an API to commit offsets

2012-12-12 Thread David Arthur (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530174#comment-13530174 ] David Arthur commented on KAFKA-657: Thanks [~jkreps], that clears things up quite a bi

[jira] [Updated] (KAFKA-669) Irrecoverable error on leader while rolling to a new segment

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-669: Labels: bugs (was: replication-performance) > Irrecoverable error on leader while rolling to a

[jira] [Updated] (KAFKA-669) Irrecoverable error on leader while rolling to a new segment

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-669: Summary: Irrecoverable error on leader while rolling to a new segment (was: Kafka server sends prod

[jira] [Updated] (KAFKA-669) Irrecoverable error on leader while rolling to a new segment

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-669: Description: I setup a test cluster of 5 brokers and setup migration tool to send data for ~1000 pa

[jira] [Issue Comment Deleted] (KAFKA-669) Irrecoverable error on leader while rolling to a new segment

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-669: Comment: was deleted (was: This issue was caused by KAFKA-654 on the leader broker 3. The root cause

[jira] [Commented] (KAFKA-604) Add missing metrics in 0.8

2012-12-12 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530200#comment-13530200 ] Swapnil Ghike commented on KAFKA-604: - Yes I like this idea, can follow this in KAFKA-6

[jira] [Updated] (KAFKA-669) Irrecoverable error on leader while rolling to a new segment

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-669: Description: I setup a test cluster of 5 brokers and setup migration tool to send data for ~1000 pa

[jira] [Updated] (KAFKA-646) Provide aggregate stats at the high level Producer and ZookeeperConsumerConnector level

2012-12-12 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-646: Attachment: kafka-646-patch-num1-v7.patch Including in patch v7 Joel's suggestion at KAFKA-604 to me

[jira] [Commented] (KAFKA-604) Add missing metrics in 0.8

2012-12-12 Thread Yang Ye (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530223#comment-13530223 ] Yang Ye commented on KAFKA-604: --- Joel, thanks for the idea, it's good and I will do a new pat

[jira] [Updated] (KAFKA-664) Kafka server threads die due to OOME during long running test

2012-12-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-664: - Attachment: KAFKA-664-v3.patch Here is a simpler and hopefully safer approach of using a global request c

[jira] [Commented] (KAFKA-624) Add 07 ConsoleConsumer to validate message content for Migration Tool testcases

2012-12-12 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530232#comment-13530232 ] John Fung commented on KAFKA-624: - Currently, the migration tool testcases (9001 ~ 9006) da

[jira] [Commented] (KAFKA-624) Add 07 ConsoleConsumer to validate message content for Migration Tool testcases

2012-12-12 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530244#comment-13530244 ] John Fung commented on KAFKA-624: - The 07 ProducerPerformance produces messages with random

[jira] [Created] (KAFKA-671) DelayedProduce requests should not hold full producer request data

2012-12-12 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-671: Summary: DelayedProduce requests should not hold full producer request data Key: KAFKA-671 URL: https://issues.apache.org/jira/browse/KAFKA-671 Project: Kafka Issue

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

2012-12-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530254#comment-13530254 ] Joel Koshy commented on KAFKA-664: -- Also, just filed KAFKA-671 to address the issue that J

[jira] [Updated] (KAFKA-670) Cleanup spurious .index files if present in the log directory on initialization

2012-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-670: Attachment: KAFKA-670-v1.patch Cleanup any bogus index files (those without a corresponding .log file) durin

[jira] [Commented] (KAFKA-330) Add delete topic support

2012-12-12 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530293#comment-13530293 ] Sam Meder commented on KAFKA-330: - I've been doing a lot of manual resetting of data in Kaf

[jira] [Created] (KAFKA-672) Merge 0.8 changes to trunk

2012-12-12 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-672: --- Summary: Merge 0.8 changes to trunk Key: KAFKA-672 URL: https://issues.apache.org/jira/browse/KAFKA-672 Project: Kafka Issue Type: Bug Reporter: Jay Kreps

[jira] [Updated] (KAFKA-672) Merge 0.8 changes to trunk

2012-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-672: Attachment: KAFKA-672.patch Here is the merge changes in case anyone wants to take a look. I will probably

[jira] [Commented] (KAFKA-626) Produce requests dropped due to socket timeouts on get metadata requests

2012-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530514#comment-13530514 ] Jay Kreps commented on KAFKA-626: - So the only bug was KAFKA-608, right? I mean it remains

[jira] [Commented] (KAFKA-670) Cleanup spurious .index files if present in the log directory on initialization

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530524#comment-13530524 ] Neha Narkhede commented on KAFKA-670: - +1. Do you think this would resolve KAFKA-669 as

[jira] [Commented] (KAFKA-626) Produce requests dropped due to socket timeouts on get metadata requests

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530526#comment-13530526 ] Neha Narkhede commented on KAFKA-626: - Correct. The right fix is to forward the request

[jira] [Closed] (KAFKA-626) Produce requests dropped due to socket timeouts on get metadata requests

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-626. --- > Produce requests dropped due to socket timeouts on get metadata requests > -

[jira] [Comment Edited] (KAFKA-626) Produce requests dropped due to socket timeouts on get metadata requests

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530526#comment-13530526 ] Neha Narkhede edited comment on KAFKA-626 at 12/13/12 12:22 AM: -

[jira] [Resolved] (KAFKA-626) Produce requests dropped due to socket timeouts on get metadata requests

2012-12-12 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-626. - Resolution: Fixed Assignee: Neha Narkhede > Produce requests dropped due to socket timeo

[jira] [Updated] (KAFKA-513) Add state change log to Kafka brokers

2012-12-12 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-513: Attachment: kafka-513-v1.patch Patch v1: 1. Created a tool PartitionStateChangeLogMerger - It looks

Re: can't commit to Kafka svn

2012-12-12 Thread David Nalley
On Wed, Dec 12, 2012 at 12:27 PM, Jun Rao wrote: > Hi, > > I can't seem to commit to Kafka svn since this morning. The commit is on a > fresh checkout. Do you know what the issue is? > > jrao-mn:0.8_top_temp jrao$ svn commit --username junrao -m "trivial fix to > increase offset commit interval to

RE: can't commit to Kafka svn

2012-12-12 Thread Gavin McDonald
Issue is you are migrating to git. https://issues.apache.org/jira/browse/INFRA-5111 From: Jun Rao [mailto:jun...@gmail.com] Sent: Thursday, 13 December 2012 3:58 AM To: Apache Infrastructure Cc: dev@kafka.apache.org Subject: can't commit to Kafka svn Hi, I can't seem to comm

[jira] [Commented] (KAFKA-374) Move to java CRC32 implementation

2012-12-12 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530654#comment-13530654 ] Scott Carey commented on KAFKA-374: --- Awesome. I was just profiling some Kafka 0.7.1 stuf

Re: can't commit to Kafka svn

2012-12-12 Thread Jay Kreps
I checked it out and ran tests. Looks good to me. Not sure if any one else wants to kick the tires before we cut over? -Jay On Wed, Dec 12, 2012 at 6:34 PM, Gavin McDonald wrote: > Issue is you are migrating to git. > > ** ** > > https://issues.apache.org/jira/browse/INFRA-5111 > > ** **

[jira] [Commented] (KAFKA-374) Move to java CRC32 implementation

2012-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530674#comment-13530674 ] Jay Kreps commented on KAFKA-374: - Hey Scott, currently the implementation of the consumer

CRC patch

2012-12-12 Thread Jay Kreps
This patch is pretty safe, I did a pretty serious test against the java impl on millions of crcs. The code change is just a few lines. I would like to get this on trunk. Review? https://issues.apache.org/jira/browse/KAFKA-374 Also, do folks have a preference between the java and scala version? -

[jira] [Commented] (KAFKA-669) Irrecoverable error on leader while rolling to a new segment

2012-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530694#comment-13530694 ] Jay Kreps commented on KAFKA-669: - I don't see the source of the logging you added to print

[jira] [Commented] (KAFKA-657) Add an API to commit offsets

2012-12-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530700#comment-13530700 ] Jay Kreps commented on KAFKA-657: - I wonder if you could take a look at the updated docs an

[jira] [Commented] (KAFKA-664) Kafka server threads die due to OOME during long running test

2012-12-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530709#comment-13530709 ] Jun Rao commented on KAFKA-664: --- Thanks for patch v3. +1 from me. One minor comment: Should w

Re: can't commit to Kafka svn

2012-12-12 Thread Joe Stein
I am getting a failed test on trunk (both on git and svn though) so don't see that related. Test Failed: testTimeBasedLogRollESC[0m junit.framework.AssertionFailedError: There should still be exactly one segment. expected:<1> but was:<2> the commits and code look alright from cursory checks of tr

Re: CRC patch

2012-12-12 Thread Joe Stein
+1 on the code change looks alright, I prefer using the java version in this case only because we are using an implementation from another project and its a drop in ... no reason to change things without good reason, yup. On Wed, Dec 12, 2012 at 11:55 PM, Jay Kreps wrote: > This patch is pretty

Re: EasyMock?

2012-12-12 Thread Joe Stein
Hey Derek, I totally got swamped at work and didn't get a chance to bang on the sbt change. I am not a sbt whiz you might be better to get this over the finish line is it something you might be able to-do? if so I can review and commit, np IMHO 0.8 should not ship with out supporting latest sbt