Re: compression performance

2013-08-16 Thread Jan Kotek
> you see that with no compression 80% of the time goes to FileChannel.write, > But with snappy enabled only 5% goes to writing data, 50% of the time goes > to byte copying and allocation, and only about 22% goes to actual I had similar problem with MapDB, it was solved by using memory mapped fil

[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-16 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742296#comment-13742296 ] Guozhang Wang commented on KAFKA-992: - As few more thoughts about 113: currently leader

[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742329#comment-13742329 ] Jun Rao commented on KAFKA-992: --- Thanks for patch v12. A few more comments. 114. If that's t

[jira] [Commented] (KAFKA-1010) Concurrency issue in getCluster() causes rebalance failure and dead consumer

2013-08-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742347#comment-13742347 ] Jun Rao commented on KAFKA-1010: Thanks for the patch. It doesn't seem to apply to 0.8 tho

[jira] [Commented] (KAFKA-1004) Handle topic event for trivial whitelist topic filters

2013-08-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742351#comment-13742351 ] Jun Rao commented on KAFKA-1004: Thanks for the patch. That looks good. However, shouldn't

[jira] [Commented] (KAFKA-1010) Concurrency issue in getCluster() causes rebalance failure and dead consumer

2013-08-16 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742354#comment-13742354 ] Sam Meder commented on KAFKA-1010: -- weird, patch -p1 -i get_cluster_0_8.patch worked, but

[jira] [Updated] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-16 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-992: Attachment: KAFKA-992.v13.patch > Double Check on Broker Registration to Avoid False NodeExist E

[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-16 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742356#comment-13742356 ] Guozhang Wang commented on KAFKA-992: - Thanks for the comments Jun. 114. Agreed, delet

[jira] [Updated] (KAFKA-1010) Concurrency issue in getCluster() causes rebalance failure and dead consumer

2013-08-16 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-1010: - Attachment: get_cluster_0_8_git.patch > Concurrency issue in getCluster() causes rebalance failure an

[jira] [Updated] (KAFKA-1010) Concurrency issue in getCluster() causes rebalance failure and dead consumer

2013-08-16 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-1010: - Attachment: (was: get_cluster_0_8.patch) > Concurrency issue in getCluster() causes rebalance fai

[jira] [Commented] (KAFKA-1010) Concurrency issue in getCluster() causes rebalance failure and dead consumer

2013-08-16 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742365#comment-13742365 ] Sam Meder commented on KAFKA-1010: -- Git formatted patch is now attached.

[jira] [Updated] (KAFKA-1004) Handle topic event for trivial whitelist topic filters

2013-08-16 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1004: - Attachment: KAFKA-1004.v2.patch > Handle topic event for trivial whitelist topic filters > --

[jira] [Commented] (KAFKA-1004) Handle topic event for trivial whitelist topic filters

2013-08-16 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742369#comment-13742369 ] Guozhang Wang commented on KAFKA-1004: -- Good point. v2 applied. Also removed part of

[jira] [Commented] (KAFKA-1004) Handle topic event for trivial whitelist topic filters

2013-08-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742386#comment-13742386 ] Jun Rao commented on KAFKA-1004: Looks good to me. Joel, do you want to take another look?

[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742385#comment-13742385 ] Jun Rao commented on KAFKA-992: --- Thanks for patch v13. Looks good. My only suggestion is to s

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-08-16 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742391#comment-13742391 ] Jay Kreps commented on KAFKA-1008: -- I think it makes sense to do it as part of this patch

[jira] [Updated] (KAFKA-1010) Concurrency issue in getCluster() causes rebalance failure and dead consumer

2013-08-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1010: --- Resolution: Fixed Assignee: Sam Meder (was: Neha Narkhede) Status: Resolved (was: Patch Availa

[jira] [Updated] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-16 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-992: Attachment: KAFKA-992.v14.patch Thanks for the comments, added to resign, rebased on 0.8.

Debug with IntelliJ

2013-08-16 Thread Timothy Chen
Hi Kafka devs, I wonder if anyone of you are using IntelliJ and tried to debug with the debugger? My breakpoint is hitting but I cannot get any local variable debug info so its quite useless. Any idea how to fix this? Thanks, Tim

[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742837#comment-13742837 ] Jun Rao commented on KAFKA-992: --- Thanks for patch v14. Committed to 0.8. > D

[jira] [Updated] (KAFKA-1008) Unmap before resizing

2013-08-16 Thread Elizabeth Wei (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elizabeth Wei updated KAFKA-1008: - Attachment: (was: unmap-v4.patch) > Unmap before resizing > - > >

[jira] [Commented] (KAFKA-1008) Unmap before resizing

2013-08-16 Thread Elizabeth Wei (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742853#comment-13742853 ] Elizabeth Wei commented on KAFKA-1008: -- Thanks Jay! Please review the patch to see if

[jira] [Updated] (KAFKA-1008) Unmap before resizing

2013-08-16 Thread Elizabeth Wei (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elizabeth Wei updated KAFKA-1008: - Attachment: unmap-v5.patch > Unmap before resizing > - > >