Chris commented in another thread about the poor compression performance in
0.8, even with snappy.
Indeed if I run the linear log write throughput test on my laptop I see
75MB/sec with no compression and 17MB/sec with snappy.
This is a little surprising as snappy claims 200MB round-trip performan
[
https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728454#comment-13728454
]
Jay Kreps commented on KAFKA-527:
-
I should mention that to run the above test you need to
[
https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps updated KAFKA-615:
Attachment: KAFKA-615-v5.patch
Attach updated patch v5. Rebased against trunk and with added support for
co
[
https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728452#comment-13728452
]
Jay Kreps commented on KAFKA-527:
-
To test performance checkout trunk and do
./sbt package
[
https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps updated KAFKA-527:
Attachment: java.hprof.snappy.text
> Compression support does numerous byte copies
> ---
[
https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps updated KAFKA-527:
Attachment: java.hprof.no-compression.txt
> Compression support does numerous byte copies
>
[
https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps updated KAFKA-527:
Description:
The data path for compressing or decompressing messages is extremely
inefficient. We do someth
Cool. With respect to compression performance, we definitely see the same
thing, no debate.
Of course if you want to just compress the message payloads you can do that
now without needing much help from kafka--just pass in the compressed data.
Whether it not it will do much depends on the size of
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728358#comment-13728358
]
Swapnil Ghike edited comment on KAFKA-992 at 8/3/13 1:39 AM:
-
A
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728358#comment-13728358
]
Swapnil Ghike commented on KAFKA-992:
-
Also the while loop should be fixed, the first s
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728354#comment-13728354
]
Swapnil Ghike commented on KAFKA-992:
-
Makes sense. Just one comment, you can use the s
Joel Koshy created KAFKA-998:
Summary: Producer should not retry on non-recoverable error codes
Key: KAFKA-998
URL: https://issues.apache.org/jira/browse/KAFKA-998
Project: Kafka
Issue Type: Bug
Thanks for the responses. Additional follow-up inline.
On Fri, Aug 2, 2013 at 2:21 PM, Jay Kreps wrote:
> Great comments, answers inline!
>
> On Fri, Aug 2, 2013 at 12:28 PM, Chris Hogue wrote:
>
> > These sounds like great steps. A couple of votes and questions:
> >
> > 1. Moving serializati
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-992:
Attachment: KAFKA-992.v3.patch
Thanks for the comments Neha.
1,2,5: Done
3. 3.1 and 3.2/3 are dist
[
https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Koshy closed KAFKA-915.
> System Test - Mirror Maker testcase_5001 failed
> ---
>
>
[
https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Koshy resolved KAFKA-915.
--
Resolution: Fixed
> System Test - Mirror Maker testcase_5001 failed
> --
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728218#comment-13728218
]
Neha Narkhede edited comment on KAFKA-992 at 8/2/13 11:15 PM:
--
[
https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728247#comment-13728247
]
Joel Koshy commented on KAFKA-915:
--
+1 on the patch. I actually could not reproduce the ot
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728230#comment-13728230
]
Guozhang Wang commented on KAFKA-992:
-
Swapnil, we also considered this option. The pro
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728221#comment-13728221
]
Neha Narkhede commented on KAFKA-992:
-
Swapnil,
- You are right in observing that zook
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728218#comment-13728218
]
Neha Narkhede commented on KAFKA-992:
-
Thanks for patch v2, Guozhang. Few review sugges
[
https://issues.apache.org/jira/browse/KAFKA-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-649:
Attachment: KAFKA-649.v6.patch
Thanks for the comments.
50,51,52: Done.
> Cleanup
[
https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-955:
Attachment: KAFKA-955.v3.patch
Add the testMessageSizeTooLargeWithAckZero to syncProducerTest, which
[
https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728209#comment-13728209
]
Joel Koshy commented on KAFKA-984:
--
Thanks for the patch - this will help a *lot* especial
Great comments, answers inline!
On Fri, Aug 2, 2013 at 12:28 PM, Chris Hogue wrote:
> These sounds like great steps. A couple of votes and questions:
>
> 1. Moving serialization out and basing it all off of byte[] for key and
> payload makes sense. Echoing a response below, we've ended up doing
[
https://issues.apache.org/jira/browse/KAFKA-989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Hargett updated KAFKA-989:
---
Status: Open (was: Patch Available)
Not good enough. Deadlocks because ShutdownableThread.shutdown g
I believe there are some open source C++ producer implementations. At
linkedin we have a C++ implementation. We would like to open source this if
there is interest. We would like to eventually include a C++ consumer as
well.
-Jay
On Mon, Jul 29, 2013 at 6:03 AM, Sybrandy, Casey <
casey.sybra...@
These sounds like great steps. A couple of votes and questions:
1. Moving serialization out and basing it all off of byte[] for key and
payload makes sense. Echoing a response below, we've ended up doing that in
some cases anyway, and the others do a trivial transform to bytes with an
Encoder.
2
[
https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727910#comment-13727910
]
Guozhang Wang commented on KAFKA-955:
-
Sorry for the name misleading, I did not shut do
[
https://issues.apache.org/jira/browse/KAFKA-989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727844#comment-13727844
]
Phil Hargett edited comment on KAFKA-989 at 8/2/13 5:33 PM:
Her
[
https://issues.apache.org/jira/browse/KAFKA-649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727846#comment-13727846
]
Jun Rao commented on KAFKA-649:
---
Thanks for patch v5.
50. ConsoleConsumer: There is no need
[
https://issues.apache.org/jira/browse/KAFKA-989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Hargett updated KAFKA-989:
---
Attachment: KAFKA-989-failed-to-find-leader.patch
Here's the patch :)
> Race conditio
[
https://issues.apache.org/jira/browse/KAFKA-989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Hargett updated KAFKA-989:
---
Status: Patch Available (was: Open)
This patch may minimize the issue, as there does seem to be a rac
[
https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727824#comment-13727824
]
Jun Rao commented on KAFKA-955:
---
Thanks for patch v2. Some more comments.
20. testSendWithAc
[
https://issues.apache.org/jira/browse/KAFKA-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Meder updated KAFKA-997:
Attachment: strict-verification-2.patch
> Provide a strict verification mode when reading configuration
[
https://issues.apache.org/jira/browse/KAFKA-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Meder updated KAFKA-997:
Attachment: (was: strict-verification.patch)
> Provide a strict verification mode when reading confi
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-992:
Description:
The current behavior of zookeeper for ephemeral nodes is that session
expiration and e
[
https://issues.apache.org/jira/browse/KAFKA-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Meder updated KAFKA-997:
Status: Patch Available (was: Open)
> Provide a strict verification mode when reading configuration pro
[
https://issues.apache.org/jira/browse/KAFKA-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Meder updated KAFKA-997:
Attachment: strict-verification.patch
Add strict verification property and use it in tests and similar situa
Sam Meder created KAFKA-997:
---
Summary: Provide a strict verification mode when reading
configuration properties
Key: KAFKA-997
URL: https://issues.apache.org/jira/browse/KAFKA-997
Project: Kafka
I
[
https://issues.apache.org/jira/browse/KAFKA-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Meder updated KAFKA-943:
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
Resolving this issue since the patch
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727438#comment-13727438
]
Swapnil Ghike edited comment on KAFKA-992 at 8/2/13 7:11 AM:
-
-
[
https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727438#comment-13727438
]
Swapnil Ghike commented on KAFKA-992:
-
- I think I am not completely clear why timestam
43 matches
Mail list logo
