Hi Jay.
Agreed, we're planning to try to do the compression before the producer
with 0.8 as we get back to this part, hopefully this week. I saw your other
JIRA issue about the copying in the code path so that looks promising as
well.
Thanks for all the responses, really appreciate the effort and
Cool. With respect to compression performance, we definitely see the same
thing, no debate.
Of course if you want to just compress the message payloads you can do that
now without needing much help from kafka--just pass in the compressed data.
Whether it not it will do much depends on the size of
Thanks for the responses. Additional follow-up inline.
On Fri, Aug 2, 2013 at 2:21 PM, Jay Kreps wrote:
> Great comments, answers inline!
>
> On Fri, Aug 2, 2013 at 12:28 PM, Chris Hogue wrote:
>
> > These sounds like great steps. A couple of votes and questions:
> >
> > 1. Moving serializati
Great comments, answers inline!
On Fri, Aug 2, 2013 at 12:28 PM, Chris Hogue wrote:
> These sounds like great steps. A couple of votes and questions:
>
> 1. Moving serialization out and basing it all off of byte[] for key and
> payload makes sense. Echoing a response below, we've ended up doing
00 PM
> To: dev@kafka.apache.org; us...@kafka.apache.org
> Subject: Client improvement discussion
>
> I sent around a wiki a few weeks back proposing a set of client
> improvements that essentially amount to a rewrite of the producer and
> consumer java clients.
>
> https://cwiki.apac
These sounds like great steps. A couple of votes and questions:
1. Moving serialization out and basing it all off of byte[] for key and
payload makes sense. Echoing a response below, we've ended up doing that in
some cases anyway, and the others do a trivial transform to bytes with an
Encoder.
2
used from within Ruby would make it easier to have the same
> version of the client in Java and Ruby.
>
> -Original Message-
> From: Jay Kreps [mailto:jay.kr...@gmail.com]
> Sent: Friday, July 26, 2013 3:00 PM
> To: dev@kafka.apache.org; us...@kafka.apache.org
> Subject
that can be used
from within Ruby would make it easier to have the same version of the client in
Java and Ruby.
-Original Message-
From: Jay Kreps [mailto:jay.kr...@gmail.com]
Sent: Friday, July 26, 2013 3:00 PM
To: dev@kafka.apache.org; us...@kafka.apache.org
Subject: Client improvement
Jay,
This seems like a great direction. Simplifying the consumer client would
be a big win, and +1 for more native java client integration.
On the last point, regarding memory usage for buffering per partition. I
would think it could be possible to devise a dynamic queuing system, to
allow high
I sent around a wiki a few weeks back proposing a set of client
improvements that essentially amount to a rewrite of the producer and
consumer java clients.
https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite
The below discussion assumes you have read this wiki.
I started to do a li
10 matches
Mail list logo
