Hello, I am currently testing the 0.8 branch (and it works quite well). We
plan to not use the replication feature for now since we don't really need
it, we can afford to lose data in case of unrecoverable failure from a
broker.
However, we really don't want to have producers/consumers fail if a b
This is a good point. We have discussed this a little bit before. The key
constraint is that with replication factor 1 you can choose one of the
following: (1) high availability, (2) correct semantic partitioning. That
is to say, if a particular partition is unavailable you have no choice but
to gi
Thanks for your response. I think the work-around is not really acceptable
for me since it will consume 3x the resources (because replication of 3 is
the minimum acceptable) and it will still make the cluster less available
anyway (unless i have only 3 brokers).
The thing is that 0.7 was making th
Maxime,
First of all, in 0.8, you can choose to have a replication factor of 2. It
just means that one can tolerate only one broker failure.
Second, our producer logic supports retries on failure. If a message can't
be sent, on retry, our default partitioner will select another random
partition t
But I think Maxime's point is valid. In the common case that you don't care
about every single message and are willing to tolerate a little loss 0.8
will seem like a pretty big step back.
I don't think the solution of just randomly partitioning works because you
will still produce 1/nth of your da
Yup, runs nightly with about 100 test scenarios. Code is in the main repo
under system_test/.
-Jay
On Tue, Jan 8, 2013 at 6:47 PM, S Ahmed wrote:
> I was just reading this:
> https://cwiki.apache.org/confluence/display/KAFKA/Performance+testing
>
> Sounds like a solid idea! Has this been impl
Thanks. Using a single producer is an interesting option. It it would limit
the compression bandwidth to single core and is would be issue a producer
handles high volumes. Plus, we just have 10 kafka servers and even 10% is
still quite large.
It may not be that hard to do a little better that curr
I have been noticing a lot of cool Kafka integrations floating around. I
took some time and went through github and emails and tried to update the
some of the pointers we have. S
Powered By - If you are using Kafka, add yourself here.
https://cwiki.apache.org/confluence/display/KAFKA/Powered
