But I think Maxime's point is valid. In the common case that you don't care
about every single message and are willing to tolerate a little loss 0.8
will seem like a pretty big step back.
I don't think the solution of just randomly partitioning works because you
will still produce 1/nth of your da
Maxime,
First of all, in 0.8, you can choose to have a replication factor of 2. It
just means that one can tolerate only one broker failure.
Second, our producer logic supports retries on failure. If a message can't
be sent, on retry, our default partitioner will select another random
partition t
Thanks for your response. I think the work-around is not really acceptable
for me since it will consume 3x the resources (because replication of 3 is
the minimum acceptable) and it will still make the cluster less available
anyway (unless i have only 3 brokers).
The thing is that 0.7 was making th
This is a good point. We have discussed this a little bit before. The key
constraint is that with replication factor 1 you can choose one of the
following: (1) high availability, (2) correct semantic partitioning. That
is to say, if a particular partition is unavailable you have no choice but
to gi
Hello, I am currently testing the 0.8 branch (and it works quite well). We
plan to not use the replication feature for now since we don't really need
it, we can afford to lose data in case of unrecoverable failure from a
broker.
However, we really don't want to have producers/consumers fail if a b
