Hi Kane,
If the producer is async, the send(requests) function call would not
necessarily trigger the real sending action. The sending action is
triggered either if enough time has elapsed or enough messages have been
batched on the client side. One batch of messages to each broker will be
either send successfully or not at all (in this sense "atomic"), and if
failed the whole batch will be re-tried to that broker again.
As for consumers (I am assuming high-level consumers here), since the
consumers themselves will need to remember the offsets still which they
have consumed, if the consume request fails the consumers will just
re-issue the request starting with the previous offsets again.
Guozhang
On Wed, Oct 16, 2013 at 8:56 AM, Kane Kane <kane.ist...@gmail.com> wrote:
> Hello, as I understand send is not atomic, i.e. i have something like this
> in my code:
>
> val requests = new ArrayBuffer[KeyedMessage[AnyRef, AnyRef]]
> for (message <- messages) {
> requests += new KeyedMessage(topic, null, message, message)
> }
> producer.send(requests)
>
> That means batch can die in the middle?
> Also what happens if during this write the broker that has some partition
> leaders will die? The same question for consumer, what happens if broker
> dies while consumer reading from it?
>
> Thanks.
>
--
-- Guozhang
