Re: Question regarding Producer and Duplicates

2016-09-05 Thread Florian Hussonnois
Thanks Guozhang, So increasing batch.size can lead to more duplicates in case of failure. Also when you said : "The broker will accept a batch of records as a whole or reject them ". For example, if a producer request contains two batches, a first one for Topic A / Partition 0 and a second one fo

Re: Question regarding Producer and Duplicates

2016-08-31 Thread Guozhang Wang
Hi Florian, The broker will accept a batch of records as a whole or reject them as a whole unless it encounters an IOException while trying to append the messages, which will be treated as a fatal error anyways. Duplicates usually happen when the whole batch is accepted but the ack was not delive