Thanks for the information James, the slides are really good. One question, in the new producer the property block.on.buffer.full (in the slides put this value in TRUE is a good practice, I image that this will avoid a buffer overflow) is deprecated, and instead the use of max.block.ms, which block for X amount of ms, can someone tell me what a good value will be for this property in order to mimic the behaviour of block.on.buffer.full?
Thanks 2016-01-31 6:09 GMT+01:00 James Cheng <jch...@tivo.com>: > > > On Jan 30, 2016, at 4:21 AM, Franco Giacosa <fgiac...@gmail.com> wrote: > > > > Sorry, this solved my questions: "Setting a value greater than zero will > > cause the client to resend any record whose send fails with a potentially > > transient error. Note that this retry is no different than if the client > > resent the record upon receiving the error. Allowing retries will > > potentially change the ordering of records because if two records are > sent > > to a single partition, and the first fails and is retried but the second > > succeeds, then the second record may appear first." > > > > Franco, > > Also, you can avoid the message reordering issue in that description by > setting max.in.flight.requests.per.connector to 1. > > This slide deck has good guidelines on the types of things you are talking > about: > > http://www.slideshare.net/JiangjieQin/no-data-loss-pipeline-with-apache-kafka-49753844 > > -James > > > 2016-01-30 13:18 GMT+01:00 Franco Giacosa <fgiac...@gmail.com>: > > > >> Hi, > >> > >> The at-least-once delivery is generated in part by the network fails and > >> the retries (that may generate duplicates) right? > >> > >> In the event of a duplicated (there was an error but the first message > >> landed ok on the partition P1) the producer will recalculate the > partition > >> on the retry? is this done automatically? > >> > >> If in the retry the partition doesn't change and there is only 1 > Producer, > >> will the duplicated be written next to the original? I mean if I poll() > >> they will come one after the other? > >> > >> > >> > > > ________________________________ > > This email and any attachments may contain confidential and privileged > material for the sole use of the intended recipient. Any review, copying, > or distribution of this email (or any attachments) by others is prohibited. > If you are not the intended recipient, please contact the sender > immediately and permanently delete this email and any attachments. No > employee or agent of TiVo Inc. is authorized to conclude any binding > agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo > Inc. may only be made by a signed written agreement. >
