MAX_INT is a good value if you want to just block until the buffer has some space (and never get an exception).
On Tue, Feb 2, 2016 at 8:08 AM, Franco Giacosa <fgiac...@gmail.com> wrote: > Thanks for the information James, the slides are really good. > > One question, in the new producer the property block.on.buffer.full (in the > slides put this value in TRUE is a good practice, I image that this will > avoid a buffer overflow) is deprecated, and instead the use of > max.block.ms, > which block for X amount of ms, can someone tell me what a good value will > be for this property in order to mimic the behaviour of > block.on.buffer.full? > > Thanks > > 2016-01-31 6:09 GMT+01:00 James Cheng <jch...@tivo.com>: > > > > > > On Jan 30, 2016, at 4:21 AM, Franco Giacosa <fgiac...@gmail.com> > wrote: > > > > > > Sorry, this solved my questions: "Setting a value greater than zero > will > > > cause the client to resend any record whose send fails with a > potentially > > > transient error. Note that this retry is no different than if the > client > > > resent the record upon receiving the error. Allowing retries will > > > potentially change the ordering of records because if two records are > > sent > > > to a single partition, and the first fails and is retried but the > second > > > succeeds, then the second record may appear first." > > > > > > > Franco, > > > > Also, you can avoid the message reordering issue in that description by > > setting max.in.flight.requests.per.connector to 1. > > > > This slide deck has good guidelines on the types of things you are > talking > > about: > > > > > http://www.slideshare.net/JiangjieQin/no-data-loss-pipeline-with-apache-kafka-49753844 > > > > -James > > > > > 2016-01-30 13:18 GMT+01:00 Franco Giacosa <fgiac...@gmail.com>: > > > > > >> Hi, > > >> > > >> The at-least-once delivery is generated in part by the network fails > and > > >> the retries (that may generate duplicates) right? > > >> > > >> In the event of a duplicated (there was an error but the first message > > >> landed ok on the partition P1) the producer will recalculate the > > partition > > >> on the retry? is this done automatically? > > >> > > >> If in the retry the partition doesn't change and there is only 1 > > Producer, > > >> will the duplicated be written next to the original? I mean if I > poll() > > >> they will come one after the other? > > >> > > >> > > >> > > > > > > ________________________________ > > > > This email and any attachments may contain confidential and privileged > > material for the sole use of the intended recipient. Any review, copying, > > or distribution of this email (or any attachments) by others is > prohibited. > > If you are not the intended recipient, please contact the sender > > immediately and permanently delete this email and any attachments. No > > employee or agent of TiVo Inc. is authorized to conclude any binding > > agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo > > Inc. may only be made by a signed written agreement. > > >
