Thank you a lot.
How requests in max.in.flight.requests.per.connection relates to batches? 1 request precisely means 1 batch? It would make sense if I think about it. Just to ensure I understand correctly. Petr From: Michal Borowiecki [mailto:michal.borowie...@openbet.com] Sent: 30. dubna 2017 20:51 To: users@kafka.apache.org Subject: Re: Does Kafka producer waits till previous batch returns responce before sending next one? Yes, that's what the docs say in both places: max.in.flight.requests.per.connection The maximum number of unacknowledged requests the client will send on a single connection before blocking. Note that if this setting is set to be greater than 1 and there are failed sends, there is a risk of message re-ordering due to retries (i.e., if retries are enabled). retries Setting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error. Note that this retry is no different than if the client resent the record upon receiving the error. Allowing retries without setting max.in.flight.requests.per.connection to 1 will potentially change the ordering of records because if two batches are sent to a single partition, and the first fails and is retried but the second succeeds, then the records in the second batch may appear first. Cheers, Michał On 30/04/17 19:32, Jun MA wrote: Does this mean that if the client have retry > 0 and max.in.flight.requests.per.connection > 1, then even if the topic only have one partition, there’s still no guarantee of the ordering? Thanks, Jun On Apr 30, 2017, at 7:57 AM, Hans Jespersen <mailto:h...@confluent.io> <h...@confluent.io> wrote: There is a parameter that controls this behavior called max.in. flight.requests.per.connection If you set max.in. flight.requests.per.connection = 1 then the producer waits until previous produce requests returns a response before sending the next one (or retrying). The retries parameter controller the number of times to attempt to produce a batch after a failure. If flight.requests.per.connection = 1 and retries is get to the maximum then ordering is guaranteed. If there is a timeout then the producer library would try again and again to produce the message and will not skip over to try and produce the next message. If you set flight.requests.per.connection > 1 (I think the default is 5) then you can get a commit log with messages out of order wrt the original published order (because retries are done in parallel rather then in series) -hans On Apr 30, 2017, at 3:13 AM, Petr Novak <mailto:oss.mli...@gmail.com> <oss.mli...@gmail.com> wrote: Hello, Does Kafka producer waits till previous batch returns response before sending next one? Do I assume correctly that it does not when retries can change ordering? Hence batches delay is introduced only by producer internal send loop time and linger? If a timeout would be localized only to a single batch send request for some reason, does it affect the next batch (assuming this batch can go through successfully)? Many thanks, Petr -- <http://www.openbet.com/> Michal Borowiecki Senior Software Engineer L4 T: +44 208 742 1600 +44 203 249 8448 E: michal.borowie...@openbet.com W: www.openbet.com <http://www.openbet.com/> OpenBet Ltd Chiswick Park Building 9 566 Chiswick High Rd London W4 5XT UK <https://www.openbet.com/email_promo> This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmas...@openbet.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by OpenBet for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company registered in England and Wales. Registered no. 3134634. VAT no. GB927523612
