That's correct: If EOS is enabled, we enforce some producer configs:

https://github.com/apache/kafka/blob/0.11.0.1/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java#L678-L688

https://github.com/apache/kafka/blob/0.11.0.1/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java#L691

https://github.com/apache/kafka/blob/0.11.0.1/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java#L493-L496


Note, that by default we set retries to Integer.MAX_VALUE but we do not
enforce this setting (as pointed out by Damian already). So you could
overwrite it with a smaller value (what is of course not recommended).

I was not sure though, if you enabled EOS or just enabled idempotency
only for the producer -- what you can easily do by providing the
corresponding producer configs.

If you did enable EOS in StreamsConfig, Producer will take care of
OutOfOrderSequenceException in general.

However, there are scenarios for which Producer cannot handle it. For
those case, OutOfOrderSequenceException indicates that there has been
data loss on the broker, ie, a previously acknowledged message no longer
exists. For most part, this should only occur in rare situations
(simultaneous power outages, multiple disk losses, software bugs
resulting in data corruption, etc.).


-Matthias

On 9/29/17 7:55 AM, Damian Guy wrote:
> You can set ProducerConfig.RETRIES_CONFIG in your StreamsConfig, i.e,
> 
> Properties props = new Properties();
> props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE);
> ...
> 
> On Fri, 29 Sep 2017 at 13:17 Sameer Kumar <sam.kum.w...@gmail.com> wrote:
> 
>> I guess once stream app are enabled exactly-once, producer idempotence get
>> enabled by default and so do the retries. I guess producer retries are
>> managed internally and not exposed through streamconfig.
>>
>> https://kafka.apache.org/0110/documentation/#streamsconfigs
>>
>> -Sameer.
>>
>> On Thu, Sep 28, 2017 at 12:12 AM, Matthias J. Sax <matth...@confluent.io>
>> wrote:
>>
>>> An OutOfOrderSequenceException should only occur if a idempotent
>>> producer gets out of sync with the broker. If you set
>>> `enable.idempotence = true` on your producer, you might want to set
>>> `retries = Integer.MAX_VALUE`.
>>>
>>> -Matthias
>>>
>>> On 9/26/17 11:30 PM, Sameer Kumar wrote:
>>>> Hi,
>>>>
>>>> I again received this exception while running my streams app. I am
>> using
>>>> Kafka 11.0.1. After restarting my app, this error got fixed.
>>>>
>>>> I guess this might be due to bad network. Any pointers. Any config
>>>> wherein I can configure it for retries.
>>>>
>>>> Exception trace is attached.
>>>>
>>>> Regards,
>>>> -Sameer.
>>>
>>>
>>
> 

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to