I had enabled eos through streams config and as explained in the
documentation, I have not added anything else other than following config.

  streamsConfiguration.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
StreamsConfig.EXACTLY_ONCE);

As explained by you, I think producer idempotence and retries would be
automatically picked up. I was thinking if there is someway i can print the
configs picked up the streams app for e.g. the current retries.

Do i need to add any other configuration.

-Sameer.

On Tue, Oct 3, 2017 at 11:59 AM, Sameer Kumar <sam.kum.w...@gmail.com>
wrote:

> I had enabled eos through streams config.
>
>
> On Fri, Sep 29, 2017 at 11:12 PM, Matthias J. Sax <matth...@confluent.io>
> wrote:
>
>> That's correct: If EOS is enabled, we enforce some producer configs:
>>
>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/
>> main/java/org/apache/kafka/streams/StreamsConfig.java#L678-L688
>>
>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/
>> main/java/org/apache/kafka/streams/StreamsConfig.java#L691
>>
>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/
>> main/java/org/apache/kafka/streams/StreamsConfig.java#L493-L496
>>
>>
>> Note, that by default we set retries to Integer.MAX_VALUE but we do not
>> enforce this setting (as pointed out by Damian already). So you could
>> overwrite it with a smaller value (what is of course not recommended).
>>
>> I was not sure though, if you enabled EOS or just enabled idempotency
>> only for the producer -- what you can easily do by providing the
>> corresponding producer configs.
>>
>> If you did enable EOS in StreamsConfig, Producer will take care of
>> OutOfOrderSequenceException in general.
>>
>> However, there are scenarios for which Producer cannot handle it. For
>> those case, OutOfOrderSequenceException indicates that there has been
>> data loss on the broker, ie, a previously acknowledged message no longer
>> exists. For most part, this should only occur in rare situations
>> (simultaneous power outages, multiple disk losses, software bugs
>> resulting in data corruption, etc.).
>>
>>
>> -Matthias
>>
>> On 9/29/17 7:55 AM, Damian Guy wrote:
>> > You can set ProducerConfig.RETRIES_CONFIG in your StreamsConfig, i.e,
>> >
>> > Properties props = new Properties();
>> > props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE);
>> > ...
>> >
>> > On Fri, 29 Sep 2017 at 13:17 Sameer Kumar <sam.kum.w...@gmail.com>
>> wrote:
>> >
>> >> I guess once stream app are enabled exactly-once, producer idempotence
>> get
>> >> enabled by default and so do the retries. I guess producer retries are
>> >> managed internally and not exposed through streamconfig.
>> >>
>> >> https://kafka.apache.org/0110/documentation/#streamsconfigs
>> >>
>> >> -Sameer.
>> >>
>> >> On Thu, Sep 28, 2017 at 12:12 AM, Matthias J. Sax <
>> matth...@confluent.io>
>> >> wrote:
>> >>
>> >>> An OutOfOrderSequenceException should only occur if a idempotent
>> >>> producer gets out of sync with the broker. If you set
>> >>> `enable.idempotence = true` on your producer, you might want to set
>> >>> `retries = Integer.MAX_VALUE`.
>> >>>
>> >>> -Matthias
>> >>>
>> >>> On 9/26/17 11:30 PM, Sameer Kumar wrote:
>> >>>> Hi,
>> >>>>
>> >>>> I again received this exception while running my streams app. I am
>> >> using
>> >>>> Kafka 11.0.1. After restarting my app, this error got fixed.
>> >>>>
>> >>>> I guess this might be due to bad network. Any pointers. Any config
>> >>>> wherein I can configure it for retries.
>> >>>>
>> >>>> Exception trace is attached.
>> >>>>
>> >>>> Regards,
>> >>>> -Sameer.
>> >>>
>> >>>
>> >>
>> >
>>
>>
>

Reply via email to