I had enabled eos through streams config.

On Fri, Sep 29, 2017 at 11:12 PM, Matthias J. Sax <matth...@confluent.io>
wrote:

> That's correct: If EOS is enabled, we enforce some producer configs:
>
> https://github.com/apache/kafka/blob/0.11.0.1/streams/
> src/main/java/org/apache/kafka/streams/StreamsConfig.java#L678-L688
>
> https://github.com/apache/kafka/blob/0.11.0.1/streams/
> src/main/java/org/apache/kafka/streams/StreamsConfig.java#L691
>
> https://github.com/apache/kafka/blob/0.11.0.1/streams/
> src/main/java/org/apache/kafka/streams/StreamsConfig.java#L493-L496
>
>
> Note, that by default we set retries to Integer.MAX_VALUE but we do not
> enforce this setting (as pointed out by Damian already). So you could
> overwrite it with a smaller value (what is of course not recommended).
>
> I was not sure though, if you enabled EOS or just enabled idempotency
> only for the producer -- what you can easily do by providing the
> corresponding producer configs.
>
> If you did enable EOS in StreamsConfig, Producer will take care of
> OutOfOrderSequenceException in general.
>
> However, there are scenarios for which Producer cannot handle it. For
> those case, OutOfOrderSequenceException indicates that there has been
> data loss on the broker, ie, a previously acknowledged message no longer
> exists. For most part, this should only occur in rare situations
> (simultaneous power outages, multiple disk losses, software bugs
> resulting in data corruption, etc.).
>
>
> -Matthias
>
> On 9/29/17 7:55 AM, Damian Guy wrote:
> > You can set ProducerConfig.RETRIES_CONFIG in your StreamsConfig, i.e,
> >
> > Properties props = new Properties();
> > props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE);
> > ...
> >
> > On Fri, 29 Sep 2017 at 13:17 Sameer Kumar <sam.kum.w...@gmail.com>
> wrote:
> >
> >> I guess once stream app are enabled exactly-once, producer idempotence
> get
> >> enabled by default and so do the retries. I guess producer retries are
> >> managed internally and not exposed through streamconfig.
> >>
> >> https://kafka.apache.org/0110/documentation/#streamsconfigs
> >>
> >> -Sameer.
> >>
> >> On Thu, Sep 28, 2017 at 12:12 AM, Matthias J. Sax <
> matth...@confluent.io>
> >> wrote:
> >>
> >>> An OutOfOrderSequenceException should only occur if a idempotent
> >>> producer gets out of sync with the broker. If you set
> >>> `enable.idempotence = true` on your producer, you might want to set
> >>> `retries = Integer.MAX_VALUE`.
> >>>
> >>> -Matthias
> >>>
> >>> On 9/26/17 11:30 PM, Sameer Kumar wrote:
> >>>> Hi,
> >>>>
> >>>> I again received this exception while running my streams app. I am
> >> using
> >>>> Kafka 11.0.1. After restarting my app, this error got fixed.
> >>>>
> >>>> I guess this might be due to bad network. Any pointers. Any config
> >>>> wherein I can configure it for retries.
> >>>>
> >>>> Exception trace is attached.
> >>>>
> >>>> Regards,
> >>>> -Sameer.
> >>>
> >>>
> >>
> >
>
>

Reply via email to