I had enabled eos through streams config.
On Fri, Sep 29, 2017 at 11:12 PM, Matthias J. Sax <matth...@confluent.io> wrote: > That's correct: If EOS is enabled, we enforce some producer configs: > > https://github.com/apache/kafka/blob/0.11.0.1/streams/ > src/main/java/org/apache/kafka/streams/StreamsConfig.java#L678-L688 > > https://github.com/apache/kafka/blob/0.11.0.1/streams/ > src/main/java/org/apache/kafka/streams/StreamsConfig.java#L691 > > https://github.com/apache/kafka/blob/0.11.0.1/streams/ > src/main/java/org/apache/kafka/streams/StreamsConfig.java#L493-L496 > > > Note, that by default we set retries to Integer.MAX_VALUE but we do not > enforce this setting (as pointed out by Damian already). So you could > overwrite it with a smaller value (what is of course not recommended). > > I was not sure though, if you enabled EOS or just enabled idempotency > only for the producer -- what you can easily do by providing the > corresponding producer configs. > > If you did enable EOS in StreamsConfig, Producer will take care of > OutOfOrderSequenceException in general. > > However, there are scenarios for which Producer cannot handle it. For > those case, OutOfOrderSequenceException indicates that there has been > data loss on the broker, ie, a previously acknowledged message no longer > exists. For most part, this should only occur in rare situations > (simultaneous power outages, multiple disk losses, software bugs > resulting in data corruption, etc.). > > > -Matthias > > On 9/29/17 7:55 AM, Damian Guy wrote: > > You can set ProducerConfig.RETRIES_CONFIG in your StreamsConfig, i.e, > > > > Properties props = new Properties(); > > props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE); > > ... > > > > On Fri, 29 Sep 2017 at 13:17 Sameer Kumar <sam.kum.w...@gmail.com> > wrote: > > > >> I guess once stream app are enabled exactly-once, producer idempotence > get > >> enabled by default and so do the retries. I guess producer retries are > >> managed internally and not exposed through streamconfig. > >> > >> https://kafka.apache.org/0110/documentation/#streamsconfigs > >> > >> -Sameer. > >> > >> On Thu, Sep 28, 2017 at 12:12 AM, Matthias J. Sax < > matth...@confluent.io> > >> wrote: > >> > >>> An OutOfOrderSequenceException should only occur if a idempotent > >>> producer gets out of sync with the broker. If you set > >>> `enable.idempotence = true` on your producer, you might want to set > >>> `retries = Integer.MAX_VALUE`. > >>> > >>> -Matthias > >>> > >>> On 9/26/17 11:30 PM, Sameer Kumar wrote: > >>>> Hi, > >>>> > >>>> I again received this exception while running my streams app. I am > >> using > >>>> Kafka 11.0.1. After restarting my app, this error got fixed. > >>>> > >>>> I guess this might be due to bad network. Any pointers. Any config > >>>> wherein I can configure it for retries. > >>>> > >>>> Exception trace is attached. > >>>> > >>>> Regards, > >>>> -Sameer. > >>> > >>> > >> > > > >
