I am receiving this exception lot more frequently, also this occurs generally at the start of starting the streams app. I was also running 3 seperate instances of my streams app all with different state stores on 3 different machines.
I am using Kafka 11.0.1. -Sameer. On Tue, Oct 3, 2017 at 12:03 PM, Sameer Kumar <sam.kum.w...@gmail.com> wrote: > I had enabled eos through streams config and as explained in the > documentation, I have not added anything else other than following config. > > streamsConfiguration.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG, > StreamsConfig.EXACTLY_ONCE); > > As explained by you, I think producer idempotence and retries would be > automatically picked up. I was thinking if there is someway i can print the > configs picked up the streams app for e.g. the current retries. > > Do i need to add any other configuration. > > -Sameer. > > On Tue, Oct 3, 2017 at 11:59 AM, Sameer Kumar <sam.kum.w...@gmail.com> > wrote: > >> I had enabled eos through streams config. >> >> >> On Fri, Sep 29, 2017 at 11:12 PM, Matthias J. Sax <matth...@confluent.io> >> wrote: >> >>> That's correct: If EOS is enabled, we enforce some producer configs: >>> >>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/ma >>> in/java/org/apache/kafka/streams/StreamsConfig.java#L678-L688 >>> >>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/ma >>> in/java/org/apache/kafka/streams/StreamsConfig.java#L691 >>> >>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/ma >>> in/java/org/apache/kafka/streams/StreamsConfig.java#L493-L496 >>> >>> >>> Note, that by default we set retries to Integer.MAX_VALUE but we do not >>> enforce this setting (as pointed out by Damian already). So you could >>> overwrite it with a smaller value (what is of course not recommended). >>> >>> I was not sure though, if you enabled EOS or just enabled idempotency >>> only for the producer -- what you can easily do by providing the >>> corresponding producer configs. >>> >>> If you did enable EOS in StreamsConfig, Producer will take care of >>> OutOfOrderSequenceException in general. >>> >>> However, there are scenarios for which Producer cannot handle it. For >>> those case, OutOfOrderSequenceException indicates that there has been >>> data loss on the broker, ie, a previously acknowledged message no longer >>> exists. For most part, this should only occur in rare situations >>> (simultaneous power outages, multiple disk losses, software bugs >>> resulting in data corruption, etc.). >>> >>> >>> -Matthias >>> >>> On 9/29/17 7:55 AM, Damian Guy wrote: >>> > You can set ProducerConfig.RETRIES_CONFIG in your StreamsConfig, i.e, >>> > >>> > Properties props = new Properties(); >>> > props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE); >>> > ... >>> > >>> > On Fri, 29 Sep 2017 at 13:17 Sameer Kumar <sam.kum.w...@gmail.com> >>> wrote: >>> > >>> >> I guess once stream app are enabled exactly-once, producer >>> idempotence get >>> >> enabled by default and so do the retries. I guess producer retries are >>> >> managed internally and not exposed through streamconfig. >>> >> >>> >> https://kafka.apache.org/0110/documentation/#streamsconfigs >>> >> >>> >> -Sameer. >>> >> >>> >> On Thu, Sep 28, 2017 at 12:12 AM, Matthias J. Sax < >>> matth...@confluent.io> >>> >> wrote: >>> >> >>> >>> An OutOfOrderSequenceException should only occur if a idempotent >>> >>> producer gets out of sync with the broker. If you set >>> >>> `enable.idempotence = true` on your producer, you might want to set >>> >>> `retries = Integer.MAX_VALUE`. >>> >>> >>> >>> -Matthias >>> >>> >>> >>> On 9/26/17 11:30 PM, Sameer Kumar wrote: >>> >>>> Hi, >>> >>>> >>> >>>> I again received this exception while running my streams app. I am >>> >> using >>> >>>> Kafka 11.0.1. After restarting my app, this error got fixed. >>> >>>> >>> >>>> I guess this might be due to bad network. Any pointers. Any config >>> >>>> wherein I can configure it for retries. >>> >>>> >>> >>>> Exception trace is attached. >>> >>>> >>> >>>> Regards, >>> >>>> -Sameer. >>> >>> >>> >>> >>> >> >>> > >>> >>> >> >
