I suggest filing a JIRA with as much information as possible (logs, configs, code, etc.) so that we can try to reproduce it.
Ismael On Wed, Oct 4, 2017 at 9:48 AM, Sameer Kumar <sam.kum.w...@gmail.com> wrote: > I am receiving this exception lot more frequently, also this occurs > generally at the start of starting the streams app. I was also running 3 > seperate instances of my streams app all with different state stores on 3 > different machines. > > I am using Kafka 11.0.1. > > -Sameer. > > On Tue, Oct 3, 2017 at 12:03 PM, Sameer Kumar <sam.kum.w...@gmail.com> > wrote: > > > I had enabled eos through streams config and as explained in the > > documentation, I have not added anything else other than following > config. > > > > streamsConfiguration.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG, > > StreamsConfig.EXACTLY_ONCE); > > > > As explained by you, I think producer idempotence and retries would be > > automatically picked up. I was thinking if there is someway i can print > the > > configs picked up the streams app for e.g. the current retries. > > > > Do i need to add any other configuration. > > > > -Sameer. > > > > On Tue, Oct 3, 2017 at 11:59 AM, Sameer Kumar <sam.kum.w...@gmail.com> > > wrote: > > > >> I had enabled eos through streams config. > >> > >> > >> On Fri, Sep 29, 2017 at 11:12 PM, Matthias J. Sax < > matth...@confluent.io> > >> wrote: > >> > >>> That's correct: If EOS is enabled, we enforce some producer configs: > >>> > >>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/ma > >>> in/java/org/apache/kafka/streams/StreamsConfig.java#L678-L688 > >>> > >>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/ma > >>> in/java/org/apache/kafka/streams/StreamsConfig.java#L691 > >>> > >>> https://github.com/apache/kafka/blob/0.11.0.1/streams/src/ma > >>> in/java/org/apache/kafka/streams/StreamsConfig.java#L493-L496 > >>> > >>> > >>> Note, that by default we set retries to Integer.MAX_VALUE but we do not > >>> enforce this setting (as pointed out by Damian already). So you could > >>> overwrite it with a smaller value (what is of course not recommended). > >>> > >>> I was not sure though, if you enabled EOS or just enabled idempotency > >>> only for the producer -- what you can easily do by providing the > >>> corresponding producer configs. > >>> > >>> If you did enable EOS in StreamsConfig, Producer will take care of > >>> OutOfOrderSequenceException in general. > >>> > >>> However, there are scenarios for which Producer cannot handle it. For > >>> those case, OutOfOrderSequenceException indicates that there has been > >>> data loss on the broker, ie, a previously acknowledged message no > longer > >>> exists. For most part, this should only occur in rare situations > >>> (simultaneous power outages, multiple disk losses, software bugs > >>> resulting in data corruption, etc.). > >>> > >>> > >>> -Matthias > >>> > >>> On 9/29/17 7:55 AM, Damian Guy wrote: > >>> > You can set ProducerConfig.RETRIES_CONFIG in your StreamsConfig, i.e, > >>> > > >>> > Properties props = new Properties(); > >>> > props.put(ProducerConfig.RETRIES_CONFIG, Integer.MAX_VALUE); > >>> > ... > >>> > > >>> > On Fri, 29 Sep 2017 at 13:17 Sameer Kumar <sam.kum.w...@gmail.com> > >>> wrote: > >>> > > >>> >> I guess once stream app are enabled exactly-once, producer > >>> idempotence get > >>> >> enabled by default and so do the retries. I guess producer retries > are > >>> >> managed internally and not exposed through streamconfig. > >>> >> > >>> >> https://kafka.apache.org/0110/documentation/#streamsconfigs > >>> >> > >>> >> -Sameer. > >>> >> > >>> >> On Thu, Sep 28, 2017 at 12:12 AM, Matthias J. Sax < > >>> matth...@confluent.io> > >>> >> wrote: > >>> >> > >>> >>> An OutOfOrderSequenceException should only occur if a idempotent > >>> >>> producer gets out of sync with the broker. If you set > >>> >>> `enable.idempotence = true` on your producer, you might want to set > >>> >>> `retries = Integer.MAX_VALUE`. > >>> >>> > >>> >>> -Matthias > >>> >>> > >>> >>> On 9/26/17 11:30 PM, Sameer Kumar wrote: > >>> >>>> Hi, > >>> >>>> > >>> >>>> I again received this exception while running my streams app. I am > >>> >> using > >>> >>>> Kafka 11.0.1. After restarting my app, this error got fixed. > >>> >>>> > >>> >>>> I guess this might be due to bad network. Any pointers. Any config > >>> >>>> wherein I can configure it for retries. > >>> >>>> > >>> >>>> Exception trace is attached. > >>> >>>> > >>> >>>> Regards, > >>> >>>> -Sameer. > >>> >>> > >>> >>> > >>> >> > >>> > > >>> > >>> > >> > > >
