Whey does the window final result is not emitted after the window has elapsed?

2018-12-30 Thread jingguo yao
I followed [1] to code a simple example to try suppress operator. Here is the simple code: final Serde stringSerde = Serdes.String(); final StreamsBuilder builder = new StreamsBuilder(); builder.stream("TextLinesTopic", Consumed.with(Serdes.String(), Serdes.String())) .flatMapValues(value -> Ar

Kafka Streams 2.1.0, 3rd time data lose investigation

2018-12-30 Thread Nitay Kufert
Hey everybody, We are running Kafka streams in production for the last year or so - we currently using the latest version (2.1.0) and we suffered from data lose several times before. The first time we noticed a data loss, we were able to trace it back to Exception that we were getting in the code -

Why did the Kafka stream fail to produce data after a long time?

2018-12-30 Thread JongHyeob Park
I cannot find the reason why KafkaStream is dead. My question is below: - https://stackoverflow.com/questions/53976117/why-did-the-kafka-stream-fail-to-produce-data-after-a-long-time Thank you.

Re: Why do I get an IllegalStateException when accessing record metadata?

2018-12-30 Thread Eric Lalonde
Recently I encountered IllegalStateExceptions in my KafkaTransformer. My application was exhibiting all of the behavior as discussed in the FAQ (https://cwiki.apache.org/confluence/display/KAFKA/FAQ ), under the section: Why do I get an Ill

Re: Whey does the window final result is not emitted after the window has elapsed?

2018-12-30 Thread jingguo yao
Sorry for my typo in the mail. "Whey" should be "Why" in "Whey does the window final result is not emitted after the window has elapsed?" I have browsed the Kafka source code and found the cause of the mentioned behaviour. org.apache.kafka.streams.kstream.internals.suppress.KTableSuppressProcesso