Kafka Streams window retention period question

2017-01-04 Thread Alexander Demidko
Hi folks, I'm experimenting with Kafka Streams windowed aggregation and came across window retention period behavior I don't fully understand. I'm using custom timestamp extractor which gets the timestamp from the payload. Values are aggregated using tumbling time windows and summed by the key. I

Re: Kafka Streams window retention period question

2017-01-05 Thread Alexander Demidko
er) and thus, the next b-record with TS=15 > seconds will be added the the still open (first) b-window and both > values 2 and 10 get added to 12. > > > Also keep in mind that we do some deduplication on KTable result using > an internal cache. This can also influence what output re

Re: Kafka Streams window retention period question

2017-01-05 Thread Alexander Demidko
further notice. > > About stream time and partition: yes. And how time is advanced/tracked > in independent for the window type. > > > -Matthias > > On 1/5/17 9:14 PM, Alexander Demidko wrote: > > Hi Matthias, > > > > Thanks for such a thorough response! &g

Re: KafkaException: This operation cannot be completed on a complete request

2014-04-18 Thread Alexander Demidko
Have tried to reproduce this error, and it occurs pretty consistently when node being forcefully shutdown w/o graceful termination. When graceful shutdown was successful no errors occur in a log when the instance was rebooted starts. On Fri, Apr 18, 2014 at 11:17 AM, Alex Demidko wrote: > These