Re: Kafka Streams Daily Aggregation

2019-10-31 Thread Matthias J. Sax
Btw: There is an example implementation of a custom daily window that considers time zones. Maybe it helps: https://github.com/confluentinc/kafka-streams-examples/blob/5.3.1-post/src/test/java/io/confluent/examples/streams/window/DailyTimeWindows.java -Matthias On 10/23/19 10:02 PM, 董宗桢 wrote: >

Re:Re: Kafka Streams Daily Aggregation

2019-10-23 Thread 董宗桢
Hello Boyang, If I start the Kafka Stream process at the middle of a day, say, 10/24 16:00 pm, and with a tumbling window size of 1 day(24 hours). Would the next aggregation run at 10/25 00:00 AM? or at 10/25 16:00 PM? 在 2019-10-24 11:06:24,"Boyang Chen" 写道: >Hey Zongzhen, > >I have im

Re: Kafka Streams Daily Aggregation

2019-10-23 Thread Matthias J. Sax
One issue to consider is timezones thought. Tumbling windows align at timetamp zero, but zero is the start of the day in UTC only. If you are in a different timezone, you would need to "shift" the timestamps accordingly. For example, you can shift them using a custom TimestampExtractor, or you use

Re: Kafka Streams Daily Aggregation

2019-10-23 Thread Boyang Chen
Hey Zongzhen, I have implemented some similar functionality with KStream before. You could just set tumbling window to 24 hours to get daily aggregation result. As you just need calendar dates, the tumbling window computation starts from system time 0 which is exactly cut-off daily. Boyang On We

Kafka Streams Daily Aggregation

2019-10-23 Thread 董宗桢
Hello, I wanna run Kafka Streams on my system to aggregate the users' sales order transactions based on "daily". I know that Kafka Streams provides such mechanisms called tumbling window, but it seems to be just setting an interval to run the aggregation function. What I want is to aggregate b