Re: Triggers for windowed aggregations in Table API

2021-09-06 Thread Guowei Ma
Hi, John If you want to trigger the aggregation calculation of the window “earlier” I think you might need to define your customized window trigger in DataStream yourself. I think you could find more detailed information from the doc [1] If you think that the customize the trigger is a little har

Re: Triggers for windowed aggregations in Table API

2021-09-03 Thread John Smith
Thanks Guowei and Caizhi. As Guowei noted, I am using Table API and it seems that it does not support triggers at the moment. Is there a plan to support custom triggers in Table API/SQL too? Also, if I follow Guowei's suggestion, should I use DataStream for other parts of the aggregate computation

Re: Triggers for windowed aggregations in Table API

2021-09-02 Thread Guowei Ma
Hi, John I agree with Caizhi that you might need to customize a window trigger. But there is a small addition, you need to convert Table to DataStream first. Then you can customize the trigger of the window. Because as far as I know, Table API does not support custom windows yet. For details on ho

Re: Triggers for windowed aggregations in Table API

2021-09-02 Thread Caizhi Weng
Hi! You might want to use your custom trigger to achieve this. Tumble windows are using EventTimeTrigger by default. Flink has another built-in trigger called CountTrigger but it only fires for every X records, ignoring the event time completely. You might want to create your own trigger to combi

Triggers for windowed aggregations in Table API

2021-09-02 Thread John Smith
Hi, Sorry if this has been answered previously but I couldn't find any answer for the question and would appreciate any help. Context: Let's say I have a log stream in Kafka where message values have an *id* field along with a few other fields. I want to count the number of messages for each id fo