Hi Caizhi, Thanks for responding. >So you'd like to flatten the traffic by materializing the results of different parallelisms at different times? Yes.
> What's your use case for streaming windows? In short, summarizing many-many-many millions of sessions every minute involving mostly stateless, but a few stateful metrics. Meaning the metric depends on some trait of the sessions that has been seen. However, state management is fairly minimal. > Could you please elaborate more on the reason you'd like to flatten the traffic? There's hints of performance issues and effects on the Kafka cluster, which I anticipate only get worse as similar, but different apps are developed for different parts of the org. The issues are mostly hidden by over-provisioning at the moment. > different times is the session window [1]. Does that meet your needs? I played around with this a little. However it's not realistic to use this window because the memory requirements would be enormous in our case. I did an experiment with using the session window in combination with the continuous and purging trigger. However that resulted in the session timeout window not being evaluated because well the events had been purged. And we do need that timeout as well. So in short I believe I'm looking for a session-aware unaligned tumbling window. On Sun, Jan 16, 2022 at 9:10 PM Caizhi Weng <tsreape...@gmail.com> wrote: > Hi! > > So you'd like to flatten the traffic by materializing the results of > different parallelisms at different times? > > As far as I know this is not possible. Could you please elaborate more on > the reason you'd like to flatten the traffic? Is it causing any problems > with your job? What's your use case for streaming windows? > > Also the only type of window capable of emitting results for different > parallelisms at different times is the session window [1]. Does that meet > your needs? > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/windows/#session-windows > > Aeden Jameson <aeden.jame...@gmail.com> 于2022年1月15日周六 02:16写道： > >> When using tumbling windows the windows materialize all at once which >> results in burst-y traffic. How does one go about unaligned tumbling >> windows? Does this require going down the road of custom window, assigner >> and triggers? >> >> -- >> Cheers, >> Aeden >> >> >> -- Cheers, Aeden GitHub: https://github.com/aedenj Linked In: http://www.linkedin.com/in/aedenjameson Blah Blah Blah: http://www.twitter.com/daliful
