Re: Simulating Time-based and Count-based Custom Windows with ProcessFunction

2018-04-15 Thread Tzu-Li (Gordon) Tai
Hi Max! Before we jump into the custom ProcessFunction approach: Have you also checked out using the RocksDB state backend, and whether or not it is suitable for your use case? For state that would not fit into memory, that is usually the to-go state backend to use. If you’re sure a custom Proc

Simulating Time-based and Count-based Custom Windows with ProcessFunction

2018-04-13 Thread m@xi
Hello Flinkers! Around here and there one may find some post for sliding windows in Flink. I have read that default sliding windows of Flink, the system maintains each window separately in memory, which in my case is prohibitive. Therefore, I want to implement my own sliding windows through Proc