Re: Over Window Aggregation Tuning

2021-08-03 Thread JING ZHANG
Hi Wanghui, Caizhi AFAIK, over window doesn't support mini batch optimization yet. Caizhi Weng 于2021年8月2日周一 上午10:20写道: > Hi! > > As the state grows the processing speed will slow down a bit. Which state > backend are you using? Is mini batch enabled[1]? > > [1] > https://ci.apache.org/projects/f

Re: Over Window Aggregation Tuning

2021-08-01 Thread Wanghui (HiCampus)
1. I use the filesystem as the state backend, and the state should be in memory. 2. The mini-batch function is disabled. 3. Does mini-batch reduce memory usage? I found that the memory usage of the overwindows grew fast and the JVM FunllGC was frequent. Tenured Gen occupies a large amount of me

Re: Over Window Aggregation Tuning

2021-08-01 Thread Caizhi Weng
Hi! As the state grows the processing speed will slow down a bit. Which state backend are you using? Is mini batch enabled[1]? [1] https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/table/config/#table-exec-mini-batch-enabled Wanghui (HiCampus) 于2021年7月30日周五 下午3:59写道: > Hi : > > W

Over Window Aggregation Tuning

2021-07-30 Thread Wanghui (HiCampus)
Hi : When I use Over Window Aggregation,windows size is 1 hour,I find the processing speed decreases over time. How can I tuning the over window? Best regards Hui Wang