Re: Re: Gradually increasing checkpoint size

2021-03-08 Thread Dan Hill
Hi Yun! That advice was useful. The state for that operator is very small (31kb). Most of the checkpoint size is in a couple simple DataStream.intervalJoin operators. The time intervals are fairly short. I'm going to try running the code with some small configuration changes. One thing I did no

Re: Re: Gradually increasing checkpoint size

2021-03-08 Thread Yun Gao
Hi Dan, Regarding the original checkpoint size problem, could you also have a check which tasks' state are increasing from the checkpoint UI ? For example, the attached operator has a `alreadyOutputed` value state, which seems to keep increasing if there are always new keys ? Best, Yun