HI: Generally speaking, when we submitting the flink program, the number of taskmanager and the memory of each tn will be specified. And the smallest real execution unit of flink should be operator. Since the calculation logic corresponding to each operator is different, some need to save the state, and some don't. Therefore, the memory size required by each operator should be different. How does the flink program allocate taskmanager memory to the operator by default? In our production practice, with the increase of traffic, some operators (mainly stateful such as join and groupby) often have insufficient memory, resulting in slower calculations. The usual approach is to increase the entire taskmanager memory. But will this part of the increased memory be allocated to the map-like operator, or that the memory itself is fetched on demand in the same taskmanager whoever needs the memory will fetch it until the memory is used up, in other words, there is no preset memory allocation ratio. For a complex streaming job, is there any way to tilt the memory towards stateful operators?
Thanks.
