Re: Memory Management in Streaming mode

2016-08-29 Thread Robert Metzger
Hi Shen, What you've read in the documentation is correct. The managed memory is only used by the batch operators. For streaming, we have different state backends with different characteristics. The FsStateBackend keeps the state (including window contents) on the heap and makes backups to the (di

Memory Management in Streaming mode

2016-08-29 Thread Shen Li
Hi, I have a question regarding memory management in the Streaming mode. The documents state that the memory management module is only used in the Batch mode, and Flink Streaming directly operates on JVM heap. Then, what if the volume of data in a window becomes too large to fit in the heap? Will