Re: Insufficient number of network buffers for simple last_value aggregate

2020-10-30 Thread Arvid Heise
Hi Thilo, the number of required network buffers depends on your data exchanges and parallelism. For each shuffling data exchange (what you need for group by), you ideally have #slots-per-TM^2 * #TMs * 4 buffers. So I'm assuming you have 11 machines and 8 slots per machine. Then for best performa

Re: Insufficient number of network buffers for simple last_value aggregate

2020-10-30 Thread Xintong Song
Hi Schneider, The error message suggests that your task managers are not configured with enough network memory. You would need to increase the network memory configuration. See this doc [1] for more details. Thank you~ Xintong Song [1] https://ci.apache.org/projects/flink/flink-docs-release-1.

Insufficient number of network buffers for simple last_value aggregate

2020-10-29 Thread Schneider, Thilo
Dear list, when trying to compute a simple last_value aggregate, flink fails with an IOException. The query is defined as follows: from pyflink.table import EnvironmentSettings, StreamTableEnvironment env_settings = EnvironmentSettings.new_instance().in_streaming_mode().use_blink_planner().buil