azagrebin commented on a change in pull request #10999: [FLINK-15143] Docs for FLIP-49 TM memory model and configuration guide URL: https://github.com/apache/flink/pull/10999#discussion_r374669258
########## File path: docs/ops/memory/mem_setup.md ########## @@ -0,0 +1,252 @@ +--- +title: "Setup Task Executor Memory" +nav-parent_id: ops_mem +nav-pos: 1 +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +Apache Flink provides efficient workloads on top of the JVM by tightly controlling the memory usage of its various components. +While the community strives to offer sensible defaults to all configurations, the full breadth of applications +that users deploy on Flink means this isn't always possible. To provide the most production value to our users, +Flink allows both high level and fine-grained tuning of memory allocation within clusters. + +* toc +{:toc} + +The further described memory configuration is applicable starting with the release version 1.10. If you upgrade Flink +from earlier versions, check the [migration guide](mem_migration.html) because many changes were introduced with the 1.10 release. + +<strong>Note: This memory setup guide is relevant only for Task Executors!</strong> +Check Job Manager [related configuration options](../config.html#jobmanager) for the memory setup of Job Manager. + +## Configure Total Memory + +The *total process memory* of Flink JVM processes consists of memory consumed by Flink application (*total Flink memory*) +and by the JVM to run the process. The *total Flink memory* consumption includes usage of JVM heap, +*managed memory* (managed by Flink) and other direct (or native) memory. + +<center> + <img src="{{ site.baseurl }}/fig/simple_mem_model.svg" width="300px" alt="Simple memory model" usemap="#simple-mem-model"> +</center> +<br /> + +If you run FIink locally (e.g. from your IDE) without creating a cluster, then only a subset of the memory configuration +options are relevant, see also [local execution](#local-execution) for more details. + +Otherwise, the simplest way to setup memory in Flink is to configure either of the two following options: +* Total Flink memory ([taskmanager.memory.flink.size](../config.html#taskmanager-memory-flink-size-1)) Review comment: offline discussion outcome: align with the rest of doc and use back-ticks we can think about more readable config option links as a follow-up in the whole docs ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
