[
https://issues.apache.org/jira/browse/FLINK-10951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696266#comment-16696266
]
ASF GitHub Bot commented on FLINK-10951:
----------------------------------------
GJL closed pull request #7149: [FLINK-10951][tests] Set
yarn.nodemanager.vmem-check-enabled to false
URL: https://github.com/apache/flink/pull/7149
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/flink-jepsen/src/jepsen/flink/hadoop.clj
b/flink-jepsen/src/jepsen/flink/hadoop.clj
index f633d07d621..99f3f72d45c 100644
--- a/flink-jepsen/src/jepsen/flink/hadoop.clj
+++ b/flink-jepsen/src/jepsen/flink/hadoop.clj
@@ -40,11 +40,14 @@
(defn yarn-site-config
[test]
- {:yarn.resourcemanager.hostname (resource-manager (:nodes test))
- :yarn.log-aggregation-enable "true"
+ {:yarn.log-aggregation-enable "true"
+
+ :yarn.nodemanager.log-dirs yarn-log-dir
:yarn.nodemanager.resource.cpu-vcores "8"
+ :yarn.nodemanager.vmem-check-enabled "false"
+
:yarn.resourcemanager.am.max-attempts "99999"
- :yarn.nodemanager.log-dirs yarn-log-dir})
+ :yarn.resourcemanager.hostname (resource-manager (:nodes test))})
(defn core-site-config
[test]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Disable enforcing of YARN container virtual memory limits in tests
> ------------------------------------------------------------------
>
> Key: FLINK-10951
> URL: https://issues.apache.org/jira/browse/FLINK-10951
> Project: Flink
> Issue Type: Improvement
> Components: Tests
> Affects Versions: 1.8.0
> Reporter: Gary Yao
> Assignee: Gary Yao
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.6.3, 1.8.0, 1.7.1
>
>
> *Description*
> The Jepsen YARN tests sporadically fail because TM containers are exceeding
> their virtual memory limits:
> {noformat}
> Closing TaskExecutor connection container_1541436244107_0001_01_000005
> because: Container
> [pid=32403,containerID=container_1541436244107_0001_01_000005] is running
> beyond virtual memory limits. Current usage: 970.2 MB of 2 GB physical memory
> used; 4.2 GB of 4.2 GB virtual memory used. Killing container.
> {noformat}
> By default YARN enforces a virtual memory limit of 2.1 times the requested
> physical memory. However, in my experiments, the virtual memory of a JVM
> process running the {{ClusterEntryPoint}} (without submitting job) is already
> in the region of 3.3 GB. Hence, the virtual memory enforcement should be
> disabled.
> *Acceptance Criteria*
> * {{yarn.nodemanager.vmem-check-enabled}} is {{false}} in {{yarn-site.xml}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
