[
https://issues.apache.org/jira/browse/FLINK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018645#comment-17018645
]
Rong Rong edited comment on FLINK-15447 at 1/18/20 5:01 PM:
------------------------------------------------------------
Ahh. I see your intention. Let me clarify what I understood. please correct me
if I were wrong:
* Both Flink-YARN (JM/TM) and some other 3rd party uses {{java.io.tmpdir}}
which is a JVM system env key.
* This means, potentially, whatever directory configured, all Java process will
be sharing this directory as tmp folder.
So could you clarify which is the main concern:
1. The default key is set to {{/tmp}} - which potentially will be used also by
NON-JVM process.
2. In addition, we also want Flink JM/TM to NOT share with others JVM process
or other YARN containers.
If the above analysis is correct,
For #1, we actually creates dedicate partitions to put {{/tmp}} in our YARN
node, which resolves the issue. Not sure if this can be a solution on your
case.
For #2, yes I think the question is not easy to answer especially we want
fine-grain control on disk resource.
was (Author: rongr):
Ahh. I see your intention. Let me clarify what I understood. please correct me
if I were wrong:
* Both Flink-YARN (JM/TM) and some other 3rd party uses {{java.io.tmpdir}}
which is a JVM system env key.
* This means, potentially, whatever directory configured via system env, all
Java process will be sharing this location.
So could you clarify which is the main concern:
1. The default key is set to {{/tmp}} - which potentially will be used also by
NON-JVM process.
2. In addition, we also want Flink JM/TM to NOT share with others JVM process
or other YARN containers.
If the above analysis is correct,
For #1, we actually creates dedicate partitions to put {{/tmp}} in our YARN
node, which resolves the issue. Not sure if this can be a solution on your
case.
For #2, yes I think the question is not easy to answer especially we want
fine-grain control on disk resource.
> Change "java.io.tmpdir" of JM/TM on Yarn to "{{PWD}}/tmp"
> -----------------------------------------------------------
>
> Key: FLINK-15447
> URL: https://issues.apache.org/jira/browse/FLINK-15447
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Affects Versions: 1.9.1
> Reporter: Victor Wong
> Priority: Major
>
> Currently, when running Flink on Yarn, the "java.io.tmpdir" property is set
> to the default value, which is "/tmp".
>
> Sometimes we ran into exceptions caused by a full "/tmp" directory, which
> would not be cleaned automatically after applications finished.
> I think we can set "java.io.tmpdir" to "{{{{PWD}}}}/tmp" directory, or
> something similar. "{{{{PWD}}}}" will be replaced with the true working
> directory of JM/TM by Yarn, which will be cleaned automatically.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
