Ruguo Yu created FLINK-20682:
--------------------------------
Summary: Add configuration options related to hadoop
Key: FLINK-20682
URL: https://issues.apache.org/jira/browse/FLINK-20682
Project: Flink
Issue Type: Improvement
Components: Deployment / YARN
Affects Versions: 1.12.0
Reporter: Ruguo Yu
Fix For: 1.13.0, 1.12.1
{{Current, we submit flink job to yarn with run-application target and need to
specify some configuration related to hadoop, because we use distributed
filesystem similar to Ali oss to storage resources, in this case, we will pass
special configuration option and set them to hadoopConfiguration.}}
{{In order to solve such problems, we can provide a configuration option
prefixed with "flink.hadoop."(such as -Dflink.hadoop.x.y.z), and then take it
into H}}{{adoopConfiguration.}}
{{A simple implementation code is as follows:}}{{}}
{code:java}
module: flink-filesystems/flink-hadoop-fs
class: org.apache.flink.runtime.util.HadoopUtils
//代码占位符
public static Configuration
getHadoopConfiguration(org.apache.flink.configuration.Configuration
flinkConfiguration) {
......
// Copy any "flink.hadoop.xxx=yyy" flink configuration to hadoop
configuration as "xxx=yyy"
for (String key : flinkConfiguration.keySet()) {
if (key.startsWith("flink.hadoop.")) {
result.set(key.substring("flink.hadoop.".length()),
flinkConfiguration.getString(key, null));
}
}
return result;
}{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
