[
https://issues.apache.org/jira/browse/FLINK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yiheng tang updated FLINK-35898:
--------------------------------
Description:
We use Yarn to launch Flink jobs and try to add custom parameters to configure
the log output format in the YarnLogConfigUtil.getLog4jCommand method. However,
when the parameter value contains spaces, it causes the startup to fail.
The start command in the Yarn container is as follows, omitting some irrelevant
parameters.
```bash
/bin/bash -c "$JAVA_HOME/bin/java
-Dlog.file='/data08/yarn/userlogs/application_1719804259110_16060/container_e43_1719804259110_16060_01_000001/jobmanager.log'
-Dlog4j.configuration=file.properties
-Dlog4j.configurationFile=file.properties
-Dlog4j2.isThreadContextMapInheritable=true
-Dlog.layout.pattern='%d\{yyyy-MM-dd HH:mm} %-5p %-60c %-60t %x - %m%n %ex'
-Dlog.level=INFO"
```
We discovered that when the value of log.layout.pattern contains spaces, the
command is incorrectly truncated, causing the content after the space (`HH:mm}
%-5p %-60c %-60t %x - %m%n %ex' -Dlog.level=INFO"`) to be unrecognized, and
therefore the Flink job cannot be launched in the Yarn container.
Therefore, we suggest using single quotes (') instead of double quotes (") to
wrap the values of parameters in the `getLogBackCommand` method and
`getLog4jCommand` method of the `YarnLogConfigUtil` class.
!image-2024-07-25-21-52-26-973.png|width=843,height=506!!image-2024-07-25-21-51-50-072.png|width=417,height=218!
was:
We use Yarn to launch Flink jobs and try to add custom parameters to configure
the log output format in the YarnLogConfigUtil.getLog4jCommand method. However,
when the parameter value contains spaces, it causes the startup to fail.
The start command in the Yarn container is as follows, omitting some irrelevant
parameters.
```bash
/bin/bash -c "$JAVA_HOME/bin/java
-Dlog.file='/data08/yarn/userlogs/application_1719804259110_16060/container_e43_1719804259110_16060_01_000001/jobmanager.log'
-Dlog4j.configuration=file.properties
-Dlog4j.configurationFile=file.properties
-Dlog4j2.isThreadContextMapInheritable=true
-Dlog.layout.pattern='%d\{yyyy-MM-dd HH:mm} %-5p %-60c %-60t %x - %m%n %ex'
-Dlog.level=INFO"
```
We discovered that when the value of log.layout.pattern contains spaces, the
command is incorrectly truncated, causing the content after the space (`HH:mm}
%-5p %-60c %-60t %x - %m%n %ex' -Dlog.level=INFO"`) to be unrecognized, and
therefore the Flink job cannot be launched in the Yarn container.
Therefore, we suggest using single quotes (') instead of double quotes (") to
wrap the values of parameters in the `getLogBackCommand` method and
`getLog4jCommand` method of the `YarnLogConfigUtil` class.
> When running Flink on Yarn, if the log file path contains a space, the
> startup will fail.
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-35898
> URL: https://issues.apache.org/jira/browse/FLINK-35898
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.20.0
> Reporter: yiheng tang
> Priority: Minor
> Attachments: image-2024-07-25-21-51-50-072.png,
> image-2024-07-25-21-52-26-973.png
>
>
> We use Yarn to launch Flink jobs and try to add custom parameters to
> configure the log output format in the YarnLogConfigUtil.getLog4jCommand
> method. However, when the parameter value contains spaces, it causes the
> startup to fail.
> The start command in the Yarn container is as follows, omitting some
> irrelevant parameters.
> ```bash
> /bin/bash -c "$JAVA_HOME/bin/java
> -Dlog.file='/data08/yarn/userlogs/application_1719804259110_16060/container_e43_1719804259110_16060_01_000001/jobmanager.log'
> -Dlog4j.configuration=file.properties
> -Dlog4j.configurationFile=file.properties
> -Dlog4j2.isThreadContextMapInheritable=true
> -Dlog.layout.pattern='%d\{yyyy-MM-dd HH:mm} %-5p %-60c %-60t %x - %m%n %ex'
> -Dlog.level=INFO"
> ```
> We discovered that when the value of log.layout.pattern contains spaces, the
> command is incorrectly truncated, causing the content after the space
> (`HH:mm} %-5p %-60c %-60t %x - %m%n %ex' -Dlog.level=INFO"`) to be
> unrecognized, and therefore the Flink job cannot be launched in the Yarn
> container.
> Therefore, we suggest using single quotes (') instead of double quotes (") to
> wrap the values of parameters in the `getLogBackCommand` method and
> `getLog4jCommand` method of the `YarnLogConfigUtil` class.
> !image-2024-07-25-21-52-26-973.png|width=843,height=506!!image-2024-07-25-21-51-50-072.png|width=417,height=218!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
