[
https://issues.apache.org/jira/browse/FLINK-21024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17267780#comment-17267780
]
Xintong Song edited comment on FLINK-21024 at 1/19/21, 9:37 AM:
----------------------------------------------------------------
I think the problem is that, `standalone-job.sh` failed to make sure all flink
options come before the job arguments.
{{ClusterEntrypoint}} reads flink options from the beginning of the JVM process
argument list. It stops at the first unrecognized option, and treat the rest as
the job arguments.
This is not reproduced on a standalone session cluster because job arguments
are provided at submitting the job rather than at starting the cluster.
I can try to provide a fix.
was (Author: xintongsong):
I think the problem is that, `standalone-job.sh` failed to make sure all flink
options comes before the job arguments.
{{ClusterEntrypoint}} reads flink options from the beginning of the JVM process
argument list. It stops at the first unrecognized option, and treat the rest as
the job arguments.
This is not reproduced on a standalone session cluster because job arguments
are provided at submitting the job rather than at starting the cluster.
I can try to provide a fix.
> Dynamic properties get exposed to job's main method if user parameters are
> passed
> ---------------------------------------------------------------------------------
>
> Key: FLINK-21024
> URL: https://issues.apache.org/jira/browse/FLINK-21024
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Configuration
> Affects Versions: 1.12.1
> Reporter: Matthias
> Priority: Major
> Labels: starter
> Attachments: WordCount.jar
>
>
> A bug was identified in the [user
> ML|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Application-cluster-standalone-job-some-JVM-Options-added-to-Program-Arguments-td40719.html]
> by Alexey exposing dynamic properties into the job user code.
> I was able to reproduce this issue by slightly adapting the WordCount example
> ({{org.apache.flink.streaming.examples.wordcount.WordCount2}} in attached
> [^WordCount.jar] ).
> Initiating a standalone job without using the {{--input}} parameter will
> result in printing an empty array:
> {code}
> ./bin/standalone-job.sh start --job-classname
> org.apache.flink.streaming.examples.wordcount.WordCount2
> {code}
> The corresponding {{*.out}} file looks like this:
> {code}
> []
> Executing WordCount2 example with default input data set.
> Use --input to specify file input.
> Printing result to stdout. Use --output to specify output path.
> {code}
> In contrast, initiating the standalone job using the {{--input}} parameter
> will expose the dynamic properties:
> {code}
> ./bin/standalone-job.sh start --job-classname
> org.apache.flink.streaming.examples.wordcount.WordCount2 --input
> /opt/flink/config/flink-conf.yaml
> {code}
> Resulting in the following output:
> {code}
> [--input, /opt/flink/config/flink-conf.yaml, -D,
> jobmanager.memory.off-heap.size=134217728b, -D,
> jobmanager.memory.jvm-overhead.min=201326592b, -D,
> jobmanager.memory.jvm-metaspace.size=268435456b, -D,
> jobmanager.memory.heap.size=1073741824b, -D,
> jobmanager.memory.jvm-overhead.max=201326592b]
> Printing result to stdout. Use --output to specify output path.
> {code}
> Interestingly, this cannot be reproduced on a local standalone session
> cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
