[
https://issues.apache.org/jira/browse/FLINK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083070#comment-17083070
]
Wenlong Lyu commented on FLINK-17110:
-------------------------------------
[~aljoscha] I mean configuration in StreamExecutionEnvironment, which will be
used to generated the job graph and setup per job cluster I think.
[~kkl0u] I propose to merge the configuration provided by the method of
StreamExecutionEnvironment#configure to
StreamExecutionEnvironment#configuration, so that we can change more
configuration of runtime programmatically.
In my use case, I am trying to create a tool to support running arbitrary sql
jobs in per job cluster mode, which needs to load connector jars according to
the sql dynamically. Currently I have to merge all library jars together, which
will make it inconvenient to introduce a new connector and easy to cause class
conflicts. It would be possible if we can set PipelineOptions.CLASSPATHS
programmatically and store the library jars in some http server, which can be
configured by StreamExecutionEnvironment#configuration.
Another case, it could be also useful to optimize advanced runtime
configuration according to job we are running, such as, set a larger resource
request timeout for a large job. It could be configured in the commandLine too,
but it will be more inconvenient I think.
> Make StreamExecutionEnvironment#configure also affects configuration
> --------------------------------------------------------------------
>
> Key: FLINK-17110
> URL: https://issues.apache.org/jira/browse/FLINK-17110
> Project: Flink
> Issue Type: Improvement
> Reporter: Wenlong Lyu
> Priority: Major
>
> If StreamExecutionEnvironment#configure can also affect the configuration in
> StreamExecutionEnvironment, we can easily not only add some library jars or
> classpaths dynamically according to the job we want to run which is quite
> important for a platform product, but also optimize some runtime
> configuration in program.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
