[
https://issues.apache.org/jira/browse/FLINK-30530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677432#comment-17677432
]
Arseniy Tashoyan edited comment on FLINK-30530 at 1/19/23 9:16 AM:
-------------------------------------------------------------------
Helm chart is a complementary tool, but not a solution. For a Flink deployment
we have a 200-lines template file. This file still contains 3-in-1:
flink-conf.yaml, log4j.properties and the YAML-descriptor.
Instead we would rather have 2 smaller templates (flink-conf.yaml,
YAML-descriptor) and 1 simple file (log4j.properties).
{*}UPDATE{*}: The configuration for logging in JSON format is even longer. This
makes the template file huge.
was (Author: tashoyan):
Helm chart is a complementary tool, but not a solution. For a Flink deployment
we have a 200-lines template file. This file still contains 3-in-1:
flink-conf.yaml, log4j.properties and the YAML-descriptor.
Instead we would rather have 2 smaller templates (flink-conf.yaml,
YAML-descriptor) and 1 simple file (log4j.properties).
> Flink configuration from user-provided ConfigMap
> ------------------------------------------------
>
> Key: FLINK-30530
> URL: https://issues.apache.org/jira/browse/FLINK-30530
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Environment: Flink 1.15.2
> Flink Kubernetes operator 1.2.0
> Reporter: Arseniy Tashoyan
> Priority: Major
>
> Currently the Flink configuration can be specified in the YAML descriptor of
> FlinkDeployment via the _flinkConfiguration_ setting:
> {code:yaml}
> flinkConfiguration:
> taskmanager.numberOfTaskSlots: "2"
> ...
> {code}
> Same for the logging configuration:
> {code:yaml}
> logConfiguration:
> "log4j-console.properties": |
> rootLogger.level = DEBUG
> ...{code}
> This makes the YAML descriptor overloaded and huge. In addition, Flink and
> logging configuration may differ for different applications, while the
> Kubernetes settings maybe same for all applications. Therefore it makes sense
> to extract Flink and logging configurations from the YAML descriptor.
> This can be done via a user-provided ConfigMap:
> {code:yaml}
> flinkConfigMap: basic-example-flink-config
> {code}
> In this example we have a Flink application {_}basic-example{_}. The
> _basic-example-flink-config_ ConfigMap contains all config files used by
> Flink: flink-conf.yaml, log4j-console.properties, possibly other files. The
> content of this ConfigMap gets mounted as a volume to {_}/opt/flink/conf{_}.
> Therefore we can have different Flink settings for different applications and
> the same YAML descriptor for all of them (only the value for _flinkConfigMap_
> differs).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
