[
https://issues.apache.org/jira/browse/BEAM-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kyle Weaver updated BEAM-10671:
-------------------------------
Description:
The pipeline option --environment_config has completely different usages
depending on the value of --environment_type. This is confusing for the user
and hard to check. Additionally, --environment_config is a JSON blob for
--environment_type=PROCESS. This JSON blob is a pain to escape and pass around
compared to a collection of flat strings.
We should replace --environment_config with first-class / top-level pipeline
options for each environment type:
DOCKER
--environment_container_image
PROCESS
--environment_os
--environment_architecture
--environment_variables
EXTERNAL
--environment_service_address
LOOPBACK
(none)
This way we can validate that the user is configuring these options correctly
(ie give a warning or error if they use options that do not apply to their
chosen --environment_type).
We can deprecate the --environment_config option, logging a warning until
removing this option altogether in a future Beam release.
[https://beam.apache.org/documentation/runtime/sdk-harness-config/]
was:
The pipeline option --environment_config has completely different usages
depending on the value of --environment_type. This is confusing for the user.
Additionally, --environment_config is a JSON blob for
--environment_type=PROCESS. This JSON blob is a huge pain to escape and pass
around compared to a collection of flat strings.
We should replace --environment_config with first-class / top-level pipeline
options for each environment type:
DOCKER
--environment_container_image
PROCESS
--environment_os
--environment_architecture
--environment_variables
EXTERNAL
--environment_service_address
LOOPBACK
(none)
This way we can validate that the user is configuring these options correctly
(ie give a warning or error if they use options that do not apply to their
chosen --environment_type).
We can deprecate the --environment_config option, logging a warning until
removing this option altogether in a future Beam release.
[https://beam.apache.org/documentation/runtime/sdk-harness-config/]
> Add environment configuration fields as first-class pipeline options
> --------------------------------------------------------------------
>
> Key: BEAM-10671
> URL: https://issues.apache.org/jira/browse/BEAM-10671
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-harness
> Reporter: Kyle Weaver
> Assignee: Kyle Weaver
> Priority: P2
>
> The pipeline option --environment_config has completely different usages
> depending on the value of --environment_type. This is confusing for the user
> and hard to check. Additionally, --environment_config is a JSON blob for
> --environment_type=PROCESS. This JSON blob is a pain to escape and pass
> around compared to a collection of flat strings.
> We should replace --environment_config with first-class / top-level pipeline
> options for each environment type:
> DOCKER
> --environment_container_image
> PROCESS
> --environment_os
> --environment_architecture
> --environment_variables
> EXTERNAL
> --environment_service_address
> LOOPBACK
> (none)
> This way we can validate that the user is configuring these options correctly
> (ie give a warning or error if they use options that do not apply to their
> chosen --environment_type).
> We can deprecate the --environment_config option, logging a warning until
> removing this option altogether in a future Beam release.
> [https://beam.apache.org/documentation/runtime/sdk-harness-config/]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
