I think we have to assume that some user has a custom config option that
uses underscores.
That said, we can probably assume that no one uses other special
characters like question marks and such, which are indeed allowed by the
YAML spec.
These kind of questions are precisely why I prefer the DYNAMIC_VARIABLES
approach; you don't even have to worry about this stuff.
The only question we'd have to answer is whether manually defined
properties should take precedent or not.
@Uce I can see how it could be cumbersome to modify, but at the same
time you can implement whatever other approach you want on top of it:
// this is a /conceptual /example for an optional setting
DYNAMIC_VARIABLES="${REST_PORT_SETTING}"
if _someCondition_:
export REST_PORT_SETTING="-Drest.port=1234"
// this is a /conceptual /example for a configurable setting
DYNAMIC_VARIABLES="-Drest.port=${MY_FANCY_VARIABLE:-8200}"
if _someCondition_:
export MY_FANCY_VARIABLE="1234"
Additionally, this makes it quite easy to audit stuff, since we can just
eagerly log what DYNAMIC_VARIABLES is set to.
On 1/26/2021 12:48 PM, Xintong Song wrote:
@Ufuk,
I also don't find any existing options with underscores in their keys.
However, I do not find any explicit rules forbid using them either. I'm not
saying this should block the FLIP. Just it would be nice to beware of this
issue, and maybe ensure the assumption with test cases if we finally decide
to go with these mapping rules.
Thank you~
Xintong Song
On Tue, Jan 26, 2021 at 7:27 PM Ufuk Celebi <u...@apache.org> wrote:
@Xingtong: The assumption for the mapping was that we only have dots and
hyphens in the keys. Do you have an example for a key which include
underscores? If underscores are common for keys (I couldn't find any
existing options that use it), it would certainly be a blocker for the
discussed approach.
On Tue, Jan 26, 2021, at 11:46 AM, Xintong Song wrote:
- The naming conversions proposed in the FLIP seems not bijective to
me.
There could be problems if the configuration key contains underscores.
- a_b -> FLINK_CONFIG_a_b
- FLINK_CONFIG_a_b -> a.b
