Hi Gyula, Yes you are right, we were also considering the external configurer. The reason we suggest the built in method is that it is more tightly coupled with the place the options are actually set. Therefore our hope is that, whenever somebody e.g. adds new fields to the ExecutionConfig he/she updates also the configure method. I am not entirely against your suggestion though, if this is the preferred way in the community.
Does anyone has any comments regarding the option keys?
Best,
Dawid
On 30/08/2019 14:57, Gyula Fóra wrote:
> Hi Dawid,
>
> Sorry I misread one of the interfaces a little (Configuration instead of
> ConfigurationReader), you are right.
> I was referring to:
>
>
> -
>
> void StreamExecutionEnvironment.configure(ConfigurationReader)
>
>
> This might be slightly orthogonal to the changes that you made here but
> what I meant is that instead of adding methods to the
> StreamExecutionEnvironment we could make this an external interface:
>
> EnvironmentConfigurer {
> void configure(StreamExecutionEnvironment, ConfigurationReader)
> }
>
> We could then have a default implementation of the EnvironmentConfigurer
> that would understand built in options. We could also allow users to pass
> custom implementations of this, which could configure the
> StreamExecutionEnvironment based on user defined config options. This is
> just a rough idea for extensibility and probably out of scope at first.
>
> Cheers,
> Gyula
>
> On Fri, Aug 30, 2019 at 12:13 PM Dawid Wysakowicz <dwysakow...@apache.org>
> wrote:
>
>> Hi Gyula,
>>
>> Thank you for the support on those changes.
>>
>> I am not sure if I understood your idea for the "reconfiguration" logic.
>>
>> The configure method on those objects would take ConfigurationReader. So
>> user can provide a thin wrapper around Configuration for e.g. filtering
>> certain logic, changing values based on other parameters etc. Is that
>> what you had in mind?
>>
>> Best,
>>
>> Dawid
>>
>> On 29/08/2019 19:21, Gyula Fóra wrote:
>>> Hi!
>>>
>>> Huuuge +1 from me, this has been an operational pain for years.
>>> This would also introduce a nice and simple way to extend it in the
>> future
>>> if we need.
>>>
>>> Ship it!
>>>
>>> Gyula
>>>
>>> On Thu, Aug 29, 2019 at 5:05 PM Dawid Wysakowicz <dwysakow...@apache.org
>>>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I wanted to propose a new, additional way of configuring execution
>>>> parameters that can currently be set only on such objects like
>>>> ExecutionConfig, CheckpointConfig and StreamExecutionEnvironment. This
>>>> poses problems such as:
>>>>
>>>> - no easy way to configure those from a file
>>>> - there is no easy way to pass a configuration from layers built on
>>>> top of StreamExecutionEnvironment. (e.g. when we want to configure
>> those
>>>> options from TableEnvironment)
>>>> - they are not automatically documented
>>>>
>>>> Note that there are a few concepts from FLIP-54[1] that this FLIP is
>> based
>>>> on.
>>>>
>>>> Would be really grateful to know if you think this would be a valuable
>>>> addition and any other feedback.
>>>>
>>>> Best,
>>>>
>>>> Dawid
>>>>
>>>> Wiki page:
>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-59%3A+Enable+execution+configuration+from+Configuration+object
>>>> Google doc:
>>>>
>> https://docs.google.com/document/d/1l8jW2NjhwHH1mVPbLvFolnL2vNvf4buUMDZWMfN_hFM/edit?usp=sharing
>>>>
>>>> [1]
>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-54%3A+Evolve+ConfigOption+and+Configuration
>>>>
>>>>
>>
signature.asc
Description: OpenPGP digital signature
