Hi Dawid,
Sorry I misread one of the interfaces a little (Configuration instead of
ConfigurationReader), you are right.
I was referring to:
-
void StreamExecutionEnvironment.configure(ConfigurationReader)
This might be slightly orthogonal to the changes that you made here but
what I meant is that instead of adding methods to the
StreamExecutionEnvironment we could make this an external interface:
EnvironmentConfigurer {
void configure(StreamExecutionEnvironment, ConfigurationReader)
}
We could then have a default implementation of the EnvironmentConfigurer
that would understand built in options. We could also allow users to pass
custom implementations of this, which could configure the
StreamExecutionEnvironment based on user defined config options. This is
just a rough idea for extensibility and probably out of scope at first.
Cheers,
Gyula
On Fri, Aug 30, 2019 at 12:13 PM Dawid Wysakowicz <dwysakow...@apache.org>
wrote:
> Hi Gyula,
>
> Thank you for the support on those changes.
>
> I am not sure if I understood your idea for the "reconfiguration" logic.
>
> The configure method on those objects would take ConfigurationReader. So
> user can provide a thin wrapper around Configuration for e.g. filtering
> certain logic, changing values based on other parameters etc. Is that
> what you had in mind?
>
> Best,
>
> Dawid
>
> On 29/08/2019 19:21, Gyula Fóra wrote:
> > Hi!
> >
> > Huuuge +1 from me, this has been an operational pain for years.
> > This would also introduce a nice and simple way to extend it in the
> future
> > if we need.
> >
> > Ship it!
> >
> > Gyula
> >
> > On Thu, Aug 29, 2019 at 5:05 PM Dawid Wysakowicz <dwysakow...@apache.org
> >
> > wrote:
> >
> >> Hi,
> >>
> >> I wanted to propose a new, additional way of configuring execution
> >> parameters that can currently be set only on such objects like
> >> ExecutionConfig, CheckpointConfig and StreamExecutionEnvironment. This
> >> poses problems such as:
> >>
> >> - no easy way to configure those from a file
> >> - there is no easy way to pass a configuration from layers built on
> >> top of StreamExecutionEnvironment. (e.g. when we want to configure
> those
> >> options from TableEnvironment)
> >> - they are not automatically documented
> >>
> >> Note that there are a few concepts from FLIP-54[1] that this FLIP is
> based
> >> on.
> >>
> >> Would be really grateful to know if you think this would be a valuable
> >> addition and any other feedback.
> >>
> >> Best,
> >>
> >> Dawid
> >>
> >> Wiki page:
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-59%3A+Enable+execution+configuration+from+Configuration+object
> >>
> >> Google doc:
> >>
> https://docs.google.com/document/d/1l8jW2NjhwHH1mVPbLvFolnL2vNvf4buUMDZWMfN_hFM/edit?usp=sharing
> >>
> >>
> >> [1]
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-54%3A+Evolve+ConfigOption+and+Configuration
> >>
> >>
> >>
>
>
