Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-23 Thread Till Rohrmann
I would be in favour of changing interfaces only between major versions. Otherwise we risk that existing setups break when upgrading to the latest minor version. +1 for the outreach. Cheers, Till On Wed, Oct 23, 2019 at 1:11 PM Zili Chen wrote: > Thanks for your clarification Till. > > For ter

Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-23 Thread Zili Chen
Thanks for your clarification Till. For terminology pluggable and customizable, I am mainly concerning about interface audience issue. Pluggable means we have multiple high-availability implementation but closed to extend in user scope; customizable means high-availability interface are stable and

Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-21 Thread Till Rohrmann
Hi Tison, I'm not sure whether I fully understand your distinction between customizable and pluggable. Maybe you could clarify your ideas a bit because you seem to favour support for pluggable implementations. Maybe let me try to answer some other questions you raised. With the HighAvailabilitySe

Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-17 Thread Zili Chen
Another perspective is that a stable, carefully-designed interface with clear semantic could be safer to customize. Following the discussion in FLINK-10333 our JobGraphStore is actually required performing write operation only with leadership, which is a basic requirement for coordination rather

Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-17 Thread Zili Chen
A challenge is how we ensure the support for customized implementation. When we introduce JobGraphStore#releaseJobGraph we actually change quite a bit codepath in Dispatcher. While we are unable to test arbitrarily customized implementation our compatibility promise is actually no more than compila

[DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-17 Thread Zili Chen
Hi devs, Recently the community excludes customize support on new restart strategies[1], which reminds me to think of which kind of customized support a framework like Flink should provides. The key idea is pluggable is not customizable. We might handle a series of implementation of restart stra