Kezhu Wang created FLINK-14793:
----------------------------------
Summary: Add SPI interface to discovery state backend and
establish well-known names
Key: FLINK-14793
URL: https://issues.apache.org/jira/browse/FLINK-14793
Project: Flink
Issue Type: New Feature
Components: Runtime / State Backends
Reporter: Kezhu Wang
Currently, {{StateBackendLoader.loadStateBackendFromConfig}} uses hybrid of
hardcoded state backend name and class path to construct
{{StateBackendFactory}}.
I think we can provide a SPI interface, say \{{StateBackendProvider}}, to
accomplish this.
{code:title=StateBackendProvider.java}
/**
* SPI interface for {@link StateBackend} discovery and naming.
*
* @param <T> StateBackend
*/
@PublicEvolving
public interface StateBackendProvider<T extends StateBackend> extends
StateBackendFactory<T> {
/**
* Name of provided {@link StateBackend}.
*
* @return name of provided {@link StateBackend}
*/
@Nonnull
String name();
/**
* Aliases for provided {@link StateBackend}. Implementations can use
this for deprecated names.
*
* @return aliases for provided {@link StateBackend}
*/
default Set<String> aliases() {
return Collections.emptySet();
}
}
{code}
With above spi, we gain several benefits over current hardcode and class path
approach:
* Explicit and established state backend naming and discovery instead of
hardcoded. We can know exactly which state backends are avaiable and whether
they are provided by user jars or not.
* In state backend configuration, we can simply use established state backend
name while not tied to fragile class path which is easily refactored out
without great caution. These established state backend names could be flink
community level, such as `jobmanager`, `filesystem` and `rocksdb`, and
corporation level, such as `corporation-a.product.state-backend-basename`.
Together with FLINK-14790, we can switch among state backends without knowledge
of concrete implementation of target state backends in source code level.
* More compatible with java module shipped in java 9 and above.
Discovery of state backend can be restricted in {{StateBackendLoader}}, and we
can fallback to class path approach if no matching state backend found in
discovered state backends.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
