[
https://issues.apache.org/jira/browse/FLINK-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104109#comment-17104109
]
Yang Wang commented on FLINK-17598:
-----------------------------------
So do you mean to use StatefulSet to replace the Deployment in native K8s
integration? IIUC, for StatefulSet, we need to have the persistent volume. And
it is not always true in an unmanaged K8s cluster. Also we could not support
multiple jobmanagers to achieve a faster recovery.
Indeed, combined with persistent volume, {{FileSystemHAServices}} could make
the Flink deployment easier and we will not need the external system
dependency(e.g. zookeeper). I like this feature. Maybe the standalone cluster
on K8s could also benefit from this.
Moreover, our internal case is trying to go in a different way(native K8s
high-availability service). In the future, our uses could have more choices.
> Implement FileSystemHAServices for native K8s setups
> ----------------------------------------------------
>
> Key: FLINK-17598
> URL: https://issues.apache.org/jira/browse/FLINK-17598
> Project: Flink
> Issue Type: New Feature
> Components: Deployment / Kubernetes, Runtime / Coordination
> Reporter: Canbin Zheng
> Priority: Major
>
> At the moment we use Zookeeper as a distributed coordinator for implementing
> JobManager high availability services. But in the cloud-native environment,
> there is a trend that more and more users prefer to use *Kubernetes* as the
> underlying scheduler backend while *Storage Object* as the Storage medium,
> both of these two services don't require Zookeeper deployment.
> As a result, in the K8s setups, people have to deploy and maintain their
> Zookeeper clusters for solving JobManager SPOF. This ticket proposes to
> provide a simplified FileSystem HA implementation with the leader-election
> removed, which saves the efforts of Zookeeper deployment.
> To achieve this, we plan to
> # Introduce a {{FileSystemHaServices}} which implements the
> {{HighAvailabilityServices}}.
> # Replace Deployment with StatefulSet to ensure *at most one* semantics,
> preventing potential concurrent access to the underlying FileSystem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
