[
https://issues.apache.org/jira/browse/FLINK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238633#comment-17238633
]
Xintong Song commented on FLINK-20249:
--------------------------------------
[~trohrmann],
Ok, I think I see your point.
{quote}whether we really need to support JM failover w/o HA.
{quote}
I think JM failover w/o HA is currently supported. I agree this is not
something we want to promote/recommend, but I would suggest not to change it.
On the other hand, reusing TMs from previous attempt in non-HA mode is indeed
specific to the native K8s deployment. The only reason we have the internal
service is to direct old TMs to the new JM in non-HA mode. If reusing old TMs
in non-HA mode is not necessary/preferred, there would be no reason for the
internal service.
I think you are right. Given that JM failover w/o HA is a scenario that we do
not want to promote, we probably should not over-optimize things for it, which
may mislead the users thinking everything should work fine in this scenario.
I guess it then comes to the question, given that the feature is already there,
do we want to remove the internal service and disable TM reusing across
attempts for the native K8s deployment?
> Rethink the necessity of the k8s internal Service even in non-HA mode
> ---------------------------------------------------------------------
>
> Key: FLINK-20249
> URL: https://issues.apache.org/jira/browse/FLINK-20249
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / Kubernetes
> Affects Versions: 1.11.0
> Reporter: Ruguo Yu
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Attachments: k8s internal service - in english.pdf, k8s internal
> service - v2.pdf, k8s internal service.pdf
>
>
> In non-HA mode, k8s will create internal service that directs the
> communication from TaskManagers Pod to JobManager Pod, and TM Pods could
> re-register to the new JM Pod once a JM Pod failover occurs.
> However recently I do an experiment and find a problem that k8s will first
> create new TM pods and then destory old TM pods after a period of time once
> JM Pod failover (note: new JM podIP has changed), then job will be reschedule
> by JM on new TM pods, it means new TM has been registered to JM.
> During this process, internal service is active all the time, but I think it
> is not necessary that keep this internal service, In other words, wo can weed
> out internal service and use JM podIP for TM pods communication with JM pod,
> In this case, it be consistent with HA mode.
> Finally，related experiments is in attached (k8s internal service.pdf).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
