[
https://issues.apache.org/jira/browse/FLINK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238488#comment-17238488
]
Xintong Song commented on FLINK-20249:
--------------------------------------
[~jiang7chengzitc],
Your understanding is correct. Fixing (1) means Flink will not start new TMs if
there are enough recovered old ones. If not enough TMs are recovered, it will
only start as many as needed new TMs.
[~trohrmann],
Depending on the restart strategy, K8s/Yarn might bring failed JM back anyway.
W/o HA, jobs cannot recover from the latest success checkpoint, and will have
to go through a complete re-run. I'm not sure how many users are using Flink in
this way, but this is the current behavior. Therefore, if having the internal
service is not causing any problem (that's how I understand it from
[~jiang7chengzitc] and [~fly_in_gis]'s comments), I would vote for keep it and
preserve the old behavior.
FYI, I've opened FLINK-20332 for (1).
> Rethink the necessity of the k8s internal Service even in non-HA mode
> ---------------------------------------------------------------------
>
> Key: FLINK-20249
> URL: https://issues.apache.org/jira/browse/FLINK-20249
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / Kubernetes
> Affects Versions: 1.11.0
> Reporter: Ruguo Yu
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Attachments: k8s internal service - in english.pdf, k8s internal
> service - v2.pdf, k8s internal service.pdf
>
>
> In non-HA mode, k8s will create internal service that directs the
> communication from TaskManagers Pod to JobManager Pod, and TM Pods could
> re-register to the new JM Pod once a JM Pod failover occurs.
> However recently I do an experiment and find a problem that k8s will first
> create new TM pods and then destory old TM pods after a period of time once
> JM Pod failover (note: new JM podIP has changed), then job will be reschedule
> by JM on new TM pods, it means new TM has been registered to JM.
> During this process, internal service is active all the time, but I think it
> is not necessary that keep this internal service, In other words, wo can weed
> out internal service and use JM podIP for TM pods communication with JM pod,
> In this case, it be consistent with HA mode.
> Finally，related experiments is in attached (k8s internal service.pdf).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
