Mohemmad Zaid Khan created FLINK-30669:
------------------------------------------
Summary: Update recent job status in FlinkDeployment resource
object.
Key: FLINK-30669
URL: https://issues.apache.org/jira/browse/FLINK-30669
Project: Flink
Issue Type: Bug
Reporter: Mohemmad Zaid Khan
Attachments: image-2023-01-13-09-54-13-457.png,
image-2023-01-13-09-54-54-280.png
User jar has code as -
{code:java}
main() {
init env
pipelines.foreach{
env.fromSource(pipeline.getSource())
.map(pipeline.transform())
.sinkTo(pipeline.getSink())
env.execute(pipeline.getName())
}
}{code}
and below configuration -
{code:java}
execution.runtime-mode: "BATCH"
execution.attached: "true"
$internal.pipeline.job-id: "" {code}
When this single jar executed in Application Mode by using
flink-kubernetes-operator, multiple jobs are submitted sequentially and as per
design only one of the JobStatus is always associated with FlinkDeployment k8s
resource, this job status is periodically updated by operator. To update job
status in k8s resource, it fetches all of the job status from job-manager rest
endpoint and pick the first one and update that one. Problem is, job status
list returned by job-manager rest api is not sorted on time.
!image-2023-01-13-09-53-18-494.png|width=587,height=326!
!image-2023-01-13-09-54-54-280.png|width=353,height=284!
As you can see in above example, job autoscaling-3 is first one in the rest
response and same updated in FlinkDeployment resource, but FlinkDeployment
should have status of job autoscaling-19 because that is the last job finished.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
