Hi Matyas!
So to clarify your suggestion, we would have the following JobStatus fields:
jobId : String
state : String
savepointInfo : SavepointInfo
jobDetailsInfo : String (optional) - output of Flink Rest API job details
And the user could configure with a flag whether to include jobDetailsInfo
Hi Gyula,
since the jobDetailsInfo could evolve, another option would be to dump it
as yaml/json into the metadata.
Best,
Matyas
On Fri, Jul 15, 2022 at 2:58 PM Gyula Fóra wrote:
> Based on some further though, a reasonable middleground would be to add an
> optional metadata/jobDetailsInfo fie
Based on some further though, a reasonable middleground would be to add an
optional metadata/jobDetailsInfo field to the JobStatus.
We would also add an accompanying config option (default false) whether to
populate this field for jobs.
This way operator users could decide if they want to expose t
Hi All!
I fully acknowledge the general need to access more info about the running
deployments. This need however is very specific to the use-cases /
platforms built on the operator.
I think we need a good way to tackle this without growing the status
arbitrarily.
Currently the JobStatus in the o
Hi Martin,
Yes, that's understandable. I think adding job endTime, duration, jobPlan is
useful to other Flink users too as they now have info to track:
1. endTime: If the job has ended, the user can know when it has ended. If the
job is still streaming, then the user can know as it defaults to
Hi Daren,
Could you list the benefits for the users of Flink? I do think that an
internal AWS requirement is not a good argument for getting something done
in Flink.
Best regards,
Martijn
Op di 12 jul. 2022 om 21:17 schreef WONG, DAREN
:
> Hi Yang,
>
> The requirement to add *plan* currently o
Hi Yang,
The requirement to add *plan* currently originates from an internal AWS
requirement as our service needs visibility of *plan*, but we think it could be
beneficial as well to customers who uses *plan* too.
Regards,
Daren
On 12/07/2022, 13:23, "Yang Wang" wrote:
CAUTION: This
Thanks for the explanation. Only having 1 API call in most cases makes
sense to me.
Could you please elaborate more about why do we need the *plan* in CR
status?
Best,
Yang
Gyula Fóra 于2022年7月12日周二 17:36写道：
> Hi Devs!
>
> I discussed with Daren offline, and I agree with him that technically w
Hi Devs!
I discussed with Daren offline, and I agree with him that technically we
almost never need 2 API calls.
I think it's fine to have a second API call once directly after application
submission (technically even this can be eliminated by setting a fix job id
always).
+1 from me.
Cheers,
G
Hi Matyas,
Thanks for the feedback, and yes I agree. An alternative approach would instead
be:
- 2 API calls only when jobID is not available (i.e when submitting a new
application cluster, which is a one-off event).
- 1 API call when jobID is already available by directly calling "/jobs/:jobid
I share mytyas's concern if we list the jobs first and then followed by
some get-job-detail requests.
It is a bit expensive and I do not see the benefit to store jobPlan in the
CR status.
Best,
Yang
Őrhidi Mátyás 于2022年7月11日周一 21:43写道：
> Hi Daren,
>
> At the moment the Operator fetches the job
Hi Daren,
At the moment the Operator fetches the job state via
https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jobs-overview
which contains the 'end-time' and 'duration' fields already. I feel calling
the
https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_ap
12 matches
Mail list logo
