Hi Daren, Could you list the benefits for the users of Flink? I do think that an internal AWS requirement is not a good argument for getting something done in Flink.
Best regards, Martijn Op di 12 jul. 2022 om 21:17 schreef WONG, DAREN <daren...@amazon.co.uk.invalid>: > Hi Yang, > > The requirement to add *plan* currently originates from an internal AWS > requirement as our service needs visibility of *plan*, but we think it > could be beneficial as well to customers who uses *plan* too. > > Regards, > Daren > > > > > On 12/07/2022, 13:23, "Yang Wang" <danrtsey...@gmail.com> wrote: > > CAUTION: This email originated from outside of the organization. Do > not click links or open attachments unless you can confirm the sender and > know the content is safe. > > > > Thanks for the explanation. Only having 1 API call in most cases makes > sense to me. > > Could you please elaborate more about why do we need the *plan* in CR > status? > > > Best, > Yang > > Gyula Fóra <gyula.f...@gmail.com> 于2022年7月12日周二 17:36写道： > > > Hi Devs! > > > > I discussed with Daren offline, and I agree with him that > technically we > > almost never need 2 API calls. > > > > I think it's fine to have a second API call once directly after > application > > submission (technically even this can be eliminated by setting a fix > job id > > always). > > > > +1 from me. > > > > Cheers, > > Gyula > > > > > > On Tue, Jul 12, 2022 at 11:32 AM WONG, DAREN > <daren...@amazon.co.uk.invalid > > > > > wrote: > > > > > Hi Matyas, > > > > > > Thanks for the feedback, and yes I agree. An alternative approach > would > > > instead be: > > > > > > - 2 API calls only when jobID is not available (i.e when > submitting a new > > > application cluster, which is a one-off event). > > > - 1 API call when jobID is already available by directly calling > > > "/jobs/:jobid". > > > > > > With this approach, we can keep the API call to 1 in most cases. > > > > > > Regards, > > > Daren > > > > > > > > > On 11/07/2022, 14:44, "Őrhidi Mátyás" <matyas.orh...@gmail.com> > wrote: > > > > > > CAUTION: This email originated from outside of the > organization. Do > > > not click links or open attachments unless you can confirm the > sender and > > > know the content is safe. > > > > > > > > > > > > Hi Daren, > > > > > > At the moment the Operator fetches the job state via > > > > > > > > > https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jobs-overview > > > which contains the 'end-time' and 'duration' fields already. I > feel > > > calling > > > the > > > > > > > > > https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jobs-jobid > > > after the previous call for every job in every reconcile loop > would > > be > > > too > > > expensive. > > > > > > Best, > > > Matyas > > > > > > On Mon, Jul 11, 2022 at 3:17 PM WONG, DAREN > > > <daren...@amazon.co.uk.invalid> > > > wrote: > > > > > > > Hi everyone, I am Daren from AWS Kinesis Data Analytics > (KDA) team. > > > I had > > > > a quick chat with Gyula as I propose to include a few > additional > > > fields in > > > > the jobStatus CRD for Flink Kubernetes Operator such as: > > > > > > > > - endTime > > > > - duration > > > > - jobPlan > > > > > > > > Further details of each states can be found here< > > > > > > > > > > https://github.com/darenwkt/flink/blob/release-1.15.0/flink-runtime/src/main/java/org/apache/flink/runtime/rest/messages/job/JobDetailsInfo.java > > > >. > > > > Although addition of these 3 states stem from an internal > > > requirement, I > > > > think they would be beneficial to others who uses these > states in > > > their > > > > application as well. The list of states above are not > exhaustive, > > so > > > do let > > > > me know if there are other states that you would like to > include > > > together > > > > in this iteration cycle. > > > > > > > > JIRA: https://issues.apache.org/jira/browse/FLINK-28494 > > > > > > > > > > > > > >
