Hi, I’m fine with either signature for the new execute() method but I think we should focus on the executor discovery and executor configuration part in this FLIP while FLIP-74 is about the evolution of the method signature to return a future.
I understand that it’s a bit weird, that this FLIP introduces a new interface only to be changed within the same Flink release in a follow-up FLIP. But I think we can still do it. What do you think? Best, Aljoscha > On 25. Sep 2019, at 10:11, Kostas Kloudas <kklou...@gmail.com> wrote: > > Hi Thomas and Zili, > > As you both said the Executor is a new addition so there are no > compatibility concerns. > Backwards compatibility comes into play on the > (Stream)ExecutionEnvironment#execute(). > > This method has to stay and keep having the same (blocking) semantics, > but we can > add a new one, sth along the lines of executeAsync() that will return > the JobClient and > will allow the caller to interact with the job. > > Cheers, > Kostas > > On Wed, Sep 25, 2019 at 2:44 AM Zili Chen <wander4...@gmail.com> wrote: >> >>> Since Exceutor is a new interface, why is backward compatibility a concern? >> >> For backward compatibility, it is on (Stream)ExecutionEnvironment#execute. >> You're right that we don't stick to blocking to return a JobExecutionResult >> in >> Executor aspect but implementing env.execute with a unique >> >> Executor#execute(or with suffix Async): CompletableFuture<JobClient> >> >> what do you think @Kostas Kloudas? >> >>> I could see that become an issue later when replacing Executor execute with >>> executeAsync. Or are both targeted for 1.10? >> >> IIUC both Executors and JobClient are targeted for 1.10. >> >> >> Thomas Weise <t...@apache.org> 于2019年9月25日周三 上午2:39写道： >>> >>> Since Exceutor is a new interface, why is backward compatibility a concern? >>> >>> I could see that become an issue later when replacing Executor execute with >>> executeAsync. Or are both targeted for 1.10? >>> >>> >>> On Tue, Sep 24, 2019 at 10:24 AM Zili Chen <wander4...@gmail.com> wrote: >>> >>>> Hi Thomas, >>>> >>>>> Should the new Executor execute method be defined as asynchronous? It >>>> could >>>>> return a job handle to interact with the job and the legacy environments >>>>> can still block to retain their semantics. >>>> >>>> During our discussion there will be a method >>>> >>>> executeAsync(...): CompletableFuture<JobClient> >>>> >>>> where JobClient can be regarded as job handle in your context. >>>> >>>> I think we remain >>>> >>>> execute(...): JobExecutionResult >>>> >>>> just for backward compatibility because this effort towards 1.10 which is >>>> not a >>>> major version bump. >>>> >>>> BTW, I am drafting details of JobClient(as FLIP-74). Will start a separated >>>> discussion >>>> thread on that interface as soon as I finish an early version. >>>> >>>> Best, >>>> tison. >>>> >>>> >>>> Thomas Weise <t...@apache.org> 于2019年9月25日周三 上午1:17写道： >>>> >>>>> Thanks for the proposal. These changes will make it significantly easier >>>> to >>>>> programmatically use Flink in downstream frameworks. >>>>> >>>>> Should the new Executor execute method be defined as asynchronous? It >>>> could >>>>> return a job handle to interact with the job and the legacy environments >>>>> can still block to retain their semantics. >>>>> >>>>> (The blocking execution has also made things more difficult in Beam, we >>>>> could simply switch to use Executor directly.) >>>>> >>>>> Thomas >>>>> >>>>> >>>>> On Tue, Sep 24, 2019 at 6:48 AM Kostas Kloudas <kklou...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> In the context of the discussion about introducing the Job Client API >>>>> [1], >>>>>> there was a side-discussion about refactoring the way users submit jobs >>>>> in >>>>>> Flink. There were many different interesting ideas on the topic and 3 >>>>>> design documents that were trying to tackle both the issue about code >>>>>> submission and the Job Client API. >>>>>> >>>>>> This discussion thread aims at the job submission part and proposes the >>>>>> approach of introducing the Executor abstraction which will abstract >>>> the >>>>>> job submission logic from the Environments and will make it API >>>> agnostic. >>>>>> >>>>>> The FLIP can be found at [2]. >>>>>> >>>>>> Please keep the discussion here, in the mailing list. >>>>>> >>>>>> Looking forward to your opinions, >>>>>> Kostas >>>>>> >>>>>> [1] >>>>>> >>>>>> >>>>> >>>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E >>>>>> [2] >>>>>> >>>>>> >>>>> >>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-73%3A+Introducing+Executors+for+job+submission >>>>>> >>>>> >>>>
