Hi Klou, +1 for this proposal. I am missing any mention of "cancel" in the design document though. In my understanding we are not planning to deprecate "cancel" completely (just cancel-with-savepoint, which is superseded by "stop"). In any case we should consider it in the design document. It seems to me that "cancel" should be consider an ungraceful shutdown, so that the Job could be restarted from last (retained) checkpoint (as right now).
Cheers, Konstantin On Thu, Jul 4, 2019 at 3:21 PM Kostas Kloudas <kklou...@gmail.com> wrote: > Hi all, > > In many cases, UDFs (User Defined Functions) need to be able to perform > application-specific actions when they stop in an orderly manner. > Currently, Flink's UDFs, and more specifically the RichFunction which > exposes lifecycle-related hooks, only have a close() method which is called > in any case of job termination. This includes any form of orderly > termination (STOP or End-Of-Stream) and termination due to an error. > > > The FLIP in [1] and the design document in [2] propose the addition of an > interface that will allow UDFs that implement it to perform application > specific logic in the case of graceful termination. These cases include > DRAIN and SUSPEND for streaming jobs (see FLIP-34), but also reaching the > End-Of-Stream for jobs with finite sources. > > Let's have a lively discussion to solve this issue that has been around for > quite some time. > > Cheers, > Kostas > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs > > [2] > > https://docs.google.com/document/d/1SXfhmeiJfWqi2ITYgCgAoSDUv5PNq1T8Zu01nR5Ebog/edit?usp=sharing > -- Konstantin Knauf | Solutions Architect +49 160 91394525 Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2010 -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen
