Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-21 Thread Lincoln Lee
Hi everyone, Thanks for all your feedback! I started a vote for this FLIP [1], please vote there or ask additional questions here[2]. [1] https://lists.apache.org/thread/nr9wwf98fkw1tk7ycgbcfjjo5g4x8pmz [2] https://lists.apache.org/thread/m9hj60p3mntyctkbxrksm8l4d0s4q9dw Best, Lincoln Lee Piot

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-20 Thread Piotr Nowojski
Fine by me. Thanks for driving this Lincoln :) Best, Piotrek wt., 20 wrz 2022 o 09:06 Lincoln Lee napisał(a): > Hi all, >I'll start a vote if there are no more objections till this > thursday(9.22). Looking forward to your feedback! > > [1] Flip-260: > > https://cwiki.apache.org/confluence/

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-20 Thread Lincoln Lee
Hi all, I'll start a vote if there are no more objections till this thursday(9.22). Looking forward to your feedback! [1] Flip-260: https://cwiki.apache.org/confluence/display/FLINK/FLIP-260%3A+Expose+Finish+Method+For+TableFunction [2] PoC: https://github.com/lincoln-lil/flink/tree/tf-finish-p

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-19 Thread Lincoln Lee
Hi Jingsong, Thank you for participating this discussion! For the method name, I think we should follow the new finish() method in `StreamOperator`, the BoundedOneInput might be removed in the future as discussed [1] before [1] https://lists.apache.org/thread/3ozw653ql8jso9w55p4pw8p4909trvkb

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-18 Thread Jingsong Li
+1 to add `finish()` method to `TableFunction` only. Can we use `endInput` just like `BoundedOneInput`? Best, Jingsong On Fri, Sep 16, 2022 at 11:54 PM Lincoln Lee wrote: > > Hi Dawid, Piotr, >Agree with you that add finish() method to `TableFunction` only. Other > `UserDefinedFunction`s (`

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-16 Thread Lincoln Lee
Hi Dawid, Piotr, Agree with you that add finish() method to `TableFunction` only. Other `UserDefinedFunction`s (`ScalarFunction`, `AggregateFunction`, `AggregateTableFunction`) are not necessarily to have the finish method(they can not emit records in legacy close() method). A `TableFunction` i

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-15 Thread Piotr Nowojski
Hi Dawid, Lincoln, I would tend to agree with Dawid. It seems to me like `TableFunction` is the one that needs to be taken care of. Other types of `UserDefinedFunction` wouldn't be able to emit anything from the `finish()` even if we added it. And if we added `finish(Collector out)` to them, it wo

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-15 Thread Dawid Wysakowicz
Hey Lincoln, Thanks for opening the discussion. To be honest I am not convinced if emitting from close there is a contract that was envisioned and thus should be maintained. As far as I can see it does affect only the TableFunction, because it has the collect method. None of the other UDFs (S

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-14 Thread Lincoln Lee
Thanks @Piort for your valuable inputs! I did a quick read of the previous discussion you mentioned, seems my flip title doesn't give a clear scope here and make some confusions, if my understanding is correct, the UDFs in your context is the user implemented `org.apache.flink.api.common.functions

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-14 Thread Piotr Nowojski
Hi Lincoln, Thanks for the proposal. Have you seen the old discussion about adding this `finish()` method? [1] We didn't add it to UDFs, as we didn't see a motivation (maybe we have missed something), and at the same time it wasn't that easy. Plain `finish()` wouldn't be enough. Users would need a

[DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-13 Thread Lincoln Lee
Hello everyone, I’d like to open a discussion on FLIP-260[1]: expose finish method for UserDefinedFunction, this makes a chance for users who rely on finish logic in the legacy close() method (< 1.14) to migrate to the new finish() method. The task lifecycle was changed in FLINK-22972[2]: a n