Re: [VOTE] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

Thanks Allison for proposing the FLIP. +1 (binding) On Fri, Sep 8, 2023 at 4:21 AM Allison Chang wrote: > Hi everyone, > > Would like to start the VOTE for FLIP-323< > https://cwiki.apache.org/confluence/display/FLINK/FLIP-323%3A+Support+Attached+Execution+on+Flink+Application+Completion+for+Ba

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

Hi Jark, Thanks for the comments. Please see my comments inline. On Sat, Sep 9, 2023 at 4:13 PM Jark Wu wrote: > Hi Xuannan, > > I leave my comments inline. > > > In the case where a user wants to > > use a CDC source and also determine backlog status based on watermark > > lag, we still need t

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

Hi folks, Thanks for the informative discussion. @Allison @Becket currently the FLIP only focuses on Yarn, but after reading all your discussions, if I am not mistaken, both Yarn and Kubernetes clusters should be supported. Does it make sense to update the FLIP accordingly? Best regards, Jing O

Re: [VOTE] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

+1(binding) Best Regards, Jing On Sun, Sep 10, 2023 at 10:17 AM Dong Lin wrote: > Thanks Allison for proposing the FLIP. > > +1 (binding) > > On Fri, Sep 8, 2023 at 4:21 AM Allison Chang > > wrote: > > > Hi everyone, > > > > Would like to start the VOTE for FLIP-323< > > > https://cwiki.apache

Re: [DISCUSS][FLINK-31788][FLINK-33015] Add back Support emitUpdateWithRetract for TableAggregateFunction

Thanks Jane for following up on this issue! +1 for adding it back first. Supporting emitUpdateWithRetract for TableAggregateFunction is a good feature, we should support it unless there are better alternatives. Best, Feng On Thu, Sep 7, 2023 at 11:01 PM Lincoln Lee wrote: > Thanks to Jane fo

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Thanks Zhanghao for initialing this discussion, I have just one comment: I checked the classes `SubtasksAllAccumulatorsHandler`, `SubtasksTimesHandler`, `SubtaskCurrentAttemptDetailsHandler`, `JobVertexTaskManagersHandler` and `JobExceptionsHandler` you mentioned in `Public Interfaces` and they ar

回复: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Hi Shammon, I think all REST API response messages (e.g. SubtaskExecutionAttemptDetailsInfo) should be considered as part of the public APIs and therefore be marked as @Public. It is true though none of them are marked as @public yet. Maybe we should do that. ccing @chesnay

Re: [VOTE] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

+1 (binding) Best, Weihua On Mon, Sep 11, 2023 at 3:16 AM Jing Ge wrote: > +1(binding) > > Best Regards, > Jing > > On Sun, Sep 10, 2023 at 10:17 AM Dong Lin wrote: > > > Thanks Allison for proposing the FLIP. > > > > +1 (binding) > > > > On Fri, Sep 8, 2023 at 4:21 AM Allison Chang > > > >

Re: [DISCUSS] Flink annotation strategy/consensus

Hi Jing, Thanks for bringing up the discussion. My two cents: 1. All the public methods / classes / interfaces MUST be annotated with one of the @Experimental / @PublicEvolving / @Public. In practice, all the methods by default inherit the annotation from the containing class, unless annotated ot

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Hi, Zhanghao Thanks for bringing this proposal. I have a concern: I prefer to keep the "host" field and add a "location" field in future versions. Consider a scenario where a machine (host) with multiple TaskManagers has poor processing performance due to some problems. By using a host field agg

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Thanks Zhanghao driving this FLIP, adding the port in Web UI seems good to me. Hi Shammon and Zhanghao, I would like to clarify the difference between Public Interfaces in FLIP and @Public in code. As I understand, the `Public Interfaces in FLIP` means these changes will be used in user side, su

Re: [DISCUSS] Flink annotation strategy/consensus

Thanks Jing for starting this discussion. For @Becket > 1. All the public methods / classes / interfaces MUST be annotated with one of the @Experimental / @PublicEvolving / @Public. In practice, all the methods by default inherit the annotation from the containing class, unless annotated otherwise

回复: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Hi Weihua, Thanks for the suggestion on keeping the host field. However, I think the location field can serve the needs for finding host-aggregative tasks as well and I'm not sure if a separate host field is still needed. The fields in REST API are mainly used under two scenarios, UI & code pro

回复: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Hi Fan Rui, Thanks for clarifying the definition of "public interfaces", that helps a lot! Best, Zhanghao Chen 发件人: Rui Fan <1996fan...@gmail.com> 发送时间: 2023年9月11日 11:18 收件人: dev@flink.apache.org 主题: Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Hi, Zhanghao Since the meaning of "host" is not aligned, it seems good for me to remove it in the future release. Best, Weihua On Mon, Sep 11, 2023 at 11:48 AM Chen Zhanghao wrote: > Hi Fan Rui, > > Thanks for clarifying the definition of "public interfaces", that helps a > lot! > > Best, > Z

[jira] [Created] (FLINK-33069) Mysql and Postgre catalog support url extra parameters

waywtdcc created FLINK-33069: Summary: Mysql and Postgre catalog support url extra parameters Key: FLINK-33069 URL: https://issues.apache.org/jira/browse/FLINK-33069 Project: Flink Issue Type: Ne

[DISCUSS] FLIP-365: Introduce flush interval to adjust the interval of emitting results with idempotent semantics

Hi all, Dong(cc'ed) and I are opening this thread to discuss our proposal to support buffering & flushing the output of operators with idempotent semantics, which has been documented in FLIP-365

Re: [DISCUSS] FLIP-307: Flink connector Redshift

Hello Danny, I wanted to express my gratitude for your valuable feedback and insightful suggestions. I will be revising the FLIP to incorporate all of your queries and review suggestions. Additionally, I plan to provide a Proof of Concept (POC) for the connector by the end of this week. This POC

Re: [DISCUSS] FLIP-307: Flink connector Redshift

Hi Leonard, > Do we have to rely on the latest version of JDBC Connector here? No, there's no need for us to depend on the latest version of the JDBC Connector. Redshift has its dedicated JDBC driver [1], which includes custom modifications tailored to Redshift's specific implementation needs. Th