Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

2019-07-18 Thread Yun Gao
Congratulations! Best, Yun -- From:Kostas Kloudas Send Time:2019 Jul. 18 (Thu.) 17:30 To:dev Subject:Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project Congratulations Becket! Kostas On Thu

Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project

2019-07-22 Thread Yun Gao
Congratulations Zhijiang! Best, Yun -- From:Paul Lam Send Time:2019 Jul. 23 (Tue.) 09:38 To:dev Subject:Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project Congrats Zhijiang! Best, Paul Lam > 在 2019

Re: [ANNOUNCE] Kete Young is now part of the Flink PMC

2019-07-23 Thread Yun Gao
Congratulations Kurt! best, Yun -- From:Jeff Zhang Send Time:2019 Jul. 23 (Tue.) 18:14 To:dev Subject:Re: [ANNOUNCE] Kete Young is now part of the Flink PMC Congratulations Kurt Biao Liu 于2019年7月23日周二 下午6:07写道: > Congrats Kurt

Re: Re: [ANNOUNCE] Hequn becomes a Flink committer

2019-08-08 Thread Yun Gao
Congratulations Hequn! Best, Yun -- From:Congxian Qiu Send Time:2019 Aug. 8 (Thu.) 21:34 To:Yu Li Cc:Haibo Sun ; dev ; Rong Rong ; user Subject:Re: Re: [ANNOUNCE] Hequn becomes a Flink committer Congratulations Hequn! Best, Co

Re: Checkpointing under backpressure

2019-08-14 Thread Yun Gao
Hi, Very thanks for sharing the thoughts on the unaligned checkpoint ! Another question regarding I 2.C (Performance) by Paris is that do we always snapshot and broadcast the marks once the task receives the first mark from JM o? If so, then we will always need to snapshot all the

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

2019-08-14 Thread Yun Gao
Congratulations Andrey! Best, Yun -- From:Congxian Qiu Send Time:2019 Aug. 15 (Thu.) 10:28 To:dev@flink.apache.org Subject:Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer Congratulations Andery! Best, Congxian Kurt You

Re: Checkpointing under backpressure

2019-08-15 Thread Yun Gao
Hi, Very thanks for the great points! For the prioritizing inputs, from another point of view, I think it might not cause other bad effects, since we do not need to totally block the channels that have seen barriers after the operator has taking snapshot. After the snapshotting, if the

[DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-22 Thread Yun Gao
Hi everyone, In some scenarios we met a requirement that some operators want to send records to theirs downstream operators with an multicast communication pattern. In detail, for some records, the operators want to send them according to the partitioner (for example, Rebalance), and for s

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-23 Thread Yun Gao
which is hard to maintain and extend. Thanks, Zhu Zhu Yun Gao 于2019年8月22日周四 下午8:42写道: Hi everyone, In some scenarios we met a requirement that some operators want to send records to theirs downstream operators with an multicast communication pattern. In detail, for some records, the oper

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-23 Thread Yun Gao
Aug 2019, at 10:20, Yun Gao wrote: > >Hi Piotr, > > Thanks a lot for the suggestions! > > The core motivation of this discussion is to implement a new > iteration library on the DataStream, and it requires to insert special > records in

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-23 Thread Yun Gao
Best, Yun -- From:Piotr Nowojski Send Time:2019 Aug. 23 (Fri.) 22:29 To:Zhu Zhu Cc:dev ; Yun Gao Subject:Re: [DISCUSS] Enhance Support for Multicast Communication Pattern Hi, If the primary motivation is broadcasting (for the iterations) and we have no immediate need for multicast (

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-26 Thread Yun Gao
/issues.apache.org/jira/browse/FLINK-6936 On Aug 24, 2019, at 5:54 AM, Zhu Zhu wrote: Hi Piotr, Thanks for the explanation. Agreed that the broadcastEmit(record) is a better choice for broadcasting for the iterations. As broadcasting for the iterations is the first motivation, let's support

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-26 Thread Yun Gao
le to decouple the broadcasting events requirements and more generalized multicasting mechanism. :) Best, Yun -- From:SHI Xiaogang Send Time:2019 Aug. 27 (Tue.) 09:16 To:dev ; Yun Gao Cc:Piotr Nowojski Subject:Re: [DISCUSS] En

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-29 Thread Yun Gao
would also prefer to see a motivation design doc, how that feature would be used for example for cross or theta joins in the Table API, since very similar questions would apply to that as well. Piotrek > On 27 Aug 2019, at 08:10, SHI Xiaogang wrote: > > Hi Yun Gao, > > Th

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

2019-09-08 Thread Yun Gao
Congratulations, Kostas! Best, Yun‍‍‍ -- From:Becket Qin Send Time:2019 Sep. 9 (Mon.) 10:47 To:dev Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC Congrats, Kostas! On Sun, Sep 8, 2019 at 11:48 PM mya

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

2019-09-11 Thread Yun Gao
Congratulations Zili! Best, Yun -- From:Yangze Guo Send Time:2019 Sep. 12 (Thu.) 09:38 To:dev Subject:Re: [ANNOUNCE] Zili Chen becomes a Flink committer Congratulations! Best, Yangze Guo On Thu, Sep 12, 2019 at 9:

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-10-30 Thread Yun Gao
said and especially can't agree more to let > > > developers who will own it make the decision. > > > > > > Best regards, > > > Jing > > > > > > > > > On Sun, Feb 12, 2023 at 2:53 AM Dong Lin wrote: > > > > > > &

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-21 Thread Yun Gao
Congratulations Jingsong! Best, Yun -- From:Jingsong Li Send Time:2020 Feb. 21 (Fri.) 21:42 To:Hequn Cheng Cc:Yang Wang ; Zhijiang ; Zhenghua Gao ; godfrey he ; dev ; user Subject:Re: [ANNOUNCE] Jingsong Lee becomes a Fli

Re: [VOTE] FLIP-98: Better Back Pressure Detection

2020-02-23 Thread Yun Gao
+1 (non-binding) This should make tuning back pressure easier, which is one of the most common problems met for users. Best, Yun -- From:Benchao Li Send Time:2020 Feb. 24 (Mon.) 08:32 To:dev Cc:Zhijiang Subjec

Re: [VOTE] [FLIP-76] Unaligned checkpoints

2020-03-12 Thread Yun Gao
+1 (non-binding) I think the PoC result has shown the effect on reducing checkpoint time when back-pressure occurs, and I totally agree with that the feature could be implemented in steps. -- From:Roman Khachatryan Send Time:

Re: [DISCUSS] FLIP-115: Filesystem connector in Table

2020-03-13 Thread Yun Gao
Hi, Very thanks for Jinsong to bring up this discussion! It should largely improve the usability after enhancing the FileSystem connector in Table. I have the same question with Piotr. From my side, I think it should be better to be able to reuse existing StreamingFileSink.

Re: [ANNOUNCE] New Apache Flink PMC Member - Hequn Chen

2020-04-19 Thread Yun Gao
Congratulations Hequn! Best, Yun -- From:Hequn Cheng Send Time:2020 Apr. 18 (Sat.) 12:48 To:dev Subject:Re: [ANNOUNCE] New Apache Flink PMC Member - Hequn Chen Many thanks for your support. Thank you! Best,

Re: [DISCUSS] FLIP-76: Unaligned checkpoints

2019-10-10 Thread Yun Gao
Hi Arvid, Very thanks for bring up the discussion! From our side unable to finish the checkpoint is commonly met for online jobs, therefore +1 from my side to implement this. A tiny issue of the FLIP is that the Discussion Thread URL attached seems to be not right.

Re: [VOTE] FLINK-67: Cluster partition life-cycle

2019-10-16 Thread Yun Gao
Hi Chesnay, +1 (non-binding). Very thanks for driving this. Best, Yun -- From:Chesnay Schepler Send Time:2019 Oct. 16 (Wed.) 21:01 To:dev@flink.apache.org Subject:[VOTE] FLINK-67: Cluster partiti

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

2019-10-28 Thread Yun Gao
Congratulations Becket! Best, Yun -- From:Jingsong Li Send Time:2019 Oct. 29 (Tue.) 10:23 To:dev Subject:Re: [ANNOUNCE] Becket Qin joins the Flink PMC Congratulations Becket! Best, Jingsong Lee On Tue, Oct 29, 2019 at 10:18 AM

Re: [VOTE] Accept Stateful Functions into Apache Flink

2019-10-30 Thread Yun Gao
+1 (non-binding) Very thanks for bringing this to the community! -- From:jincheng sun Send Time:2019 Oct. 31 (Thu.) 10:22 To:dev Cc:Vasiliki Kalavri Subject:Re: [VOTE] Accept Stateful Functions into Apache Flink big +1 (b

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

2019-11-08 Thread Yun Gao
Congratulations Jark! Best, Yun -- From:wenlong.lwl Send Time:2019 Nov. 8 (Fri.) 18:31 To:dev Subject:Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC Congratulations Jark, well deserved! Best, Wenlong Lyu On Fri, 8 Nov 201

Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer

2019-12-15 Thread Yun Gao
Congratulations Zhuzhu! Best, Yun -- From:Guowei Ma Send Time:2019 Dec. 16 (Mon.) 11:16 To:dev Subject:Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer Congrats Zhuzhu! Best, Guowei Zhenghua Gao 于2019年12月16日周一 上午10:47写

Re: [VOTE] FLIP-92: Add N-Ary Stream Operator in Flink

2020-01-12 Thread Yun Gao
+1 (non-binding). Very thanks for introducing this topic back, and it should be able to bring improvements in the discussed scenarios. Best, Yun -- From:Arvid Heise Send Time:2020 Jan. 10 (Fri.) 16:48 To:dev ; Zhijiang Subject:

Re: BroadcastState enforce processing

2020-01-20 Thread Yun Gao
Hi Dominik, If I understand the scenario correctly, I think one possible solution is to implement one specified TwoInputStreamOperator directly, and also implements the InputSelectable interface. Then this operator could control the priority its two inputs by return proper InputSele

Re: [ANNOUNCE] Yu Li became a Flink committer

2020-01-31 Thread Yun Gao
Congratulations Yu! Best, Yun -- From:felixzheng zheng Send Time:2020 Jan. 31 (Fri.) 10:19 To:dev Subject:Re: [ANNOUNCE] Yu Li became a Flink committer Congrats! Benchao Li 于2020年1月31日周五 上午10:02写道: > Congrat

回复:Request for contributor permission

2018-10-01 Thread Yun Gao
Thanks a lot ! @Chesnay Schepler@Tzu-Li Chen : ) Yours Yun Gao -- 发件人:Chesnay Schepler 发送时间:2018年10月1日(星期一) 17:10 收件人:dev ; Tzu-Li Chen ; yungao.gy ; trohrmann ; Stephan Ewen 主 题:Re: Request for contributor permission @Yun

回复:[DISCUSS] Embracing Table API in Flink ML

2018-11-20 Thread Yun Gao
design doc! Yours sincerely Yun Gao -- 发件人:Weihua Jiang 发送时间:2018年11月20日(星期二) 20:53 收件人:dev 主 题:[DISCUSS] Embracing Table API in Flink ML ML Pipeline is the idea brought by Scikit-learn <https://arxiv.org/abs/1309.0238>

Re: [DISCUSS]Enhancing flink scheduler by implementing blacklist mechanism

2018-11-27 Thread Yun Gao
obey the blacklist, then we may need to deal with the inappropriate resource. Looking forward to the future advance of this feature! Thanks again for the exciting proposal. Best, Yun Gao -- From:zhijiang Send Time:2018 Nov

Re: [DISCUSS] Detection Flink Backpressure

2019-01-02 Thread Yun Gao
icator. Besides, another similar thought is that we may also add new InputBufferUsage and OutputBufferUsage metrics to show (number of queued buffers / number of all buffers) instead. Best, Yun Gao [1] https://issues.apache.org/jira/browse/FLINK-10981 [2] https://issues.apac

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

2019-06-24 Thread Yun Gao
Congratulations! Best, Yun -- From:Robert Metzger Send Time:2019 Jun. 24 (Mon.) 23:16 To:dev Subject:[ANNOUNCE] Jincheng Sun is now part of the Flink PMC Hi all, On behalf of the Flink PMC, I'm happy to announce that Jincheng S

Re: [VOTE] FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API)

2020-09-03 Thread Yun Gao
Very thanks for bring this up! +1 for deprecating the DataSet API and providing a unified streaming/batch programming model to users. Best, Yun -- Sender:Aljoscha Krettek Date:2020/09/02 19:22:51 Recipient:Flink Dev Theme:[VOTE]

Re: Re: [ANNOUNCE] New Apache Flink Committer - Niels Basjes

2020-09-14 Thread Yun Gao
Congratulations Niels! Best, Yun -- Sender:Congxian Qiu Date:2020/09/15 13:33:31 Recipient:dev@flink.apache.org Theme:Re: [ANNOUNCE] New Apache Flink Committer - Niels Basjes Congratulations Best, Congxian Yang Wang 于2020年9月

Re: Re: [ANNOUNCE] New Apache Flink Committer - Arvid Heise

2020-09-14 Thread Yun Gao
Congratulations Arvid ! Best, Yun --Original Mail -- Sender:Yangze Guo Send Date:Tue Sep 15 13:39:56 2020 Recipients:dev Subject:Re: [ANNOUNCE] New Apache Flink Committer - Arvid Heise Congrats! Arvid. Best, Yangze Guo On Tue, Sep 15, 2020 at 1:31 PM Congxi

Re: Re: [VOTE] FLIP-140: Introduce bounded style execution for keyed streams

2020-09-14 Thread Yun Gao
Very thanks for bring this up! +1 (non-binding) Best, Yun -- Sender:Seth Wiesman Date:2020/09/14 21:56:55 Recipient:dev Theme:Re: [VOTE] FLIP-140: Introduce bounded style execution for keyed streams +1 (binding) Seth On Thu, S

Re: Re: [ANNOUNCE] New Apache Flink Committer - Yun Tang

2020-09-15 Thread Yun Gao
Congratulations Yun Tang! Best, Yun Gao -- Sender:Zhu Zhu Date:2020/09/15 18:49:24 Recipient:dev Cc:; Yun Tang Theme:Re: [ANNOUNCE] New Apache Flink Committer - Yun Tang Congratulations! Thanks, Zhu Yu Li 于2020年9月15日周二 下午6:19写

Re: Re: [ANNOUNCE] New Apache Flink Committer - Igal Shilman

2020-09-15 Thread Yun Gao
Congratulations Igal! Best, Yun -- Sender:Stephan Ewen Date:2020/09/15 22:48:30 Recipient:dev Theme:Re: [ANNOUNCE] New Apache Flink Committer - Igal Shilman Welcome, Igal! On Tue, Sep 15, 2020 at 3:18 PM Seth Wiesman wrote:

Re: Re: [VOTE] FLIP-134: Batch execution for the DataStream API

2020-09-16 Thread Yun Gao
+1 (non-binding) Very thanks for bring this up! And the FLIP is indeed necessary for stream & batch unification. -- Sender:Dawid Wysakowicz Date:2020/09/16 15:01:08 Recipient:; Aljoscha Krettek Theme:Re: [VOTE] FLIP-134: Batch execu

Re: Re: [ANNOUNCE] Apache Flink 1.11.2 released

2020-09-18 Thread Yun Gao
Great! Very thanks @ZhuZhu for driving this and thanks for all contributed to the release! Best, Yun --Original Mail -- Sender:Jingsong Li Send Date:Thu Sep 17 13:31:41 2020 Recipients:user-zh CC:dev , user , Apache Announce List Subject:Re: [ANNOUNCE] Apac

Re: Re: [ANNOUNCE] New Apache Flink Committer - Godfrey He

2020-09-18 Thread Yun Gao
Congratulations Godfrey! Best, Yun --Original Mail -- Sender:Dawid Wysakowicz Send Date:Thu Sep 17 14:45:55 2020 Recipients:Flink Dev , 贺小令 Subject:Re: [ANNOUNCE] New Apache Flink Committer - Godfrey He Congratulations! On 16/09/2020 06:19, Jark Wu wrote: > H

Re: Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-05 Thread Yun Gao
Congratulations Zhu! Best, Yun-- Sender:Arvid Heise Date:2020/10/05 15:53:01 Recipient:dev Cc: Theme:Re: [ANNOUNCE] New PMC member: Zhu Zhu Congratulations Zhu Zhu! On Mon, Oct 5, 2020 at 9:28 AM Till Rohrmann wrote: > Congrats an

[DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-10-08 Thread Yun Gao
Hi, devs & users As discussed in FLIP-131 [1], Flink will make DataStream the unified API for processing bounded and unbounded data in both streaming and blocking modes. However, one long-standing problem for the streaming mode is that currently Flink does not support checkpoints after some tas

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-10-08 Thread Yun Gao
Hi, devs & users Very sorry for the spoiled formats, I resent the discussion as follows. As discussed in FLIP-131[1], Flink will make DataStream the unified API for processing bounded and unbounded data in both streaming and blocking modes. However, one long-standing problem for the streaming m

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-10-12 Thread Yun Gao
27;d expand the recovery section a bit. It would be the first time that we recover an incomplete DAG. Afaik the subtasks are deployed before the state is recovered, so at some point, the subtasks either need to be removed again or maybe we could even avoid them being created in the first place

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-10-13 Thread Yun Gao
completely ignore the problem on how to store and restore output buffers of a completed task (also important for the next point). 5) I think we are on the same page and I completely agree that for the MVP/first version, it's completely fine to start and immediately stop. A tad better would

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-10-13 Thread Yun Gao
hrmann Send Time:2020 Oct. 13 (Tue.) 17:25 To:Yun Gao Cc:Arvid Heise ; Flink Dev ; User-Flink Subject:Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished Thanks for starting this discussion Yun Gao, I have three comments/questions: 1) When restarting all tasks independent of the st

Re: Re: [ANNOUNCE] New Apache Flink Committer - Congxian Qiu

2020-10-29 Thread Yun Gao
Congratulations Congxian ! Best, Yun -- Sender:Leonard Xu Date:2020/10/29 21:19:56 Recipient:dev Theme:Re: [ANNOUNCE] New Apache Flink Committer - Congxian Qiu Congratulations! Congxian Best, Leonard > 在 2020年10月29日,20:55,Stephan E

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-12-14 Thread Yun Gao
Hi all, I would like to resume this discussion for supporting checkpoints after tasks Finished :) Based on the previous discussion, we now implement a version of PoC [1] to try the idea. During the PoC we also met with some possible issues: 1. To include EndOfPartition into considerat

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-12-15 Thread Yun Gao
ettek Send Time:2020 Dec. 15 (Tue.) 18:11 To:dev Subject:Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished Thanks for the thorough update! I'll answer inline. On 14.12.20 16:33, Yun Gao wrote: > 1. To include EndOfPartition into consideration for barrier alignment

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-12-24 Thread Yun Gao
47: Support Checkpoints After Tasks Finished Thanks for the thorough update! I'll answer inline. On 14.12.20 16:33, Yun Gao wrote: > 1. To include EndOfPartition into consideration for barrier alignment at > the TM side, we now tend to decouple the logic for EndOfPartit

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-05 Thread Yun Gao
g we want to have it more fine-grained on OperatorSubtaskState. Maybe we can store the flag inside managed or raw state without changing the format? On Fri, Dec 25, 2020 at 8:39 AM Yun Gao wrote: Hi all, I tested the previous PoC with the current tests and I found some new issues that mi

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-05 Thread Yun Gao
Hi Aljoscha, Very thanks for the feedbacks! For the second issue, I'm indeed thinking the race condition between deciding to trigger and operator get finished. And for this point, > One thought here is this: will there ever be intermediate operators that > should be run

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-06 Thread Yun Gao
of examples if you go beyond chained operators and fully connected exchanges. Think of any fan-in, let's assume you have source S1...S4, with S1+S2->M1, and S3+S4->M2. If S1 is finished, S2 and M1 is still running. Or I didn't get your question ;). On Tue, Jan 5, 2021 a

Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-06 Thread Yun Gao
Hi Arvid, Very thanks for the feedbacks! I'll try to answer the questions inline: > I'm also concerned about the notion of a final checkpoint. What happens > when this final checkpoint times out (checkpoint timeout > async timeout) > or fails for a different reason? I'm currently more inclined to

Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-06 Thread Yun Gao
eat trying. Best, Yun --Original Mail -- Sender:Arvid Heise Send Date:Thu Jan 7 00:52:27 2021 Recipients:Aljoscha Krettek CC:dev , Yun Gao Subject:Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished Okay then at least you guys are in sync ;) (A

Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-07 Thread Yun Gao
Hi Roman, Very thanks for the feedbacks! I'll try to answer the issues inline: > 1. Option 1 is said to be not preferable because it wastes resources and adds > complexity (new event). > However, the resources would be wasted for a relatively short time until the > job finishes completely.

Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-11 Thread Yun Gao
Hi Roman, Very thanks for the feedbacks ! > Probably it would be simpler to just decline the RPC-triggered checkpoint > if not all inputs of this task are finished (with CHECKPOINT_DECLINED_TASK_NOT_READY). > But I wonder how significantly this waiting f

Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-11 Thread Yun Gao
Hi Roman, Very thanks for the feedbacks and suggestions! > I think UC will be the common case with multiple sources each with DoP > 1. > IIUC, waiting for EoP will be needed on each subtask each time one of it's source subtask finishes. Yes, waiting for Eo

Re: [ANNOUNCE] Welcome Danny Cranmer as a new Apache Flink Committer

2021-01-12 Thread Yun Gao
Congratulations, Danny! Best, Yun -- From:Xintong Song Send Time:2021 Jan. 13 (Wed.) 15:29 To:dev Subject:Re: [ANNOUNCE] Welcome Danny Cranmer as a new Apache Flink Committer Congratulations, Danny. Welcome aboard. Thank you~

Re: Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2021-01-13 Thread Yun Gao
Hi all, I updated the FLIP[1] to reflect the major discussed points in the ML thread: 1) For the "new" root tasks finished before it received trigger message, previously we proposed to let JM re-compute and re-trigger the descendant tasks, but after the discussion we realized that it might

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-11-29 Thread Yun Gao
r.trigger() or not, thus I think now there is no divergence on this point. I also believe in we'll finally have some logic similar to the proposed one that drain all the times and process it. Best, Yun Gao [1] https://github.com/apache/flink/blob/a7fdab8b23cddf568fa32ee7eb804d7c3eb23a35/fl

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-11-30 Thread Yun Gao
hink the repeat registration is indeed a problem, but I think we could ignore the timers registered during termination). Could you further enlighten me a bit on this part? Best, Yun Gao -- From:Piotr Nowojski Send Time:2022 Nov. 30 (Wed

Re: Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-12-07 Thread Yun Gao
first have a PoC that demonstrate it is sufficient to only modify the operator implementation to handling the remaining workers properly on finish(). If there are new issues I'll post here and we could have some more discussion. Best, Yun Gao

Re: [VOTE] Release 1.17.0, release candidate #3

2023-03-22 Thread Yun Gao
: execution.checkpointing.checkpoints-after-tasks-finish.enabled: false In the following versions we'll also try to trigger a checkpoint immediately after the sources finished to reduce the waiting time. Best, Yun Gao [1] https://issues.apache.org/jira/browse/FLINK-25105 <https://issues.apache.org/jir

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-13 Thread Yun Gao
+1 (binding) On Sat, Jan 13, 2024 at 10:08 AM Rodrigo Meneses wrote: > > +1 (non binding) > > On Fri, Jan 12, 2024 at 5:45 PM Dong Lin wrote: > > > +1 (binding) > > > > On Sat, Jan 13, 2024 at 6:04 AM Austin Bennett wrote: > > > > > +1 (non-binding) > > > > > > On Fri, Jan 12, 2024 at 5:44 PM B

[ANNOUNCE] release-1.15 branch cut

2022-03-14 Thread Yun Gao
the PR into both master and release-1.15 branches - The JIRA ticket should be closed with the correct fix-versions (1.15.0 & 1.16.0). We are now creating the rc0, which is expected to be delivered soon. Thank you~ Joe, Till & Yun Gao

Re: [ANNOUNCE] New PMC member: Yuan Mei

2022-03-14 Thread Yun Gao
Congratulations, Yuan! Best, Yun Gao -- From:Francesco Guardiani Send Time:2022 Mar. 15 (Tue.) 00:21 To:dev Subject:Re: [ANNOUNCE] New PMC member: Yuan Mei Congratulations, Yuan! On Mon, Mar 14, 2022 at 3:51 PM yanfei lei

Re: [ANNOUNCE] release-1.15 branch cut

2022-03-21 Thread Yun Gao
Hi Piotr, Very thanks for the information and very sorry for the missing! Now I updated the flink.version property of the benchmark library and also update it to the checklist in the releasing introduction document~ Best, Yun Gao

Re: Re: Weekly 1.15 Sync tomorrow cancelled

2022-03-22 Thread Yun Gao
Hi Yu, Currently we still have some ongoing blocker issues tracked under [1]. We are tighly tracking the progress of these issues and we'll start creating the RC0 immediately after these blockers are solved, and hopefully inside this week. Best, Yun Gao [1] https://issues.apache.org

[ANNOUNCE] Release 1.15.0, release candidate #0

2022-03-24 Thread Yun Gao
[2]. - all artifacts that would normally be deployed to the Maven Central Repository [3]. - source code tag "release-1.15.0-rc0" [4] Your help testing the release will be greatly appreciated! And we'll create the votable RC1 as soon as all the efforts are finished. Thank you~ Joe, Ti

Re: Flink 1.15 Stabilisation Sync

2022-04-05 Thread Yun Gao
ddress the remaining blockers. Best, Yun Gao -- From:Robert Metzger Send Time:2022 Apr. 5 (Tue.) 13:34 To:dev Subject:Re: Flink 1.15 Stabilisation Sync Thanks a lot for the update. From the burndown board [1] it seems that there&

[VOTE] Release 1.15.0, release candidate #1

2022-04-11 Thread Yun Gao
lease-1.15.0-rc1" [5], * website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Joe, Till and Yun Gao [1] https://issues.apache.org/j

Re: Re: [VOTE] Release 1.15.0, release candidate #1

2022-04-12 Thread Yun Gao
lease candidate #1 Hi, Yun Gao There is a new bug [1] introduced in release-1.15. It's better to be fixed in 1.15.0 version. I'm terribly sorry to merge the pull request too late. [1] https://issues.apache.org/jira/browse/FLINK-26681 Best, Jing Zhang Johannes Moser 于2022年4月12日周二 14:53写道:

[VOTE] Release 1.15.0, release candidate #2

2022-04-12 Thread Yun Gao
lease-1.15.0-rc2" [5], * website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Joe, Till and Yun Gao [1] https://issues.apache.org/j

Re: [VOTE] Release 1.15.0, release candidate #2

2022-04-12 Thread Yun Gao
east 3 PMC affirmative votes. Thanks, Joe, Till and Yun Gao [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12350442 [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.15.0-rc2/ [3] https://dist.apache.org/repos/dist/release/flink/

Re: Re: [VOTE] Release 1.15.0, release candidate #1

2022-04-12 Thread Yun Gao
Hi Yu, Yes, the RC1 is officially canceled, the pointer has been moved to RC2 now~ Very thanks for the tips of remaining open issues! They should indeed be moved and I'll move them to the next release~ Best, Yun Gao -- Fr

Re: Re: [VOTE] Release 1.15.0, release candidate #2

2022-04-13 Thread Yun Gao
ginal Mail -- Sender:Chesnay Schepler Send Date:Thu Apr 14 00:40:32 2022 Recipients:Dev , Dawid Wysakowicz , Yun Gao Subject:Re: [VOTE] Release 1.15.0, release candidate #2 Re why CI didn't pick up on it: Currently, excessive NOTICE entries don't fail CI, only missing ones do. T

[VOTE] Release 1.15.0, release candidate #3

2022-04-14 Thread Yun Gao
"release-1.15.0-rc3" [5],* website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Joe, Till and Yun Gao [1] https://issues.apache.org/j

Re: Re: [VOTE] Release 1.15.0, release candidate #3

2022-04-19 Thread Yun Gao
Thanks Timo for checking and fixing this issue! Then this RC should be officially canceled and I'll post the next RC soon. Best, Yun Gao -- Sender:Timo Walther Date:2022/04/19 16:46:51 Recipient: Theme:Re: [VOTE] Release 1

[VOTE] Release 1.15.0, release candidate #4

2022-04-20 Thread Yun Gao
"release-1.15.0-rc4" [5],* website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Joe, Till and Yun Gao [1] https://issues.apach

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-22 Thread Yun Gao
might have other issues so that even we indeed find some blocker issues we could try best to handle them in one pass. Very thanks everyone! Best, Yun Gao -- From:Matthias Pohl Send Time:2022 Apr. 22 (Fri.) 23:45 To:dev Cc:Yu

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-24 Thread Yun Gao
Very thanks Jark for the checking! I also agree with that if possible let's first try to close this version and have a minor version soon. Best, Yun Gao -- From:Jark Wu Send Time:2022 Apr. 24 (Sun.) 21:07 To:dev ; Yu

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-24 Thread Yun Gao
Very thanks Peter for the help! The vote time would exclude the weekend, thus no worry for that. Best, Yun Gao -- From:Peter Schrott Send Time:2022 Apr. 25 (Mon.) 09:17 To:Matthias Pohl Cc:dev ; Yun Gao Subject:Re: [VOTE

Re: Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Yun Gao
Very thanks Matthias for tracking and double confirming this issue! Best, Yun Gao --Original Mail -- Sender:Matthias Pohl Send Date:Mon Apr 25 23:52:30 2022 Recipients:Yun Gao CC:dev , Yun Gao Subject:Re: [VOTE] Release 1.15.0, release candidate #4 +1 (non

Re: Re: Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Yun Gao
Very thanks everyone for the verifying! Now we should reach the condition to finalize this candidate, I'll annouce that in a separate thread. Also for the remaining non-blocking issues, perhaps we may start scheduling for the next minor version. Best, Yun Gao --Ori

[RESULT] [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Yun Gao
votes. Thanks everyone! Best, Yun Gao

Re: [RESULT] [VOTE] Release 1.15.0, release candidate #4

2022-05-02 Thread Yun Gao
Hi Thomas, Sorry for the long delay, currently we are very nearly done for finalizing the release and we should be able to finish all works and announce the release on Thursday. Best, Yun Gao -- From:Thomas Weise Send Time

Re: [RESULT] [VOTE] Release 1.15.0, release candidate #4

2022-05-02 Thread Yun Gao
Hi ChangZhuo, Yes, it is indeed the tag for 1.15.0, we have finished affairs like uploading artifacts, uploading images and creating tags. Best, Yun Gao -- From:ChangZhuo Chen (陳昌倬) Send Time:2022 May 3 (Tue.) 10:34 To:dev

[ANNOUNCE] Apache Flink 1.15.0 released

2022-05-04 Thread Yun Gao
://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12350442 We would like to thank all contributors of the Apache Flink community who made this release possible! Regards, Joe, Till, Yun Gao

Re: Re: [ANNOUNCE] New Flink PMC member: Yang Wang

2022-05-06 Thread Yun Gao
Congratulations Yang! Best, Yun Gao --Original Mail -- Sender:David Morávek Send Date:Sat May 7 01:05:41 2022 Recipients:Dev Subject:Re: [ANNOUNCE] New Flink PMC member: Yang Wang Nice! Congrats Yang, well deserved! ;) On Fri 6. 5. 2022 at 17:53, Peter Huang

Re: [ANNOUNCE] Apache Flink 1.15.0 released

2022-05-06 Thread Yun Gao
Very thanks everyone for all the help on releasing 1.15.0! And the doc links are now also fixed, thanks Peter for pointing out the issues! Best, Yun Gao -- From:Johannes Moser Send Time:2022 May 6 (Fri.) 17:42 To:dev Subject:Re

Re: [VOTE] FLIP-224: Blocklist Mechanism

2022-05-26 Thread Yun Gao
Thanks Lijie and Zhu for driving the FLIP! The blocked list functionality helps reduce the complexity in maintenance and the currently design looks good to me, thus +1 from my side (binding). Best, Yun -- From:Xintong Song Se

Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-05-30 Thread Yun Gao
Hi, Regarding the drop operation, with some offline discussion with Dalong and Zhu, we think that listening in the client side might be problematic since it would exit after submitting the jobs in detached mode, thus the operation might need to be in the JobMaster side. For the listener interf

Re: [VOTE] FLIP-232: Add Retry Support For Async I/O In DataStream API

2022-05-30 Thread Yun Gao
+1 (binding) Thanks Lincoln for driving the FLIP and discussion. Best, Yun Gao -- From:Jingsong Li Send Time:2022 May 31 (Tue.) 11:19 To:dev Subject:Re: [VOTE] FLIP-232: Add Retry Support For Async I/O In DataStream API +1

Re: [DISCUSS] Releasing 1.15.1

2022-06-09 Thread Yun Gao
Hi David, Very thanks for driving the new version, also +1 since we already accumulated some fixes. Regarding https://issues.apache.org/jira/browse/FLINK-27492, currently there are still some controversy with how to deal with the artifacts. I also agree we may not hold up the release with thi

  1   2   3   4   5   6   >