Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

2019-07-18 Thread Xintong Song
Congratulations Becket! Thank you~ Xintong Song On Thu, Jul 18, 2019 at 4:20 PM Kurt Young wrote: > Congrats Becket! > > Best, > Kurt > > > On Thu, Jul 18, 2019 at 4:12 PM JingsongLee .invalid> > wrote: > > > Congratulation

Re: [DISCUSS] Setup a bui...@flink.apache.org mailing list for travis builds

2019-07-21 Thread Xintong Song
+1. Thanks Jark for bringing this up. Thank you~ Xintong Song On Mon, Jul 22, 2019 at 12:37 PM Yun Tang wrote: > +1 sounds good, more people could be encouraged to involve in paying > attention to failure commit. > > Best > Yun Tang >

Re: Looking for reviewer: FLINK-13127

2019-07-22 Thread Xintong Song
David, Thank you for opening this PR. I also left a few comments. And I think we need a committer to assign this jira ticket to David. Maybe Till or any other committer could look into this? Thank you~ Xintong Song On Mon, Jul 22, 2019 at 8:37 PM Zili Chen wrote: > Hi David, >

Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project

2019-07-22 Thread Xintong Song
Congratulations Zhijiang! Thank you~ Xintong Song On Tue, Jul 23, 2019 at 10:13 AM Zhu Zhu wrote: > Congratulations Zhijiang! > > boshu Zheng 于2019年7月23日周二 上午9:46写道: > > > Congrats Zhijiang :) > > On 07/23/2019 09:29, Dian Fu wrote: > > Congrats Zhijiang! >

Re: [ANNOUNCE] Kete Young is now part of the Flink PMC

2019-07-23 Thread Xintong Song
Congratulations, Kurt~! Thank you~ Xintong Song On Tue, Jul 23, 2019 at 6:55 PM aihua li wrote: > Congratulations Kurt, Well deserved. > > > 在 2019年7月23日,下午5:24,Robert Metzger 写道: > > > > Hi all, > > > > On behalf of the Flink PMC, I'm happy to an

Re: [DISCUSS][CODE STYLE] Create collections always with initial capacity

2019-08-01 Thread Xintong Song
+1 on setting initial capacity only when have good expectation on the collection size. Thank you~ Xintong Song On Thu, Aug 1, 2019 at 2:32 PM Andrey Zagrebin wrote: > Hi all, > > As you probably already noticed, Stephan has triggered a discussion thread > about code style guide

[DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-07 Thread Xintong Song
rd to your feedbacks. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors [2] https://docs.google.com/document/d/1o4KvyyXsQMGUastfPin3ZWeUXWsJgoL7piqp1fFYJvA/edit?usp=sharing

Re: [ANNOUNCE] Hequn becomes a Flink committer

2019-08-07 Thread Xintong Song
Congratulations~! Thank you~ Xintong Song On Wed, Aug 7, 2019 at 4:00 PM vino yang wrote: > Congratulations! > > highfei2...@126.com 于2019年8月7日周三 下午7:09写道: > > > Congrats Hequn! > > > > Best, > > Jeff Yang > > > > > > Origi

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-09 Thread Xintong Song
does see that. What do you think? Thank you~ Xintong Song On Thu, Aug 8, 2019 at 5:09 PM Yang Wang wrote: > Hi xintong, > > > Thanks for your detailed proposal. After all the memory configuration are > introduced, it will be more powerful to control the flink memory usag

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-13 Thread Xintong Song
ative 2, users might configure them higher than what actually needed, just to avoid getting a direct OOM. For alternative 3, users do not get direct OOM, so they may not config the two options aggressively high. But the consequences are risks of overall container memory usage exceeds the budget. Than

Re: [VOTE] Flink Project Bylaws

2019-08-13 Thread Xintong Song
+1 (non-binding) Thank you~ Xintong Song On Tue, Aug 13, 2019 at 1:48 PM Robert Metzger wrote: > +1 (binding) > > On Tue, Aug 13, 2019 at 1:47 PM Becket Qin wrote: > > > Thanks everyone for voting. > > > > For those who have already voted, just want to bring

Re: [ANNOUNCE] Andrey Zagrebin becomes a Flink committer

2019-08-14 Thread Xintong Song
Congratulations Andery~! Thank you~ Xintong Song On Wed, Aug 14, 2019 at 3:31 PM Oytun Tez wrote: > Congratulations Andrey! > > I am glad the Flink committer team is growing at such a pace! > > --- > Oytun Tez > > *M O T A W O R D* > The World's Fastest

Re: why is not possible to handle a custom resource manager ?

2019-08-14 Thread Xintong Song
the flink-yarn module in the source codes for details. Thank you~ Xintong Song On Wed, Aug 14, 2019 at 1:43 PM Yang Wang wrote: > Hi Cristian, > > I think you could have a try on standalone cluster on X(underlying cluster > framework). By now it works well for kubernetes[1]. >

[DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-15 Thread Xintong Song
lease find more details in the FLIP wiki document [1]. Looking forward to your feedbacks. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Resource+Management

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-16 Thread Xintong Song
t very close to 200MB, user probably do not need to change the configurations. Therefore, I think from the user's perspective, a feasible configuration for alternative 2 may lead to lower resource utilization compared to alternative 3. Thank you~ Xintong Song On Fri, Aug 16, 2019 at 10

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-16 Thread Xintong Song
emory. The only parts of memory limited by JVM max direct memory are task off-heap memory and JVM overhead, which are exactly alternative 2 suggests to set the JVM max direct memory to. Thank you~ Xintong Song On Fri, Aug 16, 2019 at 1:48 PM Till Rohrmann wrote: > Thanks for the clarification

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-16 Thread Xintong Song
that we pass in the memory pool sizes in FLIP-49 [1]. - Regarding the return value of TaskExecutorGateway#requestResource, I think you're right. We should avoid using null as the return value. I think we probably should thrown an exception here. Thank you~ Xintong Song [1] https://cwiki.

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-19 Thread Xintong Song
n slot requests to specified default resource profiles, so they can be dynamically allocated from task executors' available resources just as other slot requests with specified resource profiles. Thank you~ Xintong Song On Mon, Aug 19, 2019 at 11:39 AM Yang Wang wrote: > Hi Xinton

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-19 Thread Xintong Song
1.1, it has the similar problem as 1.2, if the exceeded direct memory does not reach the max direct memory limit specified by the dedicated parameter. I think it is slightly better than 1.2, only because we can tune the parameter. Thank you~ Xintong Song On Mon, Aug 19, 2019 at 2:53 PM Step

[DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-08-19 Thread Xintong Song
topics, and start a separate new discussion thread as well as FLIP process for this one. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-53-Fine-Grained

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-19 Thread Xintong Song
For FLIP-56, I just started a new discussion thread [3]. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation [3] http://apach

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-19 Thread Xintong Song
lower than the direct memory allocation. Am I understanding this correctly? Thank you~ Xintong Song On Mon, Aug 19, 2019 at 4:21 PM JingsongLee wrote: > Hi stephan: > > About option 2: > > if additional threads not cleanly shut down before we can exit the task: > In the

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-08-20 Thread Xintong Song
@Zili As far as I know, Timo is drafting a FLIP that has taken the number 55. There is a round-up number maintained on the FLIP wiki page [1] shows which number should be used for the new FLIP, which should be increased by whoever takes the number for a new FLIP. Thank you~ Xintong Song [1

Re: CiBot Update

2019-08-22 Thread Xintong Song
The re-triggering travis feature is so convenient. Thanks Chesnay~! Thank you~ Xintong Song On Thu, Aug 22, 2019 at 9:26 AM Stephan Ewen wrote: > Nice, thanks! > > On Thu, Aug 22, 2019 at 3:59 AM Zili Chen wrote: > > > Thanks for your announcement. Nice work! >

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-22 Thread Xintong Song
discussions, and moved the other options to "Rejected Alternatives" for the moment. - Added implementation steps. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors On Mon, Aug 19, 2019

[VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Xintong Song
Hi all, I would like to start the voting process for FLIP-49 [1], which is discussed and reached consensus in this thread [2]. This voting will be open for at least 72 hours. I'll try to close it Aug. 30 10:00 UTC, unless there is an objection or not enough votes. Thank you~ Xintong Song

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-27 Thread Xintong Song
Added implementation steps for this FLIP on the wiki page [1]. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors On Mon, Aug 19, 2019 at 10:29 PM Xintong Song wrote: > Hi everyone, > > As Till

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Xintong Song
Alright, then let's keep the discussion in the DISCUSS mailing thread, and see whether we need to restart the vote. Thank you~ Xintong Song On Tue, Aug 27, 2019 at 8:12 PM Till Rohrmann wrote: > I had a couple of comments concerning the implementation plan. I've posted

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Xintong Song
ge network stack from using direct memory to using unsafe native memory. Network memory size is deterministic, cannot be reserved as managed memory does, and cannot be overused. I think it also works if we simply keep using direct memory for network and include it in jvm max direct memory size. Tha

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-27 Thread Xintong Song
current step 1 so we can use this option to decide whether should change the edge type. What do you think? - Agree. It should be easier to make the default value of 'allSourcesInSamePipelinedRegion' (or 'isStreamingJob') 'true', and set it to 'false' when u

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-01 Thread Xintong Song
ework for overwriting configurations with ENV variables. - Regarding memory reservation, I double checked with Yu and he will take care of it. Thank you~ Xintong Song On Thu, Aug 29, 2019 at 7:35 PM Till Rohrmann wrote: > What I forgot to add is that we could tackle specifying the confi

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-01 Thread Xintong Song
ENV variables. - Remove 'supporting memory reservation' from the scope of this FLIP. @till @stephan, please take another look see if there are any other concerns. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configu

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-09-01 Thread Xintong Song
Regarding changing edge type, I think actually we don't need to do this for batch jobs neither, because we don't have public interfaces for users to explicitly set slot sharing groups in DataSet API and SQL/Table API. We have such interfaces in DataStream API only. Thank you~ Xintong

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-09-01 Thread Xintong Song
Updated the FLIP wiki page [1], with the following changes. - Remove the step of converting pipelined edges between different slot sharing groups into blocking edges. - Set `allSourcesInSamePipelinedRegion` to true by default. Thank you~ Xintong Song On Mon, Sep 2, 2019 at 11:50 AM

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-02 Thread Xintong Song
not enough votes. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-49-Unified-Memory-Configuration-for-TaskExecutors-td31436.ht

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-09-03 Thread Xintong Song
e task should get at least the much resource, otherwise there should be an exception. That should be guaranteed by the "Dynamic Slot Allocation" approach (FLIP-56). I'll update the FLIP document addressing the comments ASAP. Thank you~ Xintong Song On Tue, Sep 3, 2019 at 2:4

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-04 Thread Xintong Song
way shuffle component use these memory (local buffer pool, netty internal memory, etc.) can be different depending on the shuffle implementation. The task executor (outside of the shuffle implementation) should only know the overall memory usage of the shuffle component but no need to understand mor

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-09-04 Thread Xintong Song
@all The FLIP document [1] has been updated. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management On Tue, Sep 3, 2019 at 7:20 PM Zhu Zhu wrote: > Thanks Xintong for the explanation. > > For question #1

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-04 Thread Xintong Song
use currently "taskmanager.network.memory" refers to shuffle buffers only, which is "one less config value to break". Thank you~ Xintong Song On Wed, Sep 4, 2019 at 3:42 PM Stephan Ewen wrote: > If we later split the network memory into "shuffle" and &qu

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-04 Thread Xintong Song
only direct memory and no native > memory. Is that correct? > Yes, this is what I mean. Thank you~ Xintong Song On Wed, Sep 4, 2019 at 4:25 PM Till Rohrmann wrote: > Just to clarify Xintong, you suggest that Task off-heap memory represents > direct and native memory. Since we don

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-09-05 Thread Xintong Song
Thanks all for joining the discussion. It seems to me that there is a consensus on the current FLIP document. So if there is no objection, I would like to start the voting process for this FLIP. Thank you~ Xintong Song On Wed, Sep 4, 2019 at 8:23 PM Andrey Zagrebin wrote: > Thanks

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-05 Thread Xintong Song
Thank you~ Xintong Song On Fri, Sep 6, 2019 at 2:43 AM Yu Li wrote: > +1 (non-binding) > > Minor: > 1. Is it worth a separate "Limitations" section to contain all relative > information like the Unsafe support issue in Java 12, just like many other > FLIPs? >

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-05 Thread Xintong Song
#x27; section, with the follow-ups of web ui and documentation issues - Add a 'Limitation' section, with the Unsafe Java 12 support issue Thank you~ Xintong Song On Fri, Sep 6, 2019 at 10:28 AM Xintong Song wrote: > +1 (non-binding) from my side. > > @Yu, thanks fo

[VOTE] FLIP-53: Fine Grained Operator Resource Management

2019-09-05 Thread Xintong Song
Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-53-Fine-Grained-Resource-Management-td31831.html

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

2019-09-07 Thread Xintong Song
Congratulations, Klou~ Thank you~ Xintong Song On Sat, Sep 7, 2019 at 2:44 PM Kurt Young wrote: > Congratulations Klou! > > Best, > Kurt > > > On Sat, Sep 7, 2019 at 2:37 PM ying wrote: > > > Congratulations Kostas! > > > > On Fr

Re: [DISCUSS] Features for Apache Flink 1.10

2019-09-07 Thread Xintong Song
Thanks Gray and Yu for compiling the feature list and kicking off this discussion. +1 for Gary and Yu being the release managers for Flink 1.10. Thank you~ Xintong Song On Sat, Sep 7, 2019 at 4:58 PM Till Rohrmann wrote: > Thanks for compiling the list of 1.10 efforts for the commun

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-09-11 Thread Xintong Song
True. It would be easier for the users to understand if JM and TM have corresponding config options. Maybe we can have another FLIP addressing this afterwards. Thank you~ Xintong Song On Mon, Sep 9, 2019 at 11:38 PM Stephan Ewen wrote: > One thing that I just came across: Some of th

Re: [VOTE] FLIP-53: Fine Grained Operator Resource Management

2019-09-11 Thread Xintong Song
+1 from my side. Thanks for bringing this up, Kurt. It's true that the accuracy loss of floating values could cause problems in some cases. And I would consider this as an implementation issue that can be discussed later in the PRs. Thank you~ Xintong Song On Sat, Sep 7, 2019 at 2:51 PM

Re: [VOTE] FLIP-53: Fine Grained Operator Resource Management

2019-09-11 Thread Xintong Song
FLIP-53 is approved to be adopted by Apache Flink. Thank you~ Xintong Song On Thu, Sep 12, 2019 at 10:31 AM Xintong Song wrote: > +1 from my side. > > Thanks for bringing this up, Kurt. It's true that the accuracy loss of > floating values could cause problems in some cases

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

2019-09-11 Thread Xintong Song
Congratulations Zili Thank you~ Xintong Song On Thu, Sep 12, 2019 at 11:03 AM Yun Tang wrote: > Congratulations Zili > > Best > Yun Tang > > From: Yun Gao > Sent: Thursday, September 12, 2019 10:12 > To: dev > Subject: Re: [AN

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-12 Thread Xintong Song
Added implementation steps for this FLIP on the wiki page [1]. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation On Tue, Aug 20, 2019 at 3:43 PM Xintong Song wrote: > @Zili > > As far as I know, Timo is drafting a

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-15 Thread Xintong Song
the shared slots. Thank you~ Xintong Song On Mon, Sep 16, 2019 at 10:42 AM wenlong.lwl wrote: > Hi, Xintong, thanks for the great proposal. big +1 for the feature! It is > something like mapreduce-1.0 to mapreduce-2.0. > > I like the design on the whole. One point may need to be

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-16 Thread Xintong Song
to put too much efforts on having a good abstraction and deduplication between the new code path and the original one that we are removing soon. What do you think? Thank you~ Xintong Song On Mon, Sep 16, 2019 at 5:59 PM Andrey Zagrebin wrote: > Hi Xintong, > > Thanks for sharing the i

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-16 Thread Xintong Song
ish Step 4 early, then it would make step 6 easier. We can start to have some IT/E2E tests, with the default slot resource profiles being available. Thank you~ Xintong Song On Mon, Sep 16, 2019 at 9:50 PM Andrey Zagrebin wrote: > @Xintong > > Thanks for the feedback. > > Just to

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-16 Thread Xintong Song
TaskExecutor to support dynamic slot allocation' - Add step for updating RestAPI / Web UI Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation On Tue, Sep 17, 2019 at 11:49 AM Xintong Song wrote: > @Till > Thanks for

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-17 Thread Xintong Song
Thanks for the feedback, Andrey. I'll start the vote. Thank you~ Xintong Song On Tue, Sep 17, 2019 at 10:09 PM Andrey Zagrebin wrote: > Thanks for the update @Xintong. > I would be ok with starting the vote. > > Best, > Andrey > > On Tue, Sep 17, 2019 at 6:1

[VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-17 Thread Xintong Song
Hi all, I would like to start the vote for FLIP-56 [1], on which a consensus is reached in this discussion thread [2]. The vote will be open for at least 72 hours. I'll try to close it after Sep. 20 15:00 UTC, unless there is an objection or not enough votes. Thank you~ Xintong Song

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-17 Thread Xintong Song
many computation loads to the task executor, rather than limit the cpu usage of each slot. Thank you~ Xintong Song On Wed, Sep 18, 2019 at 12:18 AM tao xiao wrote: > Sorry if I ask a question that has been addressed before. please point me > to the reference. > > How do we limit t

[DISCUSS] Planning Flink 2.0

2023-04-25 Thread Xintong Song
Hi everyone, I'd like to start a discussion on planning for a Flink 2.0 release. AFAIK, in the past years this topic has been mentioned from time to time, in mailing lists, jira tickets and offline discussions. However, few concrete steps have been taken, due to the significant determination and

Re: [DISCUSS] Planning Flink 2.0

2023-04-25 Thread Xintong Song
ps and collect feedback in time > Looking forward to the start of the work of Flink2.0, I am willing to > provide assistance ~ > > Xintong Song 于2023年4月25日周二 19:10写道: > > > > Hi everyone, > > > > I'd like to start a discussion on planning for a Flink 2.0 release. &g

Re: [DISCUSS] Planning Flink 2.0

2023-04-26 Thread Xintong Song
Other thoughts: > > > > > > We need to figure out what this release means for connectors > > > compatibility-wise. The current rules for which versions a connector > > > must support don't cover major releases at all. > > > (This depends a bit

Re: [DISCUSS] Planning Flink 2.0

2023-04-28 Thread Xintong Song
x and rely > on > > > > various third-party connectors written with legacy APIs, one thing > > that I > > > > have concerns about is if, one day in the future, the community > decides > > > > that new features are only given to 2.x releases, could the last > >

Re: [DISCUSS] Planning Flink 2.0

2023-04-28 Thread Xintong Song
d if there's no objections, we'll assemble the release management team as discussed, and try to figure out a proper way for the roadmap discussion next. Best, Xintong On Fri, Apr 28, 2023 at 3:43 PM Xintong Song wrote: > @Weike, > > Thanks for the suggestion. I think it makes sens

Re: [DISCUSS] Release Flink 1.16.2

2023-05-11 Thread Xintong Song
+1 I'll help with the steps that require PMC privileges. Best, Xintong On Thu, May 11, 2023 at 3:13 PM Jingsong Li wrote: > +1 for releasing 1.16.2 > > Best, > Jingsong > > On Thu, May 11, 2023 at 1:28 PM Gyula Fóra wrote: > > > > +1 for the release > > > > Gyula > > > > On Thu, 11 May 202

Re: [DISCUSS] Release Flink 1.17.1

2023-05-11 Thread Xintong Song
+1 I'll help with the steps that require PMC privileges. Best, Xintong On Thu, May 11, 2023 at 3:12 PM Jingsong Li wrote: > +1 for releasing 1.17.1 > > Best, > Jingsong > > On Thu, May 11, 2023 at 1:29 PM Gyula Fóra wrote: > > > > +1 for the release > > > > Gyula > > > > On Thu, 11 May 202

Re: [DISCUSS] Planning Flink 2.0

2023-05-12 Thread Xintong Song
features or architecture improvements to > > achieve > > > visions like streamhouse. This makes the 2.0 release be the 2.0 instead > > of > > > 1.x release, i.e. the real intention of 2.0 release with a significant > > > upgrade. > > > 2. History -

Re: [VOTE] Release 1.16.2, release candidate #1

2023-05-20 Thread Xintong Song
+1 (binding) - verified signatures and checksums - built from source - tried example jobs with a standalone cluster, everything seems fine - review release announcement PR Best, Xintong On Sat, May 20, 2023 at 6:04 PM Jing Ge wrote: > +1(non-binding) > > - reviewed Jira release notes > - bu

Re: [VOTE] Release 1.17.1, release candidate #1

2023-05-22 Thread Xintong Song
+1 (binding) - verified signatures and checksums - built from source - tried example jobs with a standalone cluster, everything seems fine - review release announcement PR Best, Xintong On Mon, May 22, 2023 at 2:18 PM weijie guo wrote: > Hi everyone, > > > Please review and vote on the rele

Release 2.0 Status Updates - 05/31/2023

2023-05-31 Thread Xintong Song
Hi devs, The release managers had some online discussions recently, and I'd like to post the summary here for transparency. We'd like to have 3 separate discussion tracks, so that each track can be more focused. - Jark will help drive a discussion on the project's long-term roadmap. Concre

[NOTICE] 1 week left for collecting release 2.0 proposals

2023-06-07 Thread Xintong Song
Hi everyone, Just want to kindly remind that, as mentioned in [1], we are trying to collect work items for release 2.0 by *June 15*, which is only one week from now. If there's anything you'd like to propose for the major release, please fill the chart on the wiki page [2]. If more time is needed,

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-14 Thread Xintong Song
Thanks for bringing up this discussion, Becket. My two cents: 1. Do we allow deprecation & removal of APIs without adding a new one as a replacement? The examples in the table give me an impression that marking an API as `@Deprecated` should only happen at the same time of introducing a new repla

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-14 Thread Xintong Song
> > imagine that the behavioral stability Stefan mentions actually could > > > become > > > > a bigger issue: Major versions down the road might include bigger > > > > behavioral changes which would prevent users from going ahead with > the > > > >

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-15 Thread Xintong Song
> > Public API is a well defined common concept, and one of its > convention is that it only changes with a major version change. > I agree. And from my understanding, demoting a Public API is also a kind of such change, just like removing one, which can only happen with major version bumps. I'm n

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-19 Thread Xintong Song
ually I think there are some dependency version resolution > policies > >> out > >> >> there which picks the highest minor version when the dependencies > pull > >> in > >> >> multiple minor versions of the same jar, which may be broken if we

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-19 Thread Xintong Song
If it doesn't work, I'd > > be in favor of not providing the migration period, but fallback to only > > guarantee the compatibility within the major version. > > The part I don't understand is if we are willing to have a migration > period, and do a minor version bum

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-20 Thread Xintong Song
xt major version. This indicates > that > > the actual problem we need to solve is to lower the maintenance overhead > of > > deprecated APIs, so that we are comfortable to keep them longer. As John > > and I mentioned earlier, there are ways to achieve this and we need to > &

[DISCUSS] Release 2.0 Work Items

2023-06-20 Thread Xintong Song
Hi devs, As previously discussed in [1], we had been collecting work item proposals for the 2.0 release until June 15th, on the wiki page [2]. - As we have passed the due date, I'd like to kindly remind everyone *not to add / remove items directly on the wiki page*. If needed, please post

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-26 Thread Xintong Song
Scenario 3: @PublicEvolving deprecation for users who ignore > > > > @deprecated > > > > > > annotation changes > > > > > > > > > > > > Sample API marked as deprecated in 1.19, kept in 1.20, removed in > > > 1.21 > > > > &g

Re: [DISCUSS] Release 2.0 Work Items

2023-06-26 Thread Xintong Song
ause it's > very unclear what it actually entails; like is it an entirely separate API > to DataStream (sounds like it is!) or an extension of DataStream. How much > will it share the internals with DataStream etc.; how does it relate to the > Table API (w.r.t. switching APIs / what T

Re: [DISCUSS] Release 2.0 Work Items

2023-06-27 Thread Xintong Song
has been > > previously discussed on the mailing list. Do we have a list of > shortcomings > > in the current DataStream API that it tries to resolve? How does the > > current ProcessFunction functionality fit into the picture? Will it be > kept > > as is or subsumed by new API

[DISUCSS] Deprecate multiple APIs in 1.18

2023-06-29 Thread Xintong Song
Hi devs, Looking at the release 2.0 proposals [1], I noticed that many APIs that are proposed to be removed in 2.0 are not (fully) deprecated yet. We might want to properly mark them as `@Deprecated` in 1.18 if we agree they should be removed in 2.0. Moreover, according to FLIP-321 [2] (not voted

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-06-29 Thread Xintong Song
t's quite well known that they are deprecated. I'm +0 for either way, > > as long as we're all agreeing that they can be removed in 2.0. > > > > With regards to Queryable State and Source/SinkFunction, +1 to mark > > these as deprecated. > > > > Best reg

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-06-29 Thread Xintong Song
wse/FLINK-28050 > [4] https://issues.apache.org/jira/browse/FLINK-28229 > [5] https://issues.apache.org/jira/browse/FLINK-28048 > [6] https://issues.apache.org/jira/browse/FLINK-28054 > [7] https://issues.apache.org/jira/browse/FLINK-28051 > [8] https://issues.apache.org/jira/browse/FLI

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-06-29 Thread Xintong Song
ee a big issue > here. > > >Do you think it is feasible to resolve them by the feature freeze date of > 1.18? > No, these are rather complex additions that would probably require FLIP(s). > > [1] > > https://flink.apache.org/2022/01/20/pravega-flink-connector-101/#checkp

Re: [VOTE] FLIP-321: introduce an API deprecation process

2023-07-03 Thread Xintong Song
+1 (binding) Best, Xintong On Sat, Jul 1, 2023 at 11:26 PM Dong Lin wrote: > Thanks for the FLIP. > > +1 (binding) > > On Fri, Jun 30, 2023 at 5:39 PM Becket Qin wrote: > > > Hi folks, > > > > I'd like to start the VOTE for FLIP-321[1] which proposes to introduce an > > API deprecation proc

Re: [DISCUSS] Release 2.0 Work Items

2023-07-03 Thread Xintong Song
;>>> "Move Calcite rules from Scala to Java": I would hope that this would > >> be > >>>>> an entirely internal change, and could thus be an incremental process > >>>>> independent of major releases. > >>>>> What is

[ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

2023-07-03 Thread Xintong Song
Dear Community, I'm pleased to share this good news with everyone. As some of you may have already heard, Apache Flink has won the 2023 SIGMOD Systems Award [1]. "Apache Flink greatly expanded the use of stream data-processing." -- SIGMOD Awards Committee SIGMOD is one of the most influential da

Re: [DISCUSS] Release 2.0 Work Items

2023-07-04 Thread Xintong Song
> >>> > > >>> Best, > > >>> Jark > > >>> > > >>> > > >>> On Wed, 28 Jun 2023 at 14:10, Sergey Nuyanzin > > >> wrote: > > >>> > > >>>> Hi Chesnay &

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-07-04 Thread Xintong Song
onnectors can > upgrade to Flink 2.0, since we moved forward with SinkV2 API without taking > care of implementations in external connectors. > > I am ok with both of them and personally prefer option 1. > > Best regards, > Jing > > > On Fri, Jun 30, 2023 at 3:41 AM Xintong

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-07-04 Thread Xintong Song
ys. And if there's no objections, I'll create JIRA tickets accordingly. Best, Xintong On Wed, Jul 5, 2023 at 1:34 PM Xintong Song wrote: > Thanks for the input, Jing. I'd also be +1 for option 1. > > Best, > > Xintong > > > > On Mon, Jul 3, 2023 at 7:20

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-07-05 Thread Xintong Song
n Wed, 5 Jul 2023 at 10:47, Chesnay Schepler wrote: > > > There's a whole bunch of metric APIs that would need to be deprecated. > > That is of course if the metric FLIPs are being accepted. > > > > Which makes me wonder if we aren't doing things the wrong way around

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-07-07 Thread Xintong Song
> > On Wed, 5 Jul 2023 at 10:47, Chesnay Schepler > wrote: > > > > > There's a whole bunch of metric APIs that would need to be deprecated. > > > That is of course if the metric FLIPs are being accepted. > > > > > > Which makes me won

Re: [DISCUSS] Release 2.0 Work Items

2023-07-07 Thread Xintong Song
Thanks all for the discussion. The wiki has been updated as discussed. I'm starting a vote now. Best, Xintong On Wed, Jul 5, 2023 at 9:52 AM Xintong Song wrote: > Hi ConradJam, > > I think Chesnay has already put his name as the Contributor for the two > tasks you list

[VOTE] Release 2.0 must-have work items

2023-07-07 Thread Xintong Song
Hi all, I'd like to start the VOTE for the must-have work items for release 2.0 [1]. The corresponding discussion thread is [2]. Please note that once the vote is approved, any changes to the must-have items (adding / removing must-have items, changing the priority) requires another vote. Assigni

Re: [DISCUSS] Release 2.0 Work Items

2023-07-10 Thread Xintong Song
yone assigned. Were these items discussed among > the release managers? So far, it looks like they are handled as > nice-to-have if someone volunteers to pick them up? > > My concern is that they will be overlooked because nobody feels to be in > charge. > > Best, > Matthias &g

Re: [DISCUSS] Release 2.0 Work Items

2023-07-10 Thread Xintong Song
1.18 is released so we can deprecate affected > APIs. > > On 10/07/2023 11:32, Xintong Song wrote: > > Hi Matthias, > > > > The questions you asked are indeed very important. Here're some quick > > responses, based on the plans I had in mind, which I have not al

Re: [DISCUSS] Release 2.0 Work Items

2023-07-10 Thread Xintong Song
on work (simply adding annotations). > WDYT? > > > > I'm also changing the priority of "Clarify the scopes of configuration > > options" > > to nice to have. I think most of the work are not breaking changes and > can > > be done in 1.x or 2.

Re: [DISUCSS] Deprecate multiple APIs in 1.18

2023-07-10 Thread Xintong Song
; > > > > > > I may not have understood all the details, but based on what you > > > described > > > > I'd hesitate to block the deprecation / removal of SourceFunction on > > > this. > > > > > > I don't think we should, just wa

Re: [VOTE] Release 2.0 must-have work items

2023-07-11 Thread Xintong Song
gt; > > >> +1 > >> > >> On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > >> > >>> +1 (binding) > >>> > >>> Thanks for driving this and great to see us moving forward. > >>> > >>> Best Regards, > >

Re: [DISCUSS] FLIP 333 - Redesign Apache Flink website

2023-07-11 Thread Xintong Song
+1 in general. Thanks for proposing this contribution, Deepthi. The new design looks very cool. I have a few questions, which might be entry-level given that I barely know anything about the website design. - Do you think it's feasible to maintain two sets of website designs at the same time? E.g

  1   2   3   4   5   6   7   8   9   10   >