Good idea to go ahead with 1.10.1 About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job" - I don't think we have consensus on the exact solution, yet, and some of the changes might also have side effects that are hard to predict, so I am not sure we should rush this in.
About "FLINK-16406 Increase default value for JVM Metaspace" - Have we consensus that this is okay for a bugfix release? It changes setups, takes away memory from heap / managed memory on existing setups that keep their flink-conf.yaml. We may need to unblock the release form these two issues and think about having 1.10.2 in the near future. On Thu, Mar 12, 2020 at 7:15 AM Yu Li <car...@gmail.com> wrote: > Thanks for the reminder Jark. Will keep an eye on these two. > > Best Regards, > Yu > > > On Thu, 12 Mar 2020 at 12:32, Jark Wu <imj...@gmail.com> wrote: > > > Thanks for driving this release, Yu! > > +1 to start 1.10.1 release cycle. > > > > From the Table SQL module, I think we should also try to get in the > > following issues: > > - FLINK-16441: Allow users to override flink-conf parameters from SQL CLI > > environment > > this allows users to set e.g. statebackend, watermark interval, > > exactly-once/at-least-once, in the SQL CLI > > - FLINK-16217: SQL Client crashed when any uncatched exception is thrown > > this will improve much experience when using SQL CLI > > > > Best, > > Jark > > > > > > On Wed, 11 Mar 2020 at 20:37, Yu Li <car...@gmail.com> wrote: > > > > > Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225 > fix > > > versions and promoted its priority to Critical. Will also watch the > > > progress of FLINK-16108/FLINK-16408. > > > > > > Best Regards, > > > Yu > > > > > > > > > On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <azagre...@apache.org> > > > wrote: > > > > > > > Hi Yu, > > > > > > > > Thanks for kicking off the 1.10.1 release discussion! > > > > > > > > Apart from > > > > - FLINK-16406 Increase default value for JVM Metaspace to minimise > its > > > > OutOfMemoryError > > > > which should be merged soon > > > > > > > > I think we should also try to get in the following issues: > > > > > > > > - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal > > Error > > > in > > > > TaskManager > > > > This should solve the Metaspace problem even in a better way because > > OOM > > > > failure should point users to the docs immediately > > > > > > > > - [FLINK-16408] Bind user code class loader to lifetime of a slot > > > > This should give a better protection against class loading leaks > > > > > > > > - [FLINK-16018] Improve error reporting when submitting batch job > > > (instead > > > > of AskTimeoutException) > > > > This problem has recently happened for multiple users > > > > > > > > Best, > > > > Andrey > > > > > > > > > > > > On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <jingsongl...@gmail.com> > > > > wrote: > > > > > > > > > Thanks for driving. Yu. +1 for starting the 1.10.1 release. > > > > > > > > > > Some issues are very important, Users are looking forward to them. > > > > > > > > > > Best, > > > > > Jingsong Lee > > > > > > > > > > On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <karma...@gmail.com> > > wrote: > > > > > > > > > > > Thanks for driving this release, Yu! > > > > > > > > > > > > +1 for starting the 1.10.1 release cycle. > > > > > > > > > > > > Best, > > > > > > Yangze Guo > > > > > > > > > > > > On Wed, Mar 11, 2020 at 1:42 PM Xintong Song < > > tonysong...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > Yu, > > > > > > > Thanks for the explanation. > > > > > > > I've no concerns. I was just trying to get some inputs for > > > > prioritizing > > > > > > > tasks on my side, and ~1month sounds good to me. > > > > > > > > > > > > > > > > > > > > > Thank you~ > > > > > > > > > > > > > > Xintong Song > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 11, 2020 at 12:15 PM Yu Li <car...@gmail.com> > wrote: > > > > > > > > > > > > > > > bq. what is the time plan for 1.10.1? > > > > > > > > > > > > > > > > According to the history, the first patch release of a major > > > > version > > > > > > will > > > > > > > > take ~1month from discussion started, depending on the speed > of > > > > > blocker > > > > > > > > issue resolving: > > > > > > > > * 1.8.1: started discussion on May 28th [1], released on Jul > > 3rd > > > > [2] > > > > > > > > * 1.9.1: started discussion on Sep 23rd [3], released on Oct > > 19th > > > > [4] > > > > > > > > > > > > > > > > We won't rush to match the history of course, but could use > it > > > as a > > > > > > > > reference. And please feel free to let me know if any > concerns > > > > > Xintong. > > > > > > > > Thanks. > > > > > > > > > > > > > > > > Best Regards, > > > > > > > > Yu > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html > > > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html > > > > > > > > [4] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 11 Mar 2020 at 11:54, Xintong Song < > > > tonysong...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks Yu, for the kick off and volunteering to be the > > release > > > > > > manager. > > > > > > > > > > > > > > > > > > +1 for the proposal. > > > > > > > > > > > > > > > > > > > > > > > > > > > One quick question, what is the time plan for 1.10.1? > > > > > > > > > > > > > > > > > > > > > > > > > > > Thank you~ > > > > > > > > > > > > > > > > > > Xintong Song > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang > > > > > > > > > <wangzhijiang...@aliyun.com.invalid> wrote: > > > > > > > > > > > > > > > > > > > Thanks for driving this release, Yu! > > > > > > > > > > +1 on my side > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > Zhijiang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > > > > > > > > > From:Yu Li <car...@gmail.com> > > > > > > > > > > Send Time:2020 Mar. 10 (Tue.) 20:25 > > > > > > > > > > To:dev <dev@flink.apache.org> > > > > > > > > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1 > > > > > > > > > > > > > > > > > > > > Thanks for the supplement Hequn. Yes will also keep an > eye > > on > > > > > these > > > > > > > > > > existing blocker issues. > > > > > > > > > > > > > > > > > > > > Best Regards, > > > > > > > > > > Yu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng < > > he...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Yu, > > > > > > > > > > > > > > > > > > > > > > Thanks a lot for raising the discussion and volunteer > as > > > the > > > > > > release > > > > > > > > > > > manager! > > > > > > > > > > > > > > > > > > > > > > I found there are some other issues[1] which are marked > > as > > > a > > > > > > blocker: > > > > > > > > > > > - FLINK-16454 Update the copyright year in NOTICE files > > > > > > > > > > > - FLINK-16262 Class loader problem with > > > > > > > > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib > > > directory > > > > > > > > > > > - FLINK-16170 SearchTemplateRequest > > ClassNotFoundException > > > > when > > > > > > use > > > > > > > > > > > flink-sql-connector-elasticsearch7 > > > > > > > > > > > - FLINK-16018 Improve error reporting when submitting > > batch > > > > job > > > > > > > > > (instead > > > > > > > > > > of > > > > > > > > > > > AskTimeoutException) > > > > > > > > > > > > > > > > > > > > > > These may also need to be resolved in 1.10.1. > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > Hequn > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li < > car...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Jincheng, > > > > > > > > > > > > > > > > > > > > > > > > Yes, your help would be very helpful. Thanks a lot! > > > > > > > > > > > > > > > > > > > > > > > > Best Regards, > > > > > > > > > > > > Yu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun < > > > > > > > > sunjincheng...@gmail.com > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for bring up the discussion Yu. I would like > > to > > > > give > > > > > > you a > > > > > > > > > > hand > > > > > > > > > > > at > > > > > > > > > > > > > the last stage when the RC is finished.(If you > need) > > > :) > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > Jincheng > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yu Li <car...@gmail.com> 于2020年3月10日周二 下午5:49写道： > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > > > > > > > > > > > > > It has been almost one month since we released > > Flink > > > > > > 1.10.0. We > > > > > > > > > > > already > > > > > > > > > > > > > > have more than 40 resolved improvements/bugs in > the > > > > > > > > release-1.10 > > > > > > > > > > > > branch, > > > > > > > > > > > > > > and I propose to start the 1.10.1 release cycle. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Most noticeable fixes are: > > > > > > > > > > > > > > > > > > > > > > > > > > > > - FLINK-16241 [legal] Remove the license and > notice > > > > file > > > > > in > > > > > > > > > > > > flink-ml-lib > > > > > > > > > > > > > > module > > > > > > > > > > > > > > - FLINK-16313 Fix RocksDB resource leak in > > > > > > > > > > flink-state-processor-api > > > > > > > > > > > > > > - FLINK-16161 Statistics zero should be known in > > > > > > HiveCatalog > > > > > > > > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in > > > > > TypeExtractor > > > > > > > > when > > > > > > > > > > > > mapping > > > > > > > > > > > > > > - FLINK-16108 StreamSQLExample is failed if > running > > > in > > > > > > blink > > > > > > > > > > planner > > > > > > > > > > > > > > - FLINK-16139 Co-location constraints are not > reset > > > on > > > > > task > > > > > > > > > > recovery > > > > > > > > > > > in > > > > > > > > > > > > > > DefaultScheduler > > > > > > > > > > > > > > - FLINK-16414 Create udaf/udtf function using sql > > > > casuing > > > > > > > > > > > > > > ValidationException: SQL validation failed > > > > > > > > > > > > > > > > > > > > > > > > > > > > Furthermore, I think the following issues should > be > > > > > merged > > > > > > > > before > > > > > > > > > > > > 1.10.1 > > > > > > > > > > > > > > release (especially the Metaspace OOM issue): > > > > > > > > > > > > > > > > > > > > > > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM > > error > > > on > > > > > > > > repeated > > > > > > > > > > job > > > > > > > > > > > > > > submission > > > > > > > > > > > > > > - FLINK-16406 Increase default value for JVM > > > Metaspace > > > > to > > > > > > > > > minimise > > > > > > > > > > > its > > > > > > > > > > > > > > OutOfMemoryError > > > > > > > > > > > > > > - FLINK-16047 Blink planner produces wrong > > aggregate > > > > > > results > > > > > > > > with > > > > > > > > > > > state > > > > > > > > > > > > > > clean up > > > > > > > > > > > > > > - FLINK-16070 Blink planner can not extract > correct > > > > > unique > > > > > > key > > > > > > > > > for > > > > > > > > > > > > > > UpsertStreamTableSink > > > > > > > > > > > > > > > > > > > > > > > > > > > > I would volunteer as the release manager and kick > > off > > > > the > > > > > > > > release > > > > > > > > > > > > process > > > > > > > > > > > > > > once blocker issues are merged. What do you > think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > If there are any concerns or missing blocker > issues > > > > need > > > > > > to be > > > > > > > > > > fixed > > > > > > > > > > > in > > > > > > > > > > > > > > 1.10.1, please let me know. Thanks. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards, > > > > > > > > > > > > > > Yu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best, Jingsong Lee > > > > > > > > > > > > > > >
