Hi all I have drafted the docs of top-level FLIPs for the individual changes proposed in FLIP-75. will update it to the cwiki page and start the voting stage soon if there is no objection.
- FLIP-98: Better Back Pressure Detection <https://docs.google.com/document/d/1b4GadCze-36x5TPHz6ie4WI9fOUuxWoT_rWWgeg68oo/edit?usp=sharing> - FLIP-99: Make Max Exception Configurable <https://docs.google.com/document/d/1tsPpTEx5WqliOAUC924xzRxYOalUuB-GoznGPcxSzJo/edit?usp=sharing> - FLIP-100: Add Attempt Information <https://docs.google.com/document/d/1Ww7biOr6WMVfoYhtBTJftRqEm9FGo33AXgYibdXy47Y/edit?usp=sharing> - FLIP-101: Add Pending Slots Tab in Job Detail <https://docs.google.com/document/d/1ttn7zIn_Z237JOHdmhiei6aCwKdjTU53I07XxA61Fis/edit?usp=sharing> - FLIP-102: Add More Metrics to TaskManager <https://docs.google.com/document/d/18yHdsqUJ1FmNRm0hyeCm3nWvPFpvpJgTJ8BYNAa6Ul8/edit?usp=sharing> - FLIP-103: Better Taskmanager Log Display <https://docs.google.com/document/d/16eEdW2KeLxvABdoXahx4MMMisW4_P9mKiqUE0F4GO1c/edit?usp=sharing> - FLIP-104: Add More Metrics to Jobmanager <https://docs.google.com/document/d/1Fak632iOroOLZFADqwZWu2SS-LUQqCLHnm8Vs3XM5to/edit?usp=sharing> - FLIP-105: Better Jobmanager Log Display <https://docs.google.com/document/d/1ayXaZflelaymQuF3l6UOuGEg6zzbSGGewoDq-9SBOPY/edit?usp=sharing> Yadong Xie <vthink...@gmail.com> 于2020年2月9日周日 下午7:24写道： > Hi Till > I got your point, will create sub FLIPs and votings according to the > FLIP-75 and previous discussion soon. > > Till Rohrmann <trohrm...@apache.org> 于2020年2月9日周日 下午5:27写道： > >> Hi Yadong, >> >> I think it would be fine to simply link to this discussion thread to keep >> the discussion history. Maybe an easier way would be to create top-level >> FLIPs for the individual changes proposed in FLIP-75. The reason I'm >> proposing this is that it would be easier to vote on it and to implement >> it >> because the scope is smaller. But maybe I'm wrong here and others could >> chime in to voice their opinion. >> >> Cheers, >> Till >> >> On Fri, Feb 7, 2020 at 9:58 AM Yadong Xie <vthink...@gmail.com> wrote: >> >> > Hi Till >> > >> > FLIP-75 has been open since September, and the design doc has been >> iterated >> > over 3 versions and more than 20 patches. >> > I had a try, but it is hard to split the design docs into sub FLIP and >> keep >> > all the discussion history at the same time. >> > >> > Maybe it is better to start another discussion to talk about the >> individual >> > sub FLIP voting? and make the next FLIP follow the new practice if >> > possible. >> > >> > Till Rohrmann <trohrm...@apache.org> 于2020年2月3日周一 下午6:28写道： >> > >> > > I think there is no such description because we never did it before. I >> > just >> > > figured that FLIP-75 could actually be a good candidate to start this >> > > practice. We would need a community discussion first, though. >> > > >> > > Cheers, >> > > Till >> > > >> > > On Mon, Feb 3, 2020 at 10:28 AM Yadong Xie <vthink...@gmail.com> >> wrote: >> > > >> > > > Hi Till >> > > > I didn’t find how to create of sub flip at cwiki.apache.org >> > > > do you mean to create 9 more FLIPS instead of FLIP-75? >> > > > >> > > > Till Rohrmann <trohrm...@apache.org> 于2020年1月30日周四 下午11:12写道： >> > > > >> > > > > Would it be easier if FLIP-75 would be the umbrella FLIP and we >> would >> > > > vote >> > > > > on the individual improvements as sub FLIPs? Decreasing the scope >> > > should >> > > > > make things easier. >> > > > > >> > > > > Cheers, >> > > > > Till >> > > > > >> > > > > On Thu, Jan 30, 2020 at 2:35 PM Robert Metzger < >> rmetz...@apache.org> >> > > > > wrote: >> > > > > >> > > > > > Thanks a lot for this work! I believe the web UI is very >> important, >> > > in >> > > > > > particular to new users. I'm very happy to see that you are >> putting >> > > > > effort >> > > > > > into improving the visibility into Flink through the proposed >> > > changes. >> > > > > > >> > > > > > I can not judge if all the changes make total sense, but the >> > > discussion >> > > > > has >> > > > > > been open since September, and a good number of people have >> > commented >> > > > in >> > > > > > the document. >> > > > > > I wonder if we can move this FLIP to the VOTing stage? >> > > > > > >> > > > > > On Wed, Jan 22, 2020 at 6:27 PM Till Rohrmann < >> > trohrm...@apache.org> >> > > > > > wrote: >> > > > > > >> > > > > > > Thanks for the update Yadong. Big +1 for the proposed >> > improvements >> > > > for >> > > > > > > Flink's web UI. I think they will be super helpful for our >> users. >> > > > > > > >> > > > > > > Cheers, >> > > > > > > Till >> > > > > > > >> > > > > > > On Tue, Jan 7, 2020 at 10:00 AM Yadong Xie < >> vthink...@gmail.com> >> > > > > wrote: >> > > > > > > >> > > > > > > > Hi everyone >> > > > > > > > >> > > > > > > > We have spent some time updating the documentation since the >> > last >> > > > > > > > discussion. >> > > > > > > > >> > > > > > > > In short, the latest FLIP-75 contains the following >> > > > > proposal(including >> > > > > > > both >> > > > > > > > frontend and RestAPI) >> > > > > > > > >> > > > > > > > 1. Job Level >> > > > > > > > - better job backpressure detection >> > > > > > > > - load more feature in job exception >> > > > > > > > - show attempt history in the subtask >> > > > > > > > - show attempt timeline >> > > > > > > > - add pending slots >> > > > > > > > 2. Task Manager Level >> > > > > > > > - add more metrics >> > > > > > > > - better log display >> > > > > > > > 3. Job Manager Level >> > > > > > > > - add metrics tab >> > > > > > > > - better log display >> > > > > > > > >> > > > > > > > To help everyone better understand the proposal, we spent >> > efforts >> > > > on >> > > > > > > making >> > > > > > > > an online POC <http://101.132.122.69:8081/web/#/overview>. >> > > > > > > > >> > > > > > > > Now you can compare the difference between the new and old >> > > > > Web/RestAPI >> > > > > > > (the >> > > > > > > > link is inside the doc)! >> > > > > > > > >> > > > > > > > Here is the latest FLIP-75 doc: >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit# >> > > > > > > > >> > > > > > > > Looking forward to your feedback >> > > > > > > > >> > > > > > > > >> > > > > > > > Best, >> > > > > > > > Yadong >> > > > > > > > >> > > > > > > > lining jing <jinglini...@gmail.com> 于2019年10月24日周四 >> 下午2:11写道： >> > > > > > > > >> > > > > > > > > Hi all, I have updated the backend design in FLIP-75 >> > > > > > > > > < >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit?usp=sharing >> > > > > > > > > > >> > > > > > > > > . >> > > > > > > > > >> > > > > > > > > Here are some brief introductions: >> > > > > > > > > >> > > > > > > > > - Add metric for manage memory FLINK-14406 >> > > > > > > > > <https://issues.apache.org/jira/browse/FLINK-14406>. >> > > > > > > > > - Expose TaskExecutor resource configurations to REST >> API >> > > > > > > FLINK-14422 >> > > > > > > > > <https://issues.apache.org/jira/browse/FLINK-14422>. >> > > > > > > > > - Add TaskManagerResourceInfo in >> TaskManagerDetailsInfo to >> > > > show >> > > > > > > > > TaskManager Resource FLINK-14435 >> > > > > > > > > <https://issues.apache.org/jira/browse/FLINK-14435>. >> > > > > > > > > >> > > > > > > > > I will continue to update the rest part of the backend >> design >> > > in >> > > > > the >> > > > > > > doc, >> > > > > > > > > let's keep discuss here, any feedback is appreciated. >> > > > > > > > > >> > > > > > > > > Yadong Xie <vthink...@gmail.com> 于2019年9月27日周五 上午10:13写道： >> > > > > > > > > >> > > > > > > > > > Hi all >> > > > > > > > > > >> > > > > > > > > > Flink Web UI is the main platform for most users to >> monitor >> > > > their >> > > > > > > jobs >> > > > > > > > > and >> > > > > > > > > > clusters. We have reconstructed Flink web in 1.9.0 >> version, >> > > but >> > > > > > there >> > > > > > > > are >> > > > > > > > > > still some shortcomings. >> > > > > > > > > > >> > > > > > > > > > This discussion thread aims to provide a better >> experience >> > > for >> > > > > > Flink >> > > > > > > UI >> > > > > > > > > > users. >> > > > > > > > > > >> > > > > > > > > > Here is the design doc I drafted: >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit?usp=sharing >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > The FLIP can be found at [2]. >> > > > > > > > > > >> > > > > > > > > > Please keep the discussion here, in the mailing list. >> > > > > > > > > > >> > > > > > > > > > Looking forward to your opinions, any feedbacks are >> > welcome. >> > > > > > > > > > >> > > > > > > > > > [1]: >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit?usp=sharing >> > > > > > > > > > < >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit# >> > > > > > > > > > > >> > > > > > > > > > [2]: >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-75%3A+Flink+Web+UI+Improvement+Proposal >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
