i updated this JIRA already, feel free to reuse this: https://issues.apache.org/jira/browse/FLINK-10706
-- *Fabian WollertZalando SE* E-Mail: fab...@zalando.de Am Di., 6. Nov. 2018 um 12:10 Uhr schrieb Shaoxuan Wang <wshaox...@gmail.com >: > Till, > Yes, it is a good idea to have a feature flag to switch the web UI before > we completely deprecate the old one. > > Yadong, > It seems that everyone likes the new web UI. Can you please open a master > Jira and start to merge the code to Flink master. What do you think? > > Regards, > Shaoxuan > > On Mon, Nov 5, 2018 at 7:04 PM Till Rohrmann <trohrm...@apache.org> wrote: > > > Thanks a lot for sharing the code with the community Yadong! > > > > It looks really cool and I also want to give it a try to see how easy it > is > > to start Flink with it. > > > > If it is already implemented and working, we could also think about > adding > > it to Flink and add a feature flag to switch between the old and new web > > UI. We could think about enabling it by default to give it more user > > exposure. After being confident and users having no complaints, we could > > think about deprecating the old web UI and eventually to drop it. Of > > course, initially we should give it a thorough review. > > > > Cheers, > > Till > > > > On Mon, Nov 5, 2018 at 8:40 AM Fabian Wollert <fab...@zalando.de> wrote: > > > > > Hi Yadong, this is awesome, thx for the code! I will try it out on our > > > infrastructure and will post my feedback here, latest next week. > > > > > > I will also check if my ideas for FLINK-10707 > > > <https://issues.apache.org/jira/browse/FLINK-10707> are doable with > your > > > code since this was what pushed this discussion initially. > > > > > > Cheers > > > > > > -- > > > > > > > > > *Fabian WollertZalando SE* > > > > > > E-Mail: fab...@zalando.de > > > > > > > > > Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie < > > vthink...@gmail.com > > > >: > > > > > > > Hi Fabian, Till, and Robert > > > > > > > > Thank you for your attention to this matter, I just push our codes to > > > > github: https://github.com/vthinkxie/flink-runtime-web. > > > > > > > > You can start the project by following the guidelines > > > > < > https://github.com/vthinkxie/flink-runtime-web#development--debugging > > > > > > > (just > > > > run `npm install && npm run proxy`), just feel free to give any > > comments > > > :) > > > > > > > > If I missed anything please let me know. Look forward to your > feedback > > > and > > > > suggestions soon. > > > > > > > > Best regards > > > > Yadong > > > > > > > > > > > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fab...@zalando.de> > > wrote: > > > > > > > > > Hi Yadong, this looks awesome. is there any chance you can already > > > share > > > > > the code of the new web UI, so we can take a look at what you guys > > > build > > > > > there? I think that would speed up the discussion. If there is > > already > > > a > > > > > fully fledged new Version with everything updated out there, and > its > > > even > > > > > battle tested in production already, that sounds like the way to go > > for > > > > me. > > > > > > > > > > i started aside from this discussion here (to strengthen and learn > > some > > > > > new React stuff) my attempt on the React version already, for > whoever > > > is > > > > > curious, you can check it out here: > > > > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base > > url > > > in > > > > > actions.js > > > > > < > > > > > > > > > > https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8 > > > > >, > > > > > npm install and then npm start) ... i just started to convert the > > first > > > > > simple pages, so dont expect the whole UI yet, its just a start ... > > > > > > > > > > Cheers > > > > > > > > > > -- > > > > > > > > > > > > > > > *Fabian WollertZalando SE* > > > > > > > > > > E-Mail: fab...@zalando.de > > > > > > > > > > > > > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang < > > zjf...@gmail.com > > > >: > > > > > > > > > >> +1 for the proposal of Yadong. I'd like to add one more comment: > > > > >> > > > > >> We should provide more development guidance when developing the > next > > > > >> dashboard via angular 7. As Till mentioned before, one big concern > > is > > > > lack > > > > >> of frontend guys in flink community, I believe more development > > > guidance > > > > >> will attract more frontend guys to contribute to Flink web ui, and > > > also > > > > be > > > > >> helpful for backend guys who want to add something to dashboard. > > > > >> > > > > >> > > > > >> > > > > >> Shaoxuan Wang <wshaox...@gmail.com>于2018年11月2日周五 上午10:06写道： > > > > >> > > > > >> > +1 to keep the web UI stack to Angular. This will make the stack > > > > >> upgrading > > > > >> > much easier and won't break the on-going efforts already made > > > towards > > > > >> the > > > > >> > improvements of Flink Web UI. > > > > >> > > > > > >> > Till, Robert, and Fabian, > > > > >> > Please take a look at the upgrading plan from Angular 1.x to 7.0 > > > > >> proposed > > > > >> > by Yadong. Any feedback is appreciated. > > > > >> > > > > > >> > Regards, > > > > >> > Shaoxuan > > > > >> > > > > > >> > > > > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vthink...@gmail.com> > > > > wrote: > > > > >> > > > > > >> > > For your interest, here are some snapshots of Flink web UI > (with > > > > >> angular > > > > >> > > 7.0) we recently reconstructed. > > > > >> > > [image: snapshot.001.jpeg] > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing > > > > >> > > > > > > >> > > We choose Angular 7.0 due to the following reasons: > > > > >> > > > > > > >> > > 1. It is easy to migrate the current codebase to Angular > 7.0 > > > > >> following > > > > >> > > the official guide. > > > > >> > > 2. It is easy for current Web UI developers who are > > experienced > > > > >> with > > > > >> > > Angular 1.x to get familiar with 7.0 reading the official > > doc. > > > > >> > > 3. Migrating other than rewriting is helpful to ensure its > > > > >> functional > > > > >> > > integrity. > > > > >> > > 4. There are over 1 million Angular developers now (2.0 and > > > > above, > > > > >> > > still increasing rapidly), which means there are > potentially > > > more > > > > >> > > developers willing to join the community. > > > > >> > > 5. No bower or complex node / webpack configuration needed > > > (only > > > > >> > > angular cli). > > > > >> > > > > > > >> > > We split Angular 1.x and 7.0 versions into two folders based > on > > > the > > > > >> > > following consideration: > > > > >> > > > > > > >> > > 1. 1.x version could be the default page because there is > no > > > > >> > > breakchange for both users and developers. > > > > >> > > 2. Make the upgrade progressive. Users can switch between > two > > > > >> versions > > > > >> > > by clicking the button on the top right corner. > > > > >> > > 3. Reduce maintenance costs and keep the performance of web > > > > without > > > > >> > > bootstrap both 1.x and 7.0 versions in the same page. > > > > >> > > 4. Won't interrupt the current development process. The > code > > > can > > > > >> still > > > > >> > > be submitted to 1.x version folder, we could help to sync > all > > > > >> > features and > > > > >> > > bug fix to 7.0 version if needed. > > > > >> > > 5. When the 7.0 version is ready and stable, it is easy to > > > change > > > > >> > > default page to it, and remove the old version. > > > > >> > > > > > > >> > > [image: test.jpg] > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing > > > > >> > > > > > > >> > > Here is the brief list of what we have done: > > > > >> > > > > > > >> > > 1. Create a new folder named next-web-dashboard in > > > > >> flink-runtime-web. > > > > >> > > 2. Migrate all Angular 1.x controllers, services, config > etc > > > one > > > > by > > > > >> > > one into 7.0 version with ngMigration. > > > > >> > > 3. Modify frontend build scripts in order to package two > > > versions > > > > >> > > together. > > > > >> > > 4. All works listed above are limited in the > > flink-runtime-web > > > > >> folder. > > > > >> > > One folder was added and no file or folder was deleted. > > > > >> > > > > > > >> > > The new version of the Web UI is running stable for several > > months > > > > in > > > > >> > > Alibaba. We are happy to contribute it to the Flink community > if > > > > >> everyone > > > > >> > > like the work. > > > > >> > > > > > > >> > > Best > > > > >> > > Yadong > > > > >> > > > > > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann < > > > trohrm...@apache.org > > > > > > > > > >> > > wrote: > > > > >> > > > > > > >> > >> I agree with Chesnay that the web UI is already fairly > isolated > > > in > > > > >> > Flink's > > > > >> > >> code base (see flink-runtime-web module). Moreover, imo it is > > > very > > > > >> > >> important that Flink comes with a nice web UI which works out > > of > > > > the > > > > >> box > > > > >> > >> because it lowers the entrance hurdle massively. What one > could > > > > >> argue is > > > > >> > >> to > > > > >> > >> make it easier to plug in your own web UI if you want to > > develop > > > > it. > > > > >> > But I > > > > >> > >> would see this as an orthogonal issue. > > > > >> > >> > > > > >> > >> I would actually argue against Robert's proposal to make a > > grand > > > > >> rewrite > > > > >> > >> if > > > > >> > >> this is by any means possible. Experience shows that these > kind > > > of > > > > >> > efforts > > > > >> > >> are usually set up to fail or take super long. If possible, > > then > > > I > > > > >> would > > > > >> > >> rather prefer to convert one component at a time to React. > But > > > I'm > > > > >> not > > > > >> > an > > > > >> > >> expert here and cannot tell whether this is possible or not. > > > > >> > >> > > > > >> > >> What we would need is active help from the community to > review > > > > these > > > > >> > >> changes. I think that is the crucial piece to make such an > > effort > > > > >> work > > > > >> > or > > > > >> > >> fail. > > > > >> > >> > > > > >> > >> Cheers, > > > > >> > >> Till > > > > >> > >> > > > > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler < > > > > >> ches...@apache.org> > > > > >> > >> wrote: > > > > >> > >> > > > > >> > >> > The WebUI goes entirely through the REST API. > > > > >> > >> > > > > > >> > >> > I don't see how moving it out would make it easier to > > > > >> > >> develop/contribute. > > > > >> > >> > > > > > >> > >> > The UI is already encapsulated in it's own module > > > > >> (flink-runtime-web) > > > > >> > >> > and changes can be made to it independently. > > > > >> > >> > Historically the issue was simply that contributions to the > > UI > > > > were > > > > >> > not > > > > >> > >> > reviewed in a timely manner / not at all, which mind you > > > > >> > >> > applies to virtually all components. This also won't change > > by > > > > >> moving > > > > >> > it > > > > >> > >> > out of Flink. > > > > >> > >> > > > > > >> > >> > Having it in Flink also guarantees that there is an > > > > >> > easily-discoverable > > > > >> > >> > version of the UI that is compatible with your Flink > > > > distribution. > > > > >> > >> > > > > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote: > > > > >> > >> > > Re: who's using the web ui > > > > >> > >> > > > > > > >> > >> > > Though many mature solutions, with a fair amount of > > > > >> time/resources > > > > >> > >> > available are likely running their own front ends, for > teams > > > like > > > > >> mine > > > > >> > >> > which are smaller and aren't focused solely on working with > > > > Flink, > > > > >> > >> having > > > > >> > >> > the web ui available removes a large barrier to getting up > > and > > > > >> running > > > > >> > >> > quickly. > > > > >> > >> > > > > > > >> > >> > > But, with the web UI being in core Flink, it does make > it a > > > bit > > > > >> more > > > > >> > >> of > > > > >> > >> > a chore to contribute. > > > > >> > >> > > > > > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy > > with > > > > >> Flink? > > > > >> > Is > > > > >> > >> > it making any calls that aren't through the REST API (and > if > > > so, > > > > >> can > > > > >> > >> those > > > > >> > >> > endpoints be added to the REST API)? > > > > >> > >> > > > > > > >> > >> > > In general I'd support moving it to its own package, > making > > > it > > > > >> > easier > > > > >> > >> to > > > > >> > >> > develop with your standard UI dev tools. This also allows > the > > > web > > > > >> UI > > > > >> > to > > > > >> > >> be > > > > >> > >> > developed and released independently to core Flink. > > > > >> > >> > > > > > > >> > >> > > Dyana > > > > >> > >> > > > > > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert < > fab...@zalando.de> > > > > >> wrote: > > > > >> > >> > >> Hi Till, I basically agree with all your points. i would > > > > stress > > > > >> the > > > > >> > >> > >> "dustiness" of the current architecture: the package > > manager > > > > >> used > > > > >> > >> > (bower) > > > > >> > >> > >> is deprecated since a long time, the chance for the > builds > > > of > > > > >> the > > > > >> > >> flink > > > > >> > >> > web > > > > >> > >> > >> dashboard not working anymore is increasing every day. > > > > >> > >> > >> > > > > >> > >> > >> About the knowledge in the community: Two days is not a > > lot > > > of > > > > >> > time, > > > > >> > >> but > > > > >> > >> > >> interest in this topic seems to be minor anyways. Is > > someone > > > > >> using > > > > >> > >> the > > > > >> > >> > >> Flink Web Dashboard at all, or is everyone running their > > own > > > > >> custom > > > > >> > >> > >> solutions? Because if there is no interest in using the > > Web > > > UI > > > > >> AND > > > > >> > no > > > > >> > >> > one > > > > >> > >> > >> interested in developing, there would be no need to > > package > > > > >> this as > > > > >> > >> > part of > > > > >> > >> > >> the official Flink package, but rather develop an > > > independent > > > > >> > >> solution > > > > >> > >> > (I'm > > > > >> > >> > >> not voting for this right now, just putting it out), if > at > > > > all. > > > > >> The > > > > >> > >> > >> official package could then just ship with the API, > which > > > > other > > > > >> > >> > solutions > > > > >> > >> > >> can build upon. This solution could be from an agile > point > > > of > > > > >> view > > > > >> > >> also > > > > >> > >> > the > > > > >> > >> > >> best (enhanced testing, independent and more effective > dev > > > > >> > workflow, > > > > >> > >> > etc.), > > > > >> > >> > >> but is bad for the usage of the Flink UI, because people > > > need > > > > to > > > > >> > >> install > > > > >> > >> > >> two things individually (Flink and the web dashboard). > > > > >> > >> > >> > > > > >> > >> > >> How did the choice for Angular1 happen back then? Who > was > > > > >> writing > > > > >> > the > > > > >> > >> > >> Dashboard in the first place? > > > > >> > >> > >> > > > > >> > >> > >> Cheers > > > > >> > >> > >> > > > > >> > >> > >> -- > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> *Fabian WollertZalando SE* > > > > >> > >> > >> > > > > >> > >> > >> E-Mail: fab...@zalando.de > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till > Rohrmann < > > > > >> > >> > >> trohrm...@apache.org>: > > > > >> > >> > >> > > > > >> > >> > >>> Thanks for starting this discussion Fabian! I think our > > web > > > > UI > > > > >> > >> > technology > > > > >> > >> > >>> stack is quite dusty by now and it would be beneficial > to > > > > think > > > > >> > >> about > > > > >> > >> > its > > > > >> > >> > >>> technological future. > > > > >> > >> > >>> > > > > >> > >> > >>> On the one hand, our current web UI works more or less > > > > reliable > > > > >> > and > > > > >> > >> > >>> changing the underlying technology has the risk of > > breaking > > > > >> > things. > > > > >> > >> > >>> Moreover, there might be the risk that the newly chosen > > > > >> technology > > > > >> > >> > will be > > > > >> > >> > >>> deprecated at some point in time as well. > > > > >> > >> > >>> > > > > >> > >> > >>> On the other hand, we don't have much Angular 1 > knowledge > > > in > > > > >> the > > > > >> > >> > community > > > > >> > >> > >>> and extending the web UI is, thus, quite hard at the > > > moment. > > > > >> Maybe > > > > >> > >> by > > > > >> > >> > using > > > > >> > >> > >>> some newer web technologies we might be able to attract > > > more > > > > >> > people > > > > >> > >> > with a > > > > >> > >> > >>> web technology background to join the community. > > > > >> > >> > >>> > > > > >> > >> > >>> The lack of people working on the web UI is for me the > > > > biggest > > > > >> > >> problem > > > > >> > >> > I > > > > >> > >> > >>> would like to address. If there is interest in the web > > UI, > > > > then > > > > >> > I'm > > > > >> > >> > quite > > > > >> > >> > >>> sure that we will be able to even migrate to some other > > > > >> technology > > > > >> > >> in > > > > >> > >> > the > > > > >> > >> > >>> future. The next important issue for me is to do the > > change > > > > >> > >> > incrementally > > > > >> > >> > >>> if possible. Ideally we never break the web UI in the > > > process > > > > >> of > > > > >> > >> > migrating > > > > >> > >> > >>> to a new technology. I'm not an expert here so it might > > or > > > > >> might > > > > >> > >> not be > > > > >> > >> > >>> possible. But if it is, then we should design the > > > > >> implementation > > > > >> > >> steps > > > > >> > >> > in > > > > >> > >> > >>> such a way. > > > > >> > >> > >>> > > > > >> > >> > >>> Cheers, > > > > >> > >> > >>> Till > > > > >> > >> > >>> > > > > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert < > > > > >> fab...@zalando.de > > > > >> > > > > > > >> > >> > wrote: > > > > >> > >> > >>> > > > > >> > >> > >>>> Hi everyone, > > > > >> > >> > >>>> > > > > >> > >> > >>>> in this email thread > > > > >> > >> > >>>> < > > > > >> > >> > >>>> > > > > >> > >> > >>> > > > > >> > >> > > > > > >> > >> > > > > >> > > > > > >> > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html > > > > >> > >> > >>>> and the tickets FLINK-10705 > > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> > and > > > > >> > >> FLINK-10706 > > > > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> > the > > > > >> > discussion > > > > >> > >> > came > > > > >> > >> > >>> up > > > > >> > >> > >>>> whether to change the underlying architecture of > Flink's > > > Web > > > > >> > >> Dashboard > > > > >> > >> > >>> from > > > > >> > >> > >>>> Angular1 to something else. This email thread should > be > > > > >> solely to > > > > >> > >> > discuss > > > > >> > >> > >>>> the pro's and con's of this, and what could be the > > target > > > > >> > >> > architecture. > > > > >> > >> > >>>> > > > > >> > >> > >>>> My choice would be React. Personally I agree with > Till's > > > > >> comments > > > > >> > >> in > > > > >> > >> > the > > > > >> > >> > >>>> ticket, Angular 1 being basically outdated and is not > > > > having a > > > > >> > >> large > > > > >> > >> > >>>> following anymore. From my experience the choice > between > > > > >> Angular > > > > >> > >> 2-7 > > > > >> > >> > or > > > > >> > >> > >>>> React is subjective, you can get things done with > both. > > I > > > > >> > >> personally > > > > >> > >> > only > > > > >> > >> > >>>> have experience with React, so I personally would be > > > faster > > > > >> to > > > > >> > >> > develop > > > > >> > >> > >>>> with this one. I currently have not planned to learn > > > Angular > > > > >> as > > > > >> > >> well > > > > >> > >> > >>> (being > > > > >> > >> > >>>> a more backend focused developer in general) so if the > > > > >> decision > > > > >> > >> would > > > > >> > >> > be > > > > >> > >> > >>> to > > > > >> > >> > >>>> go with Angular, i would be unfortunately out of this > > > rework > > > > >> of > > > > >> > the > > > > >> > >> > Flink > > > > >> > >> > >>>> Dashboard most certainly. > > > > >> > >> > >>>> > > > > >> > >> > >>>> Additionally i would like to get rid of bower, since > its > > > > >> > officially > > > > >> > >> > >>>> deprecated < > > > > >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/ > >. > > > > >> > >> > >>>> my > > > > >> > >> > >>>> idea would be to just use a create-react-app package > > with > > > > npm > > > > >> and > > > > >> > >> > webpack > > > > >> > >> > >>>> under the hood. no need for additional lib's here > imho. > > > But > > > > >> > again: > > > > >> > >> > thats > > > > >> > >> > >>>> mostly what i've been working with recently, so thats > a > > > > >> > subjective > > > > >> > >> > >>> point. I > > > > >> > >> > >>>> could imagine getting used to yarn in the future as > > well. > > > > >> > >> > >>>> > > > > >> > >> > >>>> Cheers > > > > >> > >> > >>>> Fabian > > > > >> > >> > >>>> > > > > >> > >> > >>>> -- > > > > >> > >> > >>>> > > > > >> > >> > >>>> > > > > >> > >> > >>>> *Fabian WollertZalando SE* > > > > >> > >> > >>>> > > > > >> > >> > >>>> E-Mail: fab...@zalando.de > > > > >> > >> > >>>> > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > >> > > > > > > >> > > > > > > >> > > -- > > > > >> > > Yadong Xie > > > > >> > > Ministry of Education Key Lab for Intelligent Networks and > > Network > > > > >> > Security > > > > >> > > Dept. of Computer Science and Technology > > > > >> > > School of electronic information and engineering > > > > >> > > Xi'an Jiaotong University > > > > >> > > Xi'an, 710049, P.R. China > > > > >> > > Email: vthink...@gmail.com > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > >
