Hi Yadong, this is awesome, thx for the code! I will try it out on our infrastructure and will post my feedback here, latest next week.
I will also check if my ideas for FLINK-10707 <https://issues.apache.org/jira/browse/FLINK-10707> are doable with your code since this was what pushed this discussion initially. Cheers -- *Fabian WollertZalando SE* E-Mail: fab...@zalando.de Am Mo., 5. Nov. 2018 um 04:42 Uhr schrieb Yadong Xie <vthink...@gmail.com>: > Hi Fabian, Till, and Robert > > Thank you for your attention to this matter, I just push our codes to > github: https://github.com/vthinkxie/flink-runtime-web. > > You can start the project by following the guidelines > <https://github.com/vthinkxie/flink-runtime-web#development--debugging> > (just > run `npm install && npm run proxy`), just feel free to give any comments :) > > If I missed anything please let me know. Look forward to your feedback and > suggestions soon. > > Best regards > Yadong > > > On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <fab...@zalando.de> wrote: > > > Hi Yadong, this looks awesome. is there any chance you can already share > > the code of the new web UI, so we can take a look at what you guys build > > there? I think that would speed up the discussion. If there is already a > > fully fledged new Version with everything updated out there, and its even > > battle tested in production already, that sounds like the way to go for > me. > > > > i started aside from this discussion here (to strengthen and learn some > > new React stuff) my attempt on the React version already, for whoever is > > curious, you can check it out here: > > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base url in > > actions.js > > < > https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8 > >, > > npm install and then npm start) ... i just started to convert the first > > simple pages, so dont expect the whole UI yet, its just a start ... > > > > Cheers > > > > -- > > > > > > *Fabian WollertZalando SE* > > > > E-Mail: fab...@zalando.de > > > > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <zjf...@gmail.com>: > > > >> +1 for the proposal of Yadong. I'd like to add one more comment: > >> > >> We should provide more development guidance when developing the next > >> dashboard via angular 7. As Till mentioned before, one big concern is > lack > >> of frontend guys in flink community, I believe more development guidance > >> will attract more frontend guys to contribute to Flink web ui, and also > be > >> helpful for backend guys who want to add something to dashboard. > >> > >> > >> > >> Shaoxuan Wang <wshaox...@gmail.com>于2018年11月2日周五 上午10:06写道： > >> > >> > +1 to keep the web UI stack to Angular. This will make the stack > >> upgrading > >> > much easier and won't break the on-going efforts already made towards > >> the > >> > improvements of Flink Web UI. > >> > > >> > Till, Robert, and Fabian, > >> > Please take a look at the upgrading plan from Angular 1.x to 7.0 > >> proposed > >> > by Yadong. Any feedback is appreciated. > >> > > >> > Regards, > >> > Shaoxuan > >> > > >> > > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <vthink...@gmail.com> > wrote: > >> > > >> > > For your interest, here are some snapshots of Flink web UI (with > >> angular > >> > > 7.0) we recently reconstructed. > >> > > [image: snapshot.001.jpeg] > >> > > > >> > > > >> > > >> > https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing > >> > > > >> > > We choose Angular 7.0 due to the following reasons: > >> > > > >> > > 1. It is easy to migrate the current codebase to Angular 7.0 > >> following > >> > > the official guide. > >> > > 2. It is easy for current Web UI developers who are experienced > >> with > >> > > Angular 1.x to get familiar with 7.0 reading the official doc. > >> > > 3. Migrating other than rewriting is helpful to ensure its > >> functional > >> > > integrity. > >> > > 4. There are over 1 million Angular developers now (2.0 and > above, > >> > > still increasing rapidly), which means there are potentially more > >> > > developers willing to join the community. > >> > > 5. No bower or complex node / webpack configuration needed (only > >> > > angular cli). > >> > > > >> > > We split Angular 1.x and 7.0 versions into two folders based on the > >> > > following consideration: > >> > > > >> > > 1. 1.x version could be the default page because there is no > >> > > breakchange for both users and developers. > >> > > 2. Make the upgrade progressive. Users can switch between two > >> versions > >> > > by clicking the button on the top right corner. > >> > > 3. Reduce maintenance costs and keep the performance of web > without > >> > > bootstrap both 1.x and 7.0 versions in the same page. > >> > > 4. Won't interrupt the current development process. The code can > >> still > >> > > be submitted to 1.x version folder, we could help to sync all > >> > features and > >> > > bug fix to 7.0 version if needed. > >> > > 5. When the 7.0 version is ready and stable, it is easy to change > >> > > default page to it, and remove the old version. > >> > > > >> > > [image: test.jpg] > >> > > > >> > > > >> > > > >> > > >> > https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing > >> > > > >> > > Here is the brief list of what we have done: > >> > > > >> > > 1. Create a new folder named next-web-dashboard in > >> flink-runtime-web. > >> > > 2. Migrate all Angular 1.x controllers, services, config etc one > by > >> > > one into 7.0 version with ngMigration. > >> > > 3. Modify frontend build scripts in order to package two versions > >> > > together. > >> > > 4. All works listed above are limited in the flink-runtime-web > >> folder. > >> > > One folder was added and no file or folder was deleted. > >> > > > >> > > The new version of the Web UI is running stable for several months > in > >> > > Alibaba. We are happy to contribute it to the Flink community if > >> everyone > >> > > like the work. > >> > > > >> > > Best > >> > > Yadong > >> > > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <trohrm...@apache.org > > > >> > > wrote: > >> > > > >> > >> I agree with Chesnay that the web UI is already fairly isolated in > >> > Flink's > >> > >> code base (see flink-runtime-web module). Moreover, imo it is very > >> > >> important that Flink comes with a nice web UI which works out of > the > >> box > >> > >> because it lowers the entrance hurdle massively. What one could > >> argue is > >> > >> to > >> > >> make it easier to plug in your own web UI if you want to develop > it. > >> > But I > >> > >> would see this as an orthogonal issue. > >> > >> > >> > >> I would actually argue against Robert's proposal to make a grand > >> rewrite > >> > >> if > >> > >> this is by any means possible. Experience shows that these kind of > >> > efforts > >> > >> are usually set up to fail or take super long. If possible, then I > >> would > >> > >> rather prefer to convert one component at a time to React. But I'm > >> not > >> > an > >> > >> expert here and cannot tell whether this is possible or not. > >> > >> > >> > >> What we would need is active help from the community to review > these > >> > >> changes. I think that is the crucial piece to make such an effort > >> work > >> > or > >> > >> fail. > >> > >> > >> > >> Cheers, > >> > >> Till > >> > >> > >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler < > >> ches...@apache.org> > >> > >> wrote: > >> > >> > >> > >> > The WebUI goes entirely through the REST API. > >> > >> > > >> > >> > I don't see how moving it out would make it easier to > >> > >> develop/contribute. > >> > >> > > >> > >> > The UI is already encapsulated in it's own module > >> (flink-runtime-web) > >> > >> > and changes can be made to it independently. > >> > >> > Historically the issue was simply that contributions to the UI > were > >> > not > >> > >> > reviewed in a timely manner / not at all, which mind you > >> > >> > applies to virtually all components. This also won't change by > >> moving > >> > it > >> > >> > out of Flink. > >> > >> > > >> > >> > Having it in Flink also guarantees that there is an > >> > easily-discoverable > >> > >> > version of the UI that is compatible with your Flink > distribution. > >> > >> > > >> > >> > On 31.10.2018 09:53, dyana.rose wrote: > >> > >> > > Re: who's using the web ui > >> > >> > > > >> > >> > > Though many mature solutions, with a fair amount of > >> time/resources > >> > >> > available are likely running their own front ends, for teams like > >> mine > >> > >> > which are smaller and aren't focused solely on working with > Flink, > >> > >> having > >> > >> > the web ui available removes a large barrier to getting up and > >> running > >> > >> > quickly. > >> > >> > > > >> > >> > > But, with the web UI being in core Flink, it does make it a bit > >> more > >> > >> of > >> > >> > a chore to contribute. > >> > >> > > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy with > >> Flink? > >> > Is > >> > >> > it making any calls that aren't through the REST API (and if so, > >> can > >> > >> those > >> > >> > endpoints be added to the REST API)? > >> > >> > > > >> > >> > > In general I'd support moving it to its own package, making it > >> > easier > >> > >> to > >> > >> > develop with your standard UI dev tools. This also allows the web > >> UI > >> > to > >> > >> be > >> > >> > developed and released independently to core Flink. > >> > >> > > > >> > >> > > Dyana > >> > >> > > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <fab...@zalando.de> > >> wrote: > >> > >> > >> Hi Till, I basically agree with all your points. i would > stress > >> the > >> > >> > >> "dustiness" of the current architecture: the package manager > >> used > >> > >> > (bower) > >> > >> > >> is deprecated since a long time, the chance for the builds of > >> the > >> > >> flink > >> > >> > web > >> > >> > >> dashboard not working anymore is increasing every day. > >> > >> > >> > >> > >> > >> About the knowledge in the community: Two days is not a lot of > >> > time, > >> > >> but > >> > >> > >> interest in this topic seems to be minor anyways. Is someone > >> using > >> > >> the > >> > >> > >> Flink Web Dashboard at all, or is everyone running their own > >> custom > >> > >> > >> solutions? Because if there is no interest in using the Web UI > >> AND > >> > no > >> > >> > one > >> > >> > >> interested in developing, there would be no need to package > >> this as > >> > >> > part of > >> > >> > >> the official Flink package, but rather develop an independent > >> > >> solution > >> > >> > (I'm > >> > >> > >> not voting for this right now, just putting it out), if at > all. > >> The > >> > >> > >> official package could then just ship with the API, which > other > >> > >> > solutions > >> > >> > >> can build upon. This solution could be from an agile point of > >> view > >> > >> also > >> > >> > the > >> > >> > >> best (enhanced testing, independent and more effective dev > >> > workflow, > >> > >> > etc.), > >> > >> > >> but is bad for the usage of the Flink UI, because people need > to > >> > >> install > >> > >> > >> two things individually (Flink and the web dashboard). > >> > >> > >> > >> > >> > >> How did the choice for Angular1 happen back then? Who was > >> writing > >> > the > >> > >> > >> Dashboard in the first place? > >> > >> > >> > >> > >> > >> Cheers > >> > >> > >> > >> > >> > >> -- > >> > >> > >> > >> > >> > >> > >> > >> > >> *Fabian WollertZalando SE* > >> > >> > >> > >> > >> > >> E-Mail: fab...@zalando.de > >> > >> > >> > >> > >> > >> > >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann < > >> > >> > >> trohrm...@apache.org>: > >> > >> > >> > >> > >> > >>> Thanks for starting this discussion Fabian! I think our web > UI > >> > >> > technology > >> > >> > >>> stack is quite dusty by now and it would be beneficial to > think > >> > >> about > >> > >> > its > >> > >> > >>> technological future. > >> > >> > >>> > >> > >> > >>> On the one hand, our current web UI works more or less > reliable > >> > and > >> > >> > >>> changing the underlying technology has the risk of breaking > >> > things. > >> > >> > >>> Moreover, there might be the risk that the newly chosen > >> technology > >> > >> > will be > >> > >> > >>> deprecated at some point in time as well. > >> > >> > >>> > >> > >> > >>> On the other hand, we don't have much Angular 1 knowledge in > >> the > >> > >> > community > >> > >> > >>> and extending the web UI is, thus, quite hard at the moment. > >> Maybe > >> > >> by > >> > >> > using > >> > >> > >>> some newer web technologies we might be able to attract more > >> > people > >> > >> > with a > >> > >> > >>> web technology background to join the community. > >> > >> > >>> > >> > >> > >>> The lack of people working on the web UI is for me the > biggest > >> > >> problem > >> > >> > I > >> > >> > >>> would like to address. If there is interest in the web UI, > then > >> > I'm > >> > >> > quite > >> > >> > >>> sure that we will be able to even migrate to some other > >> technology > >> > >> in > >> > >> > the > >> > >> > >>> future. The next important issue for me is to do the change > >> > >> > incrementally > >> > >> > >>> if possible. Ideally we never break the web UI in the process > >> of > >> > >> > migrating > >> > >> > >>> to a new technology. I'm not an expert here so it might or > >> might > >> > >> not be > >> > >> > >>> possible. But if it is, then we should design the > >> implementation > >> > >> steps > >> > >> > in > >> > >> > >>> such a way. > >> > >> > >>> > >> > >> > >>> Cheers, > >> > >> > >>> Till > >> > >> > >>> > >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert < > >> fab...@zalando.de > >> > > > >> > >> > wrote: > >> > >> > >>> > >> > >> > >>>> Hi everyone, > >> > >> > >>>> > >> > >> > >>>> in this email thread > >> > >> > >>>> < > >> > >> > >>>> > >> > >> > >>> > >> > >> > > >> > >> > >> > > >> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html > >> > >> > >>>> and the tickets FLINK-10705 > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and > >> > >> FLINK-10706 > >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the > >> > discussion > >> > >> > came > >> > >> > >>> up > >> > >> > >>>> whether to change the underlying architecture of Flink's Web > >> > >> Dashboard > >> > >> > >>> from > >> > >> > >>>> Angular1 to something else. This email thread should be > >> solely to > >> > >> > discuss > >> > >> > >>>> the pro's and con's of this, and what could be the target > >> > >> > architecture. > >> > >> > >>>> > >> > >> > >>>> My choice would be React. Personally I agree with Till's > >> comments > >> > >> in > >> > >> > the > >> > >> > >>>> ticket, Angular 1 being basically outdated and is not > having a > >> > >> large > >> > >> > >>>> following anymore. From my experience the choice between > >> Angular > >> > >> 2-7 > >> > >> > or > >> > >> > >>>> React is subjective, you can get things done with both. I > >> > >> personally > >> > >> > only > >> > >> > >>>> have experience with React, so I personally would be faster > >> to > >> > >> > develop > >> > >> > >>>> with this one. I currently have not planned to learn Angular > >> as > >> > >> well > >> > >> > >>> (being > >> > >> > >>>> a more backend focused developer in general) so if the > >> decision > >> > >> would > >> > >> > be > >> > >> > >>> to > >> > >> > >>>> go with Angular, i would be unfortunately out of this rework > >> of > >> > the > >> > >> > Flink > >> > >> > >>>> Dashboard most certainly. > >> > >> > >>>> > >> > >> > >>>> Additionally i would like to get rid of bower, since its > >> > officially > >> > >> > >>>> deprecated < > >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>. > >> > >> > >>>> my > >> > >> > >>>> idea would be to just use a create-react-app package with > npm > >> and > >> > >> > webpack > >> > >> > >>>> under the hood. no need for additional lib's here imho. But > >> > again: > >> > >> > thats > >> > >> > >>>> mostly what i've been working with recently, so thats a > >> > subjective > >> > >> > >>> point. I > >> > >> > >>>> could imagine getting used to yarn in the future as well. > >> > >> > >>>> > >> > >> > >>>> Cheers > >> > >> > >>>> Fabian > >> > >> > >>>> > >> > >> > >>>> -- > >> > >> > >>>> > >> > >> > >>>> > >> > >> > >>>> *Fabian WollertZalando SE* > >> > >> > >>>> > >> > >> > >>>> E-Mail: fab...@zalando.de > >> > >> > >>>> > >> > >> > > >> > >> > > >> > >> > >> > > > >> > > > >> > > -- > >> > > Yadong Xie > >> > > Ministry of Education Key Lab for Intelligent Networks and Network > >> > Security > >> > > Dept. of Computer Science and Technology > >> > > School of electronic information and engineering > >> > > Xi'an Jiaotong University > >> > > Xi'an, 710049, P.R. China > >> > > Email: vthink...@gmail.com > >> > > > >> > > >> > > >
