Hi Fabian, Till, and Robert Thank you for your attention to this matter, I just push our codes to github: https://github.com/vthinkxie/flink-runtime-web.
You can start the project by following the guidelines <https://github.com/vthinkxie/flink-runtime-web#development--debugging> (just run `npm install && npm run proxy`), just feel free to give any comments :) If I missed anything please let me know. Look forward to your feedback and suggestions soon. Best regards Yadong On Fri, Nov 2, 2018 at 5:28 PM Fabian Wollert <[email protected]> wrote: > Hi Yadong, this looks awesome. is there any chance you can already share > the code of the new web UI, so we can take a look at what you guys build > there? I think that would speed up the discussion. If there is already a > fully fledged new Version with everything updated out there, and its even > battle tested in production already, that sounds like the way to go for me. > > i started aside from this discussion here (to strengthen and learn some > new React stuff) my attempt on the React version already, for whoever is > curious, you can check it out here: > https://github.com/drummerwolli/flink-web-ui-tmp (adjust the base url in > actions.js > <https://github.com/drummerwolli/flink-web-ui-tmp/blob/master/src/actions.js#L8>, > npm install and then npm start) ... i just started to convert the first > simple pages, so dont expect the whole UI yet, its just a start ... > > Cheers > > -- > > > *Fabian WollertZalando SE* > > E-Mail: [email protected] > > > Am Fr., 2. Nov. 2018 um 07:52 Uhr schrieb Jeff Zhang <[email protected]>: > >> +1 for the proposal of Yadong. I'd like to add one more comment: >> >> We should provide more development guidance when developing the next >> dashboard via angular 7. As Till mentioned before, one big concern is lack >> of frontend guys in flink community, I believe more development guidance >> will attract more frontend guys to contribute to Flink web ui, and also be >> helpful for backend guys who want to add something to dashboard. >> >> >> >> Shaoxuan Wang <[email protected]>于2018年11月2日周五 上午10:06写道： >> >> > +1 to keep the web UI stack to Angular. This will make the stack >> upgrading >> > much easier and won't break the on-going efforts already made towards >> the >> > improvements of Flink Web UI. >> > >> > Till, Robert, and Fabian, >> > Please take a look at the upgrading plan from Angular 1.x to 7.0 >> proposed >> > by Yadong. Any feedback is appreciated. >> > >> > Regards, >> > Shaoxuan >> > >> > >> > On Thu, Nov 1, 2018 at 5:41 PM Yadong Xie <[email protected]> wrote: >> > >> > > For your interest, here are some snapshots of Flink web UI (with >> angular >> > > 7.0) we recently reconstructed. >> > > [image: snapshot.001.jpeg] >> > > >> > > >> > >> https://drive.google.com/file/d/1p7rcmgHi3ZJpjt7h1H97VIrIhNkiBs1j/view?usp=sharing >> > > >> > > We choose Angular 7.0 due to the following reasons: >> > > >> > > 1. It is easy to migrate the current codebase to Angular 7.0 >> following >> > > the official guide. >> > > 2. It is easy for current Web UI developers who are experienced >> with >> > > Angular 1.x to get familiar with 7.0 reading the official doc. >> > > 3. Migrating other than rewriting is helpful to ensure its >> functional >> > > integrity. >> > > 4. There are over 1 million Angular developers now (2.0 and above, >> > > still increasing rapidly), which means there are potentially more >> > > developers willing to join the community. >> > > 5. No bower or complex node / webpack configuration needed (only >> > > angular cli). >> > > >> > > We split Angular 1.x and 7.0 versions into two folders based on the >> > > following consideration: >> > > >> > > 1. 1.x version could be the default page because there is no >> > > breakchange for both users and developers. >> > > 2. Make the upgrade progressive. Users can switch between two >> versions >> > > by clicking the button on the top right corner. >> > > 3. Reduce maintenance costs and keep the performance of web without >> > > bootstrap both 1.x and 7.0 versions in the same page. >> > > 4. Won't interrupt the current development process. The code can >> still >> > > be submitted to 1.x version folder, we could help to sync all >> > features and >> > > bug fix to 7.0 version if needed. >> > > 5. When the 7.0 version is ready and stable, it is easy to change >> > > default page to it, and remove the old version. >> > > >> > > [image: test.jpg] >> > > >> > > >> > > >> > >> https://drive.google.com/file/d/1v7U5upP0fSepVl-E_46k-Qfj9uVfrGZf/view?usp=sharing >> > > >> > > Here is the brief list of what we have done: >> > > >> > > 1. Create a new folder named next-web-dashboard in >> flink-runtime-web. >> > > 2. Migrate all Angular 1.x controllers, services, config etc one by >> > > one into 7.0 version with ngMigration. >> > > 3. Modify frontend build scripts in order to package two versions >> > > together. >> > > 4. All works listed above are limited in the flink-runtime-web >> folder. >> > > One folder was added and no file or folder was deleted. >> > > >> > > The new version of the Web UI is running stable for several months in >> > > Alibaba. We are happy to contribute it to the Flink community if >> everyone >> > > like the work. >> > > >> > > Best >> > > Yadong >> > > >> > > On Wed, Oct 31, 2018 at 5:48 PM Till Rohrmann <[email protected]> >> > > wrote: >> > > >> > >> I agree with Chesnay that the web UI is already fairly isolated in >> > Flink's >> > >> code base (see flink-runtime-web module). Moreover, imo it is very >> > >> important that Flink comes with a nice web UI which works out of the >> box >> > >> because it lowers the entrance hurdle massively. What one could >> argue is >> > >> to >> > >> make it easier to plug in your own web UI if you want to develop it. >> > But I >> > >> would see this as an orthogonal issue. >> > >> >> > >> I would actually argue against Robert's proposal to make a grand >> rewrite >> > >> if >> > >> this is by any means possible. Experience shows that these kind of >> > efforts >> > >> are usually set up to fail or take super long. If possible, then I >> would >> > >> rather prefer to convert one component at a time to React. But I'm >> not >> > an >> > >> expert here and cannot tell whether this is possible or not. >> > >> >> > >> What we would need is active help from the community to review these >> > >> changes. I think that is the crucial piece to make such an effort >> work >> > or >> > >> fail. >> > >> >> > >> Cheers, >> > >> Till >> > >> >> > >> On Wed, Oct 31, 2018 at 10:14 AM Chesnay Schepler < >> [email protected]> >> > >> wrote: >> > >> >> > >> > The WebUI goes entirely through the REST API. >> > >> > >> > >> > I don't see how moving it out would make it easier to >> > >> develop/contribute. >> > >> > >> > >> > The UI is already encapsulated in it's own module >> (flink-runtime-web) >> > >> > and changes can be made to it independently. >> > >> > Historically the issue was simply that contributions to the UI were >> > not >> > >> > reviewed in a timely manner / not at all, which mind you >> > >> > applies to virtually all components. This also won't change by >> moving >> > it >> > >> > out of Flink. >> > >> > >> > >> > Having it in Flink also guarantees that there is an >> > easily-discoverable >> > >> > version of the UI that is compatible with your Flink distribution. >> > >> > >> > >> > On 31.10.2018 09:53, dyana.rose wrote: >> > >> > > Re: who's using the web ui >> > >> > > >> > >> > > Though many mature solutions, with a fair amount of >> time/resources >> > >> > available are likely running their own front ends, for teams like >> mine >> > >> > which are smaller and aren't focused solely on working with Flink, >> > >> having >> > >> > the web ui available removes a large barrier to getting up and >> running >> > >> > quickly. >> > >> > > >> > >> > > But, with the web UI being in core Flink, it does make it a bit >> more >> > >> of >> > >> > a chore to contribute. >> > >> > > >> > >> > > I'm unaware of how necessary it is for the UI to deploy with >> Flink? >> > Is >> > >> > it making any calls that aren't through the REST API (and if so, >> can >> > >> those >> > >> > endpoints be added to the REST API)? >> > >> > > >> > >> > > In general I'd support moving it to its own package, making it >> > easier >> > >> to >> > >> > develop with your standard UI dev tools. This also allows the web >> UI >> > to >> > >> be >> > >> > developed and released independently to core Flink. >> > >> > > >> > >> > > Dyana >> > >> > > >> > >> > > On 2018/10/31 07:47:50, Fabian Wollert <[email protected]> >> wrote: >> > >> > >> Hi Till, I basically agree with all your points. i would stress >> the >> > >> > >> "dustiness" of the current architecture: the package manager >> used >> > >> > (bower) >> > >> > >> is deprecated since a long time, the chance for the builds of >> the >> > >> flink >> > >> > web >> > >> > >> dashboard not working anymore is increasing every day. >> > >> > >> >> > >> > >> About the knowledge in the community: Two days is not a lot of >> > time, >> > >> but >> > >> > >> interest in this topic seems to be minor anyways. Is someone >> using >> > >> the >> > >> > >> Flink Web Dashboard at all, or is everyone running their own >> custom >> > >> > >> solutions? Because if there is no interest in using the Web UI >> AND >> > no >> > >> > one >> > >> > >> interested in developing, there would be no need to package >> this as >> > >> > part of >> > >> > >> the official Flink package, but rather develop an independent >> > >> solution >> > >> > (I'm >> > >> > >> not voting for this right now, just putting it out), if at all. >> The >> > >> > >> official package could then just ship with the API, which other >> > >> > solutions >> > >> > >> can build upon. This solution could be from an agile point of >> view >> > >> also >> > >> > the >> > >> > >> best (enhanced testing, independent and more effective dev >> > workflow, >> > >> > etc.), >> > >> > >> but is bad for the usage of the Flink UI, because people need to >> > >> install >> > >> > >> two things individually (Flink and the web dashboard). >> > >> > >> >> > >> > >> How did the choice for Angular1 happen back then? Who was >> writing >> > the >> > >> > >> Dashboard in the first place? >> > >> > >> >> > >> > >> Cheers >> > >> > >> >> > >> > >> -- >> > >> > >> >> > >> > >> >> > >> > >> *Fabian WollertZalando SE* >> > >> > >> >> > >> > >> E-Mail: [email protected] >> > >> > >> >> > >> > >> >> > >> > >> Am Di., 30. Okt. 2018 um 15:07 Uhr schrieb Till Rohrmann < >> > >> > >> [email protected]>: >> > >> > >> >> > >> > >>> Thanks for starting this discussion Fabian! I think our web UI >> > >> > technology >> > >> > >>> stack is quite dusty by now and it would be beneficial to think >> > >> about >> > >> > its >> > >> > >>> technological future. >> > >> > >>> >> > >> > >>> On the one hand, our current web UI works more or less reliable >> > and >> > >> > >>> changing the underlying technology has the risk of breaking >> > things. >> > >> > >>> Moreover, there might be the risk that the newly chosen >> technology >> > >> > will be >> > >> > >>> deprecated at some point in time as well. >> > >> > >>> >> > >> > >>> On the other hand, we don't have much Angular 1 knowledge in >> the >> > >> > community >> > >> > >>> and extending the web UI is, thus, quite hard at the moment. >> Maybe >> > >> by >> > >> > using >> > >> > >>> some newer web technologies we might be able to attract more >> > people >> > >> > with a >> > >> > >>> web technology background to join the community. >> > >> > >>> >> > >> > >>> The lack of people working on the web UI is for me the biggest >> > >> problem >> > >> > I >> > >> > >>> would like to address. If there is interest in the web UI, then >> > I'm >> > >> > quite >> > >> > >>> sure that we will be able to even migrate to some other >> technology >> > >> in >> > >> > the >> > >> > >>> future. The next important issue for me is to do the change >> > >> > incrementally >> > >> > >>> if possible. Ideally we never break the web UI in the process >> of >> > >> > migrating >> > >> > >>> to a new technology. I'm not an expert here so it might or >> might >> > >> not be >> > >> > >>> possible. But if it is, then we should design the >> implementation >> > >> steps >> > >> > in >> > >> > >>> such a way. >> > >> > >>> >> > >> > >>> Cheers, >> > >> > >>> Till >> > >> > >>> >> > >> > >>> On Mon, Oct 29, 2018 at 1:06 PM Fabian Wollert < >> [email protected] >> > > >> > >> > wrote: >> > >> > >>> >> > >> > >>>> Hi everyone, >> > >> > >>>> >> > >> > >>>> in this email thread >> > >> > >>>> < >> > >> > >>>> >> > >> > >>> >> > >> > >> > >> >> > >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Cluster-Overview-Dashboard-Improvement-Proposal-td24531.html >> > >> > >>>> and the tickets FLINK-10705 >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10705> and >> > >> FLINK-10706 >> > >> > >>>> <https://issues.apache.org/jira/browse/FLINK-10706> the >> > discussion >> > >> > came >> > >> > >>> up >> > >> > >>>> whether to change the underlying architecture of Flink's Web >> > >> Dashboard >> > >> > >>> from >> > >> > >>>> Angular1 to something else. This email thread should be >> solely to >> > >> > discuss >> > >> > >>>> the pro's and con's of this, and what could be the target >> > >> > architecture. >> > >> > >>>> >> > >> > >>>> My choice would be React. Personally I agree with Till's >> comments >> > >> in >> > >> > the >> > >> > >>>> ticket, Angular 1 being basically outdated and is not having a >> > >> large >> > >> > >>>> following anymore. From my experience the choice between >> Angular >> > >> 2-7 >> > >> > or >> > >> > >>>> React is subjective, you can get things done with both. I >> > >> personally >> > >> > only >> > >> > >>>> have experience with React, so I personally would be faster >> to >> > >> > develop >> > >> > >>>> with this one. I currently have not planned to learn Angular >> as >> > >> well >> > >> > >>> (being >> > >> > >>>> a more backend focused developer in general) so if the >> decision >> > >> would >> > >> > be >> > >> > >>> to >> > >> > >>>> go with Angular, i would be unfortunately out of this rework >> of >> > the >> > >> > Flink >> > >> > >>>> Dashboard most certainly. >> > >> > >>>> >> > >> > >>>> Additionally i would like to get rid of bower, since its >> > officially >> > >> > >>>> deprecated < >> > >> > https://bower.io/blog/2017/how-to-migrate-away-from-bower/>. >> > >> > >>>> my >> > >> > >>>> idea would be to just use a create-react-app package with npm >> and >> > >> > webpack >> > >> > >>>> under the hood. no need for additional lib's here imho. But >> > again: >> > >> > thats >> > >> > >>>> mostly what i've been working with recently, so thats a >> > subjective >> > >> > >>> point. I >> > >> > >>>> could imagine getting used to yarn in the future as well. >> > >> > >>>> >> > >> > >>>> Cheers >> > >> > >>>> Fabian >> > >> > >>>> >> > >> > >>>> -- >> > >> > >>>> >> > >> > >>>> >> > >> > >>>> *Fabian WollertZalando SE* >> > >> > >>>> >> > >> > >>>> E-Mail: [email protected] >> > >> > >>>> >> > >> > >> > >> > >> > >> >> > > >> > > >> > > -- >> > > Yadong Xie >> > > Ministry of Education Key Lab for Intelligent Networks and Network >> > Security >> > > Dept. of Computer Science and Technology >> > > School of electronic information and engineering >> > > Xi'an Jiaotong University >> > > Xi'an, 710049, P.R. China >> > > Email: [email protected] >> > > >> > >> >
