I'm also not convinced that a Javascript-based approach is the way to go. We shouldn't switch the default UI until (at a minimum) we have the command-line tools that Colin requested, and even then I'd still want to retain support for text-based browsers like elinks unless there are compelling technical reasons not to.
Haohui, I'm sympathetic since you've already done all this work on a pure-JS version, but it's also true that the existing JSP pages could be cleaned up to achieve basically the same visual effect while also still working in text-only browsers. Thanks, Andrew On Wed, Oct 30, 2013 at 12:34 AM, Luke Lu <l...@vicaya.com> wrote: > I don't think that we have reached a consensus that the new javascript only > UI is the right direction to go. Most people considered it "interesting". I > personally think it's inappropriate for core Hadoop UI, as it increases > attack surface of the UI and taking away existing mitigation options from > users unnecessarily. See my latest comments on HDFS-5333 for "concrete" > examples. > > __Luke > > > On Tue, Oct 29, 2013 at 11:28 AM, Haohui Mai <h...@hortonworks.com> wrote: > > > I would like to summarize the discussions so far. It seems that we have > > reached two consensus: > > > > 1. The new JavaScript-based UI is the right direction to go. > > 2. For now we should keep the old JSP pages around for compatibility > > reasons. > > > > There're some debates on the usages of the JMX / JSON APIs, but this is > > orthogonal to switching the UI, thus I consider it as a technical detail. > > We can continue the discussions in the public jira. > > > > The new UI has already landed in the trunk, based on the consensus it > seems > > that we can switch the default UI to the new one shortly. The user can > > still access the old web UI using the same URLs. > > > > The only question remain is that who is going to maintain the old web UI. > > My answer is that we should leave them as deprecated and focus the effort > > on the new web UI. > > > > Thanks, > > Haohui > > > > > > > > On Tue, Oct 29, 2013 at 5:22 AM, Zheng, Kai <kai.zh...@intel.com> wrote: > > > > > > having /JMX for monitoring integration and a /JSON end point for the > UI > > > IMHO, this makes sense, especially for the long term. JMX interface > > serves > > > as management console in admin perspective, WebUI serves as end user > > > interface. Both might share same functionality codes, but that does not > > > validate we couple them together. > > > > > > Thanks & regards, > > > Kai > > > > > > -----Original Message----- > > > From: Alejandro Abdelnur [mailto:t...@cloudera.com] > > > Sent: Tuesday, October 29, 2013 8:14 AM > > > To: hdfs-dev@hadoop.apache.org > > > Subject: Re: Replacing the JSP web UIs to HTML 5 applications > > > > > > Isn't using JMX to expose JSON for the web UI misusing JMX? > > > > > > I would think a more appropriate approach would be having /JMX for > > > monitoring integration and a /JSON end point for the UI data. > > > > > > Thanks. > > > > > > > > > On Mon, Oct 28, 2013 at 4:58 PM, Haohui Mai <h...@hortonworks.com> > > wrote: > > > > > > > Alejandro, > > > > > > > > If I understand correctly, that is the exact approach that the new > web > > > > UI is taking. The new web UI takes the output from JMX and renders > > > > them as HTML at the client side. > > > > > > > > ~Haohui > > > > > > > > > > > > On Mon, Oct 28, 2013 at 4:18 PM, Alejandro Abdelnur < > t...@cloudera.com > > > > >wrote: > > > > > > > > > Haohui, > > > > > > > > > > If you have NN and DNs producing JSON instead HTML, then you can > > > > > build JS based web UIs. Take for example Oozie, Oozie produces > JSON, > > > > > it has a > > > > built > > > > > in JS web ui that consumes JSON and Hue has built an external web > UI > > > > > that also consumes JSON. In the case of Hue UI, Oozie didn't have > to > > > > > change anything to get that UI and improvements on the Hue UI don't > > > > > require changes in Oozie unless it is to produce additional > > > information. > > > > > > > > > > hope this clarifies. > > > > > > > > > > Thx > > > > > > > > > > > > > > > On Mon, Oct 28, 2013 at 4:06 PM, Haohui Mai <h...@hortonworks.com> > > > > wrote: > > > > > > > > > > > Echo my comments on HDFS-5402: > > > > > > > > > > > > bq. If we're going to remove the old web UI, I think the new web > > > > > > UI has to have the same level of unit testing. We shouldn't go > > > > > > backwards in terms of unit testing. > > > > > > > > > > > > I take a look at TestNamenodeJspHelper / TestDatanodeJspHelper / > > > > > > TestClusterJspHelper. It seems to me that we can merge these > tests > > > > > > with > > > > > the > > > > > > unit tests on JMX. > > > > > > > > > > > > bq. If we are going to > > > > > > remove this capability, we need to add some other command-line > > > > > > tools to get the same functionality. These tools could use REST > if > > > > > > we have that, or JMX, but they need to exist before we can > > > > > > consider removing the old UI. > > > > > > > > > > > > This is a good point. Since all information are available through > > > > > > JMX, > > > > > the > > > > > > easiest way to approach it is to write some scripts using > Node.js. > > > > > > The architecture of the new Web UIs is ready for this. > > > > > > > > > > > > > > > > > > On Mon, Oct 28, 2013 at 3:57 PM, Alejandro Abdelnur > > > > > > <t...@cloudera.com > > > > > > >wrote: > > > > > > > > > > > > > Producing JSON would be great. Agree with Colin that we should > > > > > > > leave > > > > > for > > > > > > > now the current JSP based web ui. > > > > > > > > > > > > > > thx > > > > > > > > > > > > > > > > > > > > > On Mon, Oct 28, 2013 at 11:16 AM, Colin McCabe < > > > > cmcc...@alumni.cmu.edu > > > > > > > >wrote: > > > > > > > > > > > > > > > This is a really interesting project, Haohui. I think it > will > > > > > > > > make our web UI much nicer. > > > > > > > > > > > > > > > > I have a few concerns about removing the old web UI, however: > > > > > > > > > > > > > > > > * If we're going to remove the old web UI, I think the new > web > > > > > > > > UI > > > > has > > > > > > > > to have the same level of unit testing. We shouldn't go > > > > > > > > backwards > > > > in > > > > > > > > terms of unit testing. > > > > > > > > > > > > > > > > * Most of the deployments of elinks and links out there don't > > > > support > > > > > > > > Javascript. This is just a reality of life when using CentOS > > > > > > > > 5 or > > > > 6, > > > > > > > > which many users are still using. I have used "links" to > > > > > > > > diagnose problems through the web UI in the past, in systems > > > > > > > > where access to the cluster was available only through > telnet. > > > > > > > > If we are going to remove this capability, we need to add > some > > > > > > > > other command-line > > > > tools > > > > > > > > to get the same functionality. These tools could use REST if > > > > > > > > we > > > > have > > > > > > > > that, or JMX, but they need to exist before we can consider > > > > removing > > > > > > > > the old UI. > > > > > > > > > > > > > > > > best, > > > > > > > > Colin > > > > > > > > > > > > > > > > On Fri, Oct 25, 2013 at 7:31 PM, Haohui Mai > > > > > > > > <h...@hortonworks.com> > > > > > > > wrote: > > > > > > > > > Thanks for the reply, Luke. Here I just echo my response > > > > > > > > > from the > > > > > > jira: > > > > > > > > > > > > > > > > > > bq. this client-side js only approach, which is less secure > > > > > > > > > than > > > > a > > > > > > > > > progressively enhanced hybrid approach used by YARN. The > > > > > > > > > recent > > > > > gmail > > > > > > > > > XSS fiasco highlights the issue. > > > > > > > > > > > > > > > > > > I'm presenting an informal security analysis to compare the > > > > > security > > > > > > of > > > > > > > > the > > > > > > > > > old and the new web UIs. > > > > > > > > > > > > > > > > > > An attacker launches an XSS attack by injecting malicious > > > > > > > > > code > > > > > which > > > > > > > are > > > > > > > > > usually HTML or JavaScript fragments into the web page, so > > > > > > > > > that > > > > the > > > > > > > > > malicious code can have the same privileges of the web > page. > > > > > > > > > > > > > > > > > > First, in the scope of XSS attacks, note that the threat > > > > > > > > > models > > > > of > > > > > > > > > launching XSS attacks on Internet sites Gmail/Linkedin and > > > > > > > > > the > > > > one > > > > > of > > > > > > > the > > > > > > > > > Hadoop UIs are different. They have fundamental different > > > > > > > > > sets of > > > > > > > > external > > > > > > > > > inputs that the attackers have control to. Internet sites > > > > > > > > > have > > > > > little > > > > > > > > > control of these inputs. In the case of Gmail / Linkedin, > an > > > > attack > > > > > > can > > > > > > > > > send you a crafted e-mail, or put malicious description in > > > > > > > > > his / her Linkedin profile. The sets of external inputs are > > > > *restricted* > > > > > in > > > > > > > > > Hadoop UIs. The new web UIs take JMX and WebHDFS as inputs. > > > > > > > > > The attacker has to launch a XSS attack by: > > > > > > > > > > > > > > > > > > * Compromise the jars so that the output of JMX / WebHDFS > > > > > > > > > have > > > > the > > > > > > > > > malicious code. > > > > > > > > > * Replace the web UIs completely to include the malicious > > code. > > > > > > > > > > > > > > > > > > In either case *the attacker has to compromise the hadoop > > > > > > > > > core or > > > > > the > > > > > > > > > namenode*. That means the new web UIs are at least as > secure > > > > > > > > > as > > > > the > > > > > > > > hadoop > > > > > > > > > core, and the namenode machine. > > > > > > > > > > > > > > > > > > Second, I argue that using client-side templates are more > > > > > > > > > secure > > > > > than > > > > > > > the > > > > > > > > > current JSP-based server-side templates. To defend against > > > > > > > > > XSS attacks, both techniques have to filter the external > > > > > > > > > inputs at > > > > > > *every* > > > > > > > > > possible execution paths. Several facts much be taken into > > > > > > > > > plays when evaluating the security of both approaches in > > > > real-world > > > > > > > > > environments: > > > > > > > > > > > > > > > > > > * The JavaScript libraries used in the new web UIs have > > > > > > > > > survived > > > > in > > > > > > > > > extremely large-scale production tests. jQuery is used by > > > > > > > > > Google > > > > > and > > > > > > > > > Microsoft, bootstrap is used by Twitter, and dust.js is > > > > > > > > > used by > > > > > > > > Linkedin. > > > > > > > > > All libraries survived from hundreds of thousands of > attack > > > > > > > > > attempts on a daily basis. I agree that the libraries > > > > might > > > > > > > still > > > > > > > > > be imperfect, but there's no way that we can test the JSP > > > > > > > > > web UIs to achieve the same level of assurances given the > > > > > > > > > amount of > > > > > > > > resources > > > > > > > > > the community has. > > > > > > > > > * Client-side templates consolidate all filtering logic in > > > > > > > > > one > > > > > > central > > > > > > > > > place. Recall that the goal is to filter all external > inputs > > > > > > > > > at > > > > > every > > > > > > > > > execution paths, this is a much more systematic approach > > > > compared > > > > > to > > > > > > > the > > > > > > > > > server-side templates we have today. It is difficult (if > not > > > > > > > > > impossible) to do it in a JSP/ASP/PHP application, since > > > > > > > > > such > > > > > > > filtering > > > > > > > > > can be only achieved via ad-hoc approaches ([1] shows some > > > > > > > > > empirical data). Also, HDFS-4901 recently describes a XSS > > > > > > > vulnerability > > > > > > > > in > > > > > > > > > browseDirectory.jsp. > > > > > > > > > > > > > > > > > > bq. You'd require proper SSL (not self signed) setup to > > > > > > > > > avoid JS injection > > > > > > > > > > > > > > > > > > Commodity browsers enforce Same-Origin Policy to defend > > > > > > > > > against > > > > > code > > > > > > > > > injections. It has nothing to do with what kinds of SSL > > > > > certificates > > > > > > > > > you hold. > > > > > > > > > > > > > > > > > > bq. I also have concerns that we commit these changes > > > > > > > > > without > > > > > > matching > > > > > > > > > unit tests > > > > > > > > > > > > > > > > > > The JavaScript code can be automatically tested. The same > > > > > > > > > code > > > > can > > > > > be > > > > > > > run > > > > > > > > > by node.js and the test can compared with pre-defined > > > > > > > > > results. It is also possible to write an adapter to use > > > > > > > > > Rhino to > > > > > > > > accomplish > > > > > > > > > the same task. We can discuss how to integrate them into > the > > > > > > > > > maven test routines in a different thread. > > > > > > > > > > > > > > > > > > bq. Client side rendering completely breaks the workflows > > > > > > > > > for ops > > > > > who > > > > > > > > rely > > > > > > > > > on text based terminal/emacs/vim browsers (no js support) > to > > > > > > > > > monitor component UI. > > > > > > > > > > > > > > > > > > links / elinks (http://elinks.or.cz/) are text-based web > > > > browsers > > > > > > that > > > > > > > > > support JavaScript. > > > > > > > > > > > > > > > > > > bq. The priority/requirements for UI in core Hadoop should > > > > > > > > > be > > > > > > security > > > > > > > > and > > > > > > > > > correctness, which client side templating cannot address > > > > > > > > > properly so far. > > > > > > > > > > > > > > > > > > I agree that we should focus on security and correctness. > > > > > > > > > The > > > > > > > paragraphs > > > > > > > > > above explain that how the architecture of the new UIs > makes > > > > > > > > > the UIs more secure in real-world settings compared to the > > > > UI > > > > > > we > > > > > > > > have > > > > > > > > > today. > > > > > > > > > > > > > > > > > > References: > > > > > > > > > > > > > > > > > > 1. A. Yip et al. Improving Application Security with Data > > > > > > > > > Flow > > > > > > > > Assertions. > > > > > > > > > In SOSP'2009. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 25, 2013 at 10:02 AM, Luke Lu <l...@apache.org> > > > > wrote: > > > > > > > > > > > > > > > > > >> Echoing my comments on HDFS-3555: > > > > > > > > >> > > > > > > > > >> I have concerns with this client-side js only approach, > > > > > > > > >> which is > > > > > > less > > > > > > > > >> secure than a progressively enhanced hybrid approach used > > > > > > > > >> by > > > > YARN. > > > > > > The > > > > > > > > >> recent gmail XSS fiasco highlights the issue. I also have > > > > concerns > > > > > > > that > > > > > > > > we > > > > > > > > >> commit these changes without matching unit tests - the > fact > > > > > > > > >> you > > > > > > cannot > > > > > > > > >> effectively unit test these changes should tell you > > > > > > > > >> something > > > > > about > > > > > > > this > > > > > > > > >> approach. > > > > > > > > >> > > > > > > > > >> *Requiring* JS means that an admin cannot turn off js to > > > > > (partially) > > > > > > > use > > > > > > > > >> core Hadoop UI. You'd *require* proper SSL (not self > > > > > > > > >> signed) > > > > setup > > > > > > to > > > > > > > > avoid > > > > > > > > >> JS injection, even if security of js libraries used is > > > > > > > > >> perfect, > > > > > > which > > > > > > > I > > > > > > > > >> doubt (search gmail/linkedin XSS). Client side rendering > > > > > completely > > > > > > > > breaks > > > > > > > > >> the workflows for ops who rely on text based > > > > > > > > >> terminal/emacs/vim > > > > > > > browsers > > > > > > > > >> (no js support) to monitor component UI. > > > > > > > > >> > > > > > > > > >> IMO, JS-only rendering belongs to social networking sites > > > > > > > > >> and/or > > > > > > SaaS > > > > > > > > >> front-ends, where full time UI/security specialists > > > > > > > > >> babysits UI > > > > > > > > changes. I > > > > > > > > >> think eventually most users will use a self servicing UI > in > > > > > > > > >> a > > > > SaaS > > > > > > > > >> front-end that uses REST/JMX API to get data from back-end > > > > > > components, > > > > > > > > >> besides their own app master/service UI. The > > > > priority/requirements > > > > > > for > > > > > > > > UI > > > > > > > > >> in core Hadoop should be security and correctness, which > > > > > > > > >> client > > > > > side > > > > > > > > >> templating cannot address properly so far. > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> On Tue, Oct 22, 2013 at 3:59 PM, Haohui Mai < > > > > h...@hortonworks.com > > > > > > > > > > > > > > wrote: > > > > > > > > >> > > > > > > > > >> > Hi all, > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > Jing Zhao and I recently have reimplemented the > JSP-based > > > > > > > > >> > web > > > > > UIs > > > > > > in > > > > > > > > >> HTML 5 > > > > > > > > >> > applications (HDFS-5333). Based on our prelimanary > > > > > > > > >> > testing > > > > > results > > > > > > > we > > > > > > > > >> > believe thst the new web UIs of the namenodes and the > > > > > > > > >> > datanode > > > > > are > > > > > > > > ready > > > > > > > > >> > for everyday uses. > > > > > > > > >> > > > > > > > > > >> > You're more than welcome to try it out on trunk by > > > > > > > > >> > visiting > > > > > > http:// > > > > > > > > >> > <namenode>/dfshealth.html > > > > > > > > >> > > > > > > > > > >> > There are a number of benefits from this transition. > From > > > > > > > > >> > a > > > > > > > > developer's > > > > > > > > >> > prospective, the most notable one is *maintainability*: > > > > > > > > >> > > > > > > > > > >> > (1) The abstractions between the UI and the core server > > > > > > > > >> > are > > > > > > > > well-defined, > > > > > > > > >> > decoupling the UI and the core hadoop servers. > > > > > > > > >> > > > > > > > > > >> > (2) It allows us to deprecate the logic in the JSP > pages. > > > > > > > > >> > The > > > > > old > > > > > > > web > > > > > > > > UIs > > > > > > > > >> > have to duplicate the logic in the JSPs. The logic is > > > > > > > > >> > often > > > > > > > > out-of-dated > > > > > > > > >> > and not well-tested, which leads to broken pages and > > > > > > > > >> > security vulnerabilities(e.g. HDFS-5251, HDFS-5307, > > > > > > > > >> > HDFS-5308, > > > > HDFS-5317 > > > > > > and > > > > > > > > >> > HDFS-4901). The architecture of the new UIs prevent > these > > > > > > > > >> > bugs > > > > > at > > > > > > > the > > > > > > > > >> very > > > > > > > > >> > beginning. > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > I propose that deprecate the old, JSP-based web UIs in > > > > > > > > >> > 2.3. I > > > > > > opened > > > > > > > > >> > HDFS-5402 to track the relevant discussions. > > > > > > > > >> > > > > > > > > > >> > Your feedbacks are highly appreciated. > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > Sincerely, > > > > > > > > >> > > > > > > > > > >> > Haohui > > > > > > > > >> > > > > > > > > > >> > -- > > > > > > > > >> > CONFIDENTIALITY NOTICE > > > > > > > > >> > NOTICE: This message is intended for the use of the > > > > > > > > >> > individual > > > > > or > > > > > > > > entity > > > > > > > > >> to > > > > > > > > >> > which it is addressed and may contain information that > is > > > > > > > > confidential, > > > > > > > > >> > privileged and exempt from disclosure under applicable > > > > > > > > >> > law. If > > > > > the > > > > > > > > reader > > > > > > > > >> > of this message is not the intended recipient, you are > > > > > > > > >> > hereby > > > > > > > notified > > > > > > > > >> that > > > > > > > > >> > any printing, copying, dissemination, distribution, > > > > > > > > >> > disclosure > > > > > or > > > > > > > > >> > forwarding of this communication is strictly prohibited. > > > > > > > > >> > If > > > > you > > > > > > have > > > > > > > > >> > received this communication in error, please contact the > > > > sender > > > > > > > > >> immediately > > > > > > > > >> > and delete it from your system. Thank You. > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > > > > > > > CONFIDENTIALITY NOTICE > > > > > > > > > NOTICE: This message is intended for the use of the > > > > > > > > > individual or > > > > > > > entity > > > > > > > > to > > > > > > > > > which it is addressed and may contain information that is > > > > > > confidential, > > > > > > > > > privileged and exempt from disclosure under applicable law. > > > > > > > > > If > > > > the > > > > > > > reader > > > > > > > > > of this message is not the intended recipient, you are > > > > > > > > > hereby > > > > > > notified > > > > > > > > that > > > > > > > > > any printing, copying, dissemination, distribution, > > > > > > > > > disclosure or forwarding of this communication is strictly > > > > > > > > > prohibited. If you > > > > > have > > > > > > > > > received this communication in error, please contact the > > > > > > > > > sender > > > > > > > > immediately > > > > > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Alejandro > > > > > > > > > > > > > > > > > > > -- > > > > > > CONFIDENTIALITY NOTICE > > > > > > NOTICE: This message is intended for the use of the individual or > > > > entity > > > > > to > > > > > > which it is addressed and may contain information that is > > > > > > confidential, privileged and exempt from disclosure under > > > > > > applicable law. If the > > > > reader > > > > > > of this message is not the intended recipient, you are hereby > > > > > > notified > > > > > that > > > > > > any printing, copying, dissemination, distribution, disclosure or > > > > > > forwarding of this communication is strictly prohibited. If you > > > > > > have received this communication in error, please contact the > > > > > > sender > > > > > immediately > > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Alejandro > > > > > > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or > > > > entity to which it is addressed and may contain information that is > > > > confidential, privileged and exempt from disclosure under applicable > > > > law. If the reader of this message is not the intended recipient, you > > > > are hereby notified that any printing, copying, dissemination, > > > > distribution, disclosure or forwarding of this communication is > > > > strictly prohibited. If you have received this communication in > error, > > > > please contact the sender immediately and delete it from your system. > > > Thank You. > > > > > > > > > > > > > > > > -- > > > Alejandro > > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > >
