Hi Steve, Hoya has a lot of good ideas. How do think it relates to Twill (recently incubated at http://twill.incubator.apache.org/)? I feel that there is a lot of overlap in what both projects are trying to achieve. Do you think they complement each other? Is there opportunity for collaboration, perhaps cross-contributing or even converging the two projects?
-Andreas. On Wed, Jan 8, 2014 at 7:08 AM, Steve Loughran <ste...@hortonworks.com>wrote: > I'm starting to put together the incubation proposal for Hoya: a tool to > dynamically deploy applications such as HBase or Accumulo on YARN > > https://wiki.apache.org/incubator/HoyaProposal > > It does already work to the extent that it can bring up either application, > run different clusters of different versions, and remember where containers > were allocated so that on application restart it can ask for them back. > That increases data locality and makes a big difference with HBase. > > It also needs a lot more work -YARN-896 is adding YARN features that help, > but there's lots of fun to be had in Hoya including > -leading edge work in failure handling, modelling cluster unreliability > and reacting to it. Can we move beyond simple blacklisting to > "greylisting", accepting unreliable boxes if we have no altenatives > > Then there's adding more providers, to support different application > installations -I'm starting to write a functional test framework which need > provider-specific workload generations > > Other features: AM should have a web ui that redirects to the live > endpoints to all the app-specific UIs (e.g. HBase Master GUI), as well as > displaying cluster state itself, for people and for management tools.... > > To summarise: lots of fun to be had > > -steve > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
