Hi Steve,
it looks interesting. It's a kind of "advanced" provisioning for Hadoop
cluster. I see some potential interesting "work together" with other
projects.
I would be please to be mentor on Hoya.
Regards
JB
On 01/08/2014 04:08 PM, Steve Loughran wrote:
I'm starting to put together the incubation proposal for Hoya: a tool to
dynamically deploy applications such as HBase or Accumulo on YARN
https://wiki.apache.org/incubator/HoyaProposal
It does already work to the extent that it can bring up either application,
run different clusters of different versions, and remember where containers
were allocated so that on application restart it can ask for them back.
That increases data locality and makes a big difference with HBase.
It also needs a lot more work -YARN-896 is adding YARN features that help,
but there's lots of fun to be had in Hoya including
-leading edge work in failure handling, modelling cluster unreliability
and reacting to it. Can we move beyond simple blacklisting to
"greylisting", accepting unreliable boxes if we have no altenatives
Then there's adding more providers, to support different application
installations -I'm starting to write a functional test framework which need
provider-specific workload generations
Other features: AM should have a web ui that redirects to the live
endpoints to all the app-specific UIs (e.g. HBase Master GUI), as well as
displaying cluster state itself, for people and for management tools....
To summarise: lots of fun to be had
-steve
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
