On 12/09/11 19:04, Chi Chan wrote:
Are any Hadoop implementations planning to add "enterprise features"
in Platform MapReduce?
http://www.youtube.com/watch?v=QV4wJifsqbQ
http://www.youtube.com/watch?v=cDfZTx-BOyY
http://www.youtube.com/watch?v=MEKXo-1hnkQ
Platform said that its MapReduce implementation totally replaces the
JobTracker, while the rest of the Hadoop stack is unchanged. Is there
a Hadoop API that would allow external batch systems (like Grid Engine
or Open Grid Scheduler, PBS, Condor, SLURM, etc) to plug into Hadoop?
--Chi
It would be nice to have a scheduler that worked with the other native
schedulers -and have those schedulers work with Hadoop- so neither
system will overcommit. The other schedulers aren't topology aware so
will run code anywhere; it's Hadoop that cares more where stuff goes.
The way Hadoop is architected you could do the hadoop side of the
scheduler without breaking anything; I don't know about the others.
