1) it should be sort-avoidance. 2) work pool (like Tenzing) Sorry ,the adaptive heartbeat code is not in this github code, we are discussing it.
On Fri, Feb 17, 2012 at 11:00 AM, Anty <anty....@gmail.com> wrote: > Hi: Todd > > yes, the rewritten shuffle in actual a backport of the shuffle from MR2 . > We mainly add the following two features: > 1) shuffle avoidance > 2) work pool > > > On Fri, Feb 17, 2012 at 3:27 AM, Todd Lipcon <t...@cloudera.com> wrote: > >> Hey Schubert, >> >> Looking at the code on github, it looks like your rewritten shuffle is >> in fact just a backport of the shuffle from MR2. I didn't look closely >> - are there any distinguishing factors? >> Also, the OOB heartbeat and adaptive heartbeat code seems to be the >> same as what's in 1.0? >> >> -Todd >> >> On Thu, Feb 16, 2012 at 9:44 AM, Schubert Zhang <zson...@gmail.com> >> wrote: >> > Here is the presentation to describe our job, >> > >> http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a >> > Wellcome to give your advises. >> > It's just a little step, and we are continue to do more improvements, >> thanks >> > for your help. >> > >> > >> > >> > >> > On Thu, Feb 16, 2012 at 11:01 PM, Anty <anty....@gmail.com> wrote: >> >> >> >> Hi: Guys >> >> We just deliver a optimized hadoop , if you are interested, Pls >> >> refer to https://github.com/hanborq/hadoop >> >> >> >> -- >> >> Best Regards >> >> Anty Rao >> > >> > >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > > > > -- > Best Regards > Anty Rao >
