We just update the slides of this improvements: http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a
Updates: (1) modified some describes to make things more clear and accuracy. (2) add some benchmarks to make sense. On Sat, Feb 18, 2012 at 11:12 PM, Anty <anty....@gmail.com> wrote: > > > On Fri, Feb 17, 2012 at 3:27 AM, Todd Lipcon <t...@cloudera.com> wrote: > >> Hey Schubert, >> >> Looking at the code on github, it looks like your rewritten shuffle is >> in fact just a backport of the shuffle from MR2. I didn't look closely >> > > additionally, the rewritten shuffle in MR2 has some bugs, which harm the > overall performance, for which I have already file a jira to report this, > with a patch available. > MAPREDUCE-3685 <https://issues.apache.org/jira/browse/MAPREDUCE-3685> > > > >> - are there any distinguishing factors? >> Also, the OOB heartbeat and adaptive heartbeat code seems to be the >> same as what's in 1.0? >> >> -Todd >> >> On Thu, Feb 16, 2012 at 9:44 AM, Schubert Zhang <zson...@gmail.com> >> wrote: >> > Here is the presentation to describe our job, >> > >> http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a >> > Wellcome to give your advises. >> > It's just a little step, and we are continue to do more improvements, >> thanks >> > for your help. >> > >> > >> > >> > >> > On Thu, Feb 16, 2012 at 11:01 PM, Anty <anty....@gmail.com> wrote: >> >> >> >> Hi: Guys >> >> We just deliver a optimized hadoop , if you are interested, Pls >> >> refer to https://github.com/hanborq/hadoop >> >> >> >> -- >> >> Best Regards >> >> Anty Rao >> > >> > >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > > > > -- > Best Regards > Anty Rao >
