My guess is HDFS-988 caused the slowdown by coarsening some locking that was previously incorrect. Your stress test is NN-only (metadata ops), not an I/O benchmark, right? I/O should be faster in trunk than ever before.
-Todd On Fri, Jul 1, 2011 at 8:23 AM, Eric Payne <eric.payne1...@yahoo.com> wrote: > Hi gang, > > I ran some stress tests on the latest HDFS trunk yesterday, and the > performance > is a lot slower (sometimes 10 times slower) when compared with the HDFS in > MR-279. The HDFS in MR-279 is slightly behind trunk. The stability of > HDFS trunk > seems to be better than HDFS MR-279, but I'm not sure if the slowness is > just > avoiding the race contitions or if they are actually fixed in trunk. > > At this point, I'm not sure what is causing this performance disparity. I > notice > that Block management has recently undergone significant changes in trunk. > It > has some new locking and it is now in its own package. Could this be part > of the > cause? > > Thanks, > -Eric -- Todd Lipcon Software Engineer, Cloudera
