I put up a sketch of a patch for 1473. It compiles and passes at least a couple of the unit tests, but more work to be done on it. Would appreciate comments from this crowd if the new classes seem like good divisions.
-Todd On Wed, Nov 17, 2010 at 2:36 PM, Todd Lipcon <t...@cloudera.com> wrote: > On Wed, Nov 17, 2010 at 2:34 PM, Hairong Kuang <kuang.hair...@fb.com>wrote: > >> You meant HDFS-1473 right? When do you expect that it could be done? >> >> > I was working on HDFS-1473 this morning and could probably get a > preliminary patch up today.. but when I realized that we have 4 parallel > refactorings going on I paused my work. If we agree that HDFS-1473 will go > in before the other ones I can finish it up, but if we're doing the same > thing we should coordinate instead. > > >> Hairong >> >> On 11/17/10 2:29 PM, "Eli Collins" <e...@cloudera.com> wrote: >> >> > Agree we should do the refactoring before the features, we should also >> > add the tests with the refactoring to show they don't break things. >> > >> > On Wed, Nov 17, 2010 at 2:27 PM, Todd Lipcon <t...@cloudera.com> wrote: >> >> OK, looks like there is still quite a bit more in motion here, thanks >> for >> >> the list, Hairong. >> >> How would everyone feel about setting aside those improvements/fixes, >> doing >> >> the refactor first, and then going back to them? It should help with >> testing >> >> and reduce conflicts between the various ongoing projects. >> >> Maybe we can discuss quickly on this thread: what are some small >> refactors >> >> we can start with to help separate these things? Ivan's project looks >> really >> >> good but it seems like it is a pretty large change. If we can break it >> up >> >> into a couple smaller patches we can commit as we go. >> >> -Todd >> >> On Wed, Nov 17, 2010 at 2:08 PM, Hairong Kuang <kuang.hair...@fb.com> >> wrote: >> >>> >> >>> Hi Todd, >> >>> >> >>> Here is the list of fsimage jiras that I am working on: >> >>> >> >>> HDFS-1481 // small change >> >>> HDFS-1458 // small change >> >>> HDFS-1496 // should be checked into 022, I still have no clue how to >> fix >> >>> it >> >>> HDFS-1070 // this one has a simple idea, but turns out that it needs >> >>> major code re-organization. I am still struggling with how to make my >> code >> >>> change easier to review. But this will definitely collide with yours. >> >>> >> >>> Hairong >> >>> >> >>> On 11/17/10 11:57 AM, "Todd Lipcon" <t...@cloudera.com> wrote: >> >>> >> >>> Hey all, >> >>> >> >>> As many of you know, I've been working on HDFS-1073 for a few months, >> and >> >>> we're hoping to get it in for 0.22 (OOM has given the OK to merge this >> into >> >>> branch and Sanjay has helpfully gotten some commitment from Yahoo QA >> to help >> >>> test it). >> >>> >> >>> It's been difficult recently to work on the patch as a lot of changes >> have >> >>> gone in around FSImage and FSEditLog (eg checksumming, parallel load, >> etc) >> >>> so the patch falls out of date quickly and is hard to resolve the >> changes >> >>> since my patch moves a lot of code. Ivan Kelly over at Yahoo Barcelona >> has >> >>> also been working on a refactor and I think running into a similar >> issue. >> >>> >> >>> So, I just wanted to ping everyone who has been working on this area >> of >> >>> the code to find out if there are any more remaining JIRAs that you're >> >>> hoping to commit in the next several weeks that will touch FSImage and >> >>> FSEditLog. If so, we should try to agree on an order in which to >> commit them >> >>> so we don't end up stomping on each other, and we can help review each >> >>> other's work to move quicker. >> >>> >> >>> If you can just reply with a list of any remaining uncommitted JIRAs >> that >> >>> touch FSImage/FSEditLog that would be great, then we can work from >> there on >> >>> how to proceed and minimize the amount of patch rebasing we all have >> to do. >> >>> >> >>> Thanks! >> >>> -Todd >> >> >> >> >> >> >> >> -- >> >> Todd Lipcon >> >> Software Engineer, Cloudera >> >> >> >> > > > -- > Todd Lipcon > Software Engineer, Cloudera > -- Todd Lipcon Software Engineer, Cloudera
