Hi all, There's a patch up for 1473 now. If we can get a +1 we can commit this and move on to the next steps in some of these projects.
Thanks -Todd On Thu, Nov 18, 2010 at 12:42 AM, Ivan Kelly <iv...@yahoo-inc.com> wrote: > This patch looks really good. It shouldn't affect our refactoring too much > and will actually make things a lot easier for the next part (breaking the > circular dependency between namesystem and fsimage). > > -Ivan > > > On 18 Nov 2010, at 02:27, Todd Lipcon wrote: > > I put up a sketch of a patch for 1473. It compiles and passes at least a > couple of the unit tests, but more work to be done on it. Would appreciate > comments from this crowd if the new classes seem like good divisions. > > -Todd > > On Wed, Nov 17, 2010 at 2:36 PM, Todd Lipcon <t...@cloudera.com> wrote: > >> On Wed, Nov 17, 2010 at 2:34 PM, Hairong Kuang <kuang.hair...@fb.com>wrote: >> >>> You meant HDFS-1473 right? When do you expect that it could be done? >>> >>> >> I was working on HDFS-1473 this morning and could probably get a >> preliminary patch up today.. but when I realized that we have 4 parallel >> refactorings going on I paused my work. If we agree that HDFS-1473 will go >> in before the other ones I can finish it up, but if we're doing the same >> thing we should coordinate instead. >> >> >>> Hairong >>> >>> On 11/17/10 2:29 PM, "Eli Collins" <e...@cloudera.com> wrote: >>> >>> > Agree we should do the refactoring before the features, we should also >>> > add the tests with the refactoring to show they don't break things. >>> > >>> > On Wed, Nov 17, 2010 at 2:27 PM, Todd Lipcon <t...@cloudera.com> >>> wrote: >>> >> OK, looks like there is still quite a bit more in motion here, thanks >>> for >>> >> the list, Hairong. >>> >> How would everyone feel about setting aside those improvements/fixes, >>> doing >>> >> the refactor first, and then going back to them? It should help with >>> testing >>> >> and reduce conflicts between the various ongoing projects. >>> >> Maybe we can discuss quickly on this thread: what are some small >>> refactors >>> >> we can start with to help separate these things? Ivan's project looks >>> really >>> >> good but it seems like it is a pretty large change. If we can break it >>> up >>> >> into a couple smaller patches we can commit as we go. >>> >> -Todd >>> >> On Wed, Nov 17, 2010 at 2:08 PM, Hairong Kuang <kuang.hair...@fb.com> >>> wrote: >>> >>> >>> >>> Hi Todd, >>> >>> >>> >>> Here is the list of fsimage jiras that I am working on: >>> >>> >>> >>> HDFS-1481 // small change >>> >>> HDFS-1458 // small change >>> >>> HDFS-1496 // should be checked into 022, I still have no clue how to >>> fix >>> >>> it >>> >>> HDFS-1070 // this one has a simple idea, but turns out that it needs >>> >>> major code re-organization. I am still struggling with how to make my >>> code >>> >>> change easier to review. But this will definitely collide with yours. >>> >>> >>> >>> Hairong >>> >>> >>> >>> On 11/17/10 11:57 AM, "Todd Lipcon" <t...@cloudera.com> wrote: >>> >>> >>> >>> Hey all, >>> >>> >>> >>> As many of you know, I've been working on HDFS-1073 for a few months, >>> and >>> >>> we're hoping to get it in for 0.22 (OOM has given the OK to merge >>> this into >>> >>> branch and Sanjay has helpfully gotten some commitment from Yahoo QA >>> to help >>> >>> test it). >>> >>> >>> >>> It's been difficult recently to work on the patch as a lot of changes >>> have >>> >>> gone in around FSImage and FSEditLog (eg checksumming, parallel load, >>> etc) >>> >>> so the patch falls out of date quickly and is hard to resolve the >>> changes >>> >>> since my patch moves a lot of code. Ivan Kelly over at Yahoo >>> Barcelona has >>> >>> also been working on a refactor and I think running into a similar >>> issue. >>> >>> >>> >>> So, I just wanted to ping everyone who has been working on this area >>> of >>> >>> the code to find out if there are any more remaining JIRAs that >>> you're >>> >>> hoping to commit in the next several weeks that will touch FSImage >>> and >>> >>> FSEditLog. If so, we should try to agree on an order in which to >>> commit them >>> >>> so we don't end up stomping on each other, and we can help review >>> each >>> >>> other's work to move quicker. >>> >>> >>> >>> If you can just reply with a list of any remaining uncommitted JIRAs >>> that >>> >>> touch FSImage/FSEditLog that would be great, then we can work from >>> there on >>> >>> how to proceed and minimize the amount of patch rebasing we all have >>> to do. >>> >>> >>> >>> Thanks! >>> >>> -Todd >>> >> >>> >> >>> >> >>> >> -- >>> >> Todd Lipcon >>> >> Software Engineer, Cloudera >>> >> >>> >>> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera > > > -- Todd Lipcon Software Engineer, Cloudera
