Hey Todd, Ivan, I've committed the patch for 1473.
Thanks, Eli On Wed, Nov 24, 2010 at 2:25 PM, Todd Lipcon <t...@cloudera.com> wrote: > Hi all, > > There's a patch up for 1473 now. If we can get a +1 we can commit this and > move on to the next steps in some of these projects. > > Thanks > -Todd > > On Thu, Nov 18, 2010 at 12:42 AM, Ivan Kelly <iv...@yahoo-inc.com> wrote: > >> This patch looks really good. It shouldn't affect our refactoring too much >> and will actually make things a lot easier for the next part (breaking the >> circular dependency between namesystem and fsimage). >> >> -Ivan >> >> >> On 18 Nov 2010, at 02:27, Todd Lipcon wrote: >> >> I put up a sketch of a patch for 1473. It compiles and passes at least a >> couple of the unit tests, but more work to be done on it. Would appreciate >> comments from this crowd if the new classes seem like good divisions. >> >> -Todd >> >> On Wed, Nov 17, 2010 at 2:36 PM, Todd Lipcon <t...@cloudera.com> wrote: >> >>> On Wed, Nov 17, 2010 at 2:34 PM, Hairong Kuang <kuang.hair...@fb.com>wrote: >>> >>>> You meant HDFS-1473 right? When do you expect that it could be done? >>>> >>>> >>> I was working on HDFS-1473 this morning and could probably get a >>> preliminary patch up today.. but when I realized that we have 4 parallel >>> refactorings going on I paused my work. If we agree that HDFS-1473 will go >>> in before the other ones I can finish it up, but if we're doing the same >>> thing we should coordinate instead. >>> >>> >>>> Hairong >>>> >>>> On 11/17/10 2:29 PM, "Eli Collins" <e...@cloudera.com> wrote: >>>> >>>> > Agree we should do the refactoring before the features, we should also >>>> > add the tests with the refactoring to show they don't break things. >>>> > >>>> > On Wed, Nov 17, 2010 at 2:27 PM, Todd Lipcon <t...@cloudera.com> >>>> wrote: >>>> >> OK, looks like there is still quite a bit more in motion here, thanks >>>> for >>>> >> the list, Hairong. >>>> >> How would everyone feel about setting aside those improvements/fixes, >>>> doing >>>> >> the refactor first, and then going back to them? It should help with >>>> testing >>>> >> and reduce conflicts between the various ongoing projects. >>>> >> Maybe we can discuss quickly on this thread: what are some small >>>> refactors >>>> >> we can start with to help separate these things? Ivan's project looks >>>> really >>>> >> good but it seems like it is a pretty large change. If we can break it >>>> up >>>> >> into a couple smaller patches we can commit as we go. >>>> >> -Todd >>>> >> On Wed, Nov 17, 2010 at 2:08 PM, Hairong Kuang <kuang.hair...@fb.com> >>>> wrote: >>>> >>> >>>> >>> Hi Todd, >>>> >>> >>>> >>> Here is the list of fsimage jiras that I am working on: >>>> >>> >>>> >>> HDFS-1481 // small change >>>> >>> HDFS-1458 // small change >>>> >>> HDFS-1496 // should be checked into 022, I still have no clue how to >>>> fix >>>> >>> it >>>> >>> HDFS-1070 // this one has a simple idea, but turns out that it needs >>>> >>> major code re-organization. I am still struggling with how to make my >>>> code >>>> >>> change easier to review. But this will definitely collide with yours. >>>> >>> >>>> >>> Hairong >>>> >>> >>>> >>> On 11/17/10 11:57 AM, "Todd Lipcon" <t...@cloudera.com> wrote: >>>> >>> >>>> >>> Hey all, >>>> >>> >>>> >>> As many of you know, I've been working on HDFS-1073 for a few months, >>>> and >>>> >>> we're hoping to get it in for 0.22 (OOM has given the OK to merge >>>> this into >>>> >>> branch and Sanjay has helpfully gotten some commitment from Yahoo QA >>>> to help >>>> >>> test it). >>>> >>> >>>> >>> It's been difficult recently to work on the patch as a lot of changes >>>> have >>>> >>> gone in around FSImage and FSEditLog (eg checksumming, parallel load, >>>> etc) >>>> >>> so the patch falls out of date quickly and is hard to resolve the >>>> changes >>>> >>> since my patch moves a lot of code. Ivan Kelly over at Yahoo >>>> Barcelona has >>>> >>> also been working on a refactor and I think running into a similar >>>> issue. >>>> >>> >>>> >>> So, I just wanted to ping everyone who has been working on this area >>>> of >>>> >>> the code to find out if there are any more remaining JIRAs that >>>> you're >>>> >>> hoping to commit in the next several weeks that will touch FSImage >>>> and >>>> >>> FSEditLog. If so, we should try to agree on an order in which to >>>> commit them >>>> >>> so we don't end up stomping on each other, and we can help review >>>> each >>>> >>> other's work to move quicker. >>>> >>> >>>> >>> If you can just reply with a list of any remaining uncommitted JIRAs >>>> that >>>> >>> touch FSImage/FSEditLog that would be great, then we can work from >>>> there on >>>> >>> how to proceed and minimize the amount of patch rebasing we all have >>>> to do. >>>> >>> >>>> >>> Thanks! >>>> >>> -Todd >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Todd Lipcon >>>> >> Software Engineer, Cloudera >>>> >> >>>> >>>> >>> >>> >>> -- >>> Todd Lipcon >>> Software Engineer, Cloudera >>> >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> >> >> > > > -- > Todd Lipcon > Software Engineer, Cloudera >
