On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon <t...@cloudera.com> wrote:
> On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas <sur...@hortonworks.com > >wrote: > > > Todd, > > > > As I indicated in my comments on the jira, I think some of the design > > discussions and further simplification of design should happen before the > > merge. See - > > > > > https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470680 > > > > > I still haven't heard any actual concrete suggestion for a design > improvement. Just abstract notions that "it should be simpler". I could say > the same about several other features that have been done in the past (e.g > federation or YARN) but chose not to because I didn't participate in the > development. I generally have faith that, if several other people spent > several months working on a project, then there must be good reasons for > the complexity that I'm probably overlooking. > I think I have participated enough in this work. Merge time seemed like a good time to review this more thoroughly given this jira has required quite a bit of educating myself and spending a lot of my time on this because of need to understand the complexities/subtletie how paxos and ZAB applies to the namenode journals. Please do not misunderstand that I am trying to block this work from being merged. I am supportive of this as you have seen in my previous response in this thread. All I want to see is if we can conclude the discussions and incorporate some the comments that come up after it. > > Several of us (not just I) spent lots of time on this over the last several > months, with all of the work going on in the open. My issue with this > discussion happening now is that no one saw fit to raise these questions > months ago when the design was first proposed. For example, this most > recent question about whether NewEpoch and PrepareRecovery should be > separate RPCs could have been raised in late June when the code in question > was first introduced as a patch. > > Nor was anything raised when I gave a "heads up" that the branch was > complete and nearly ready for merge ~3 weeks ago. Only once I actually > called a merge vote has this discussion started. That doesn't seem like a > healthy way to do development. > Again Todd, you are reading more than what is intended. As I said earlier merge time is a good time to have complete picture and an opportunity to look at final design and implementation. > What are the criteria, then, for merging? I don't think it's possible to > definitively "prove" that a design is "simple". At some level it's a matter > of taste. So if the current design works, and is tested, and has people > signed up to support it and run it, why not merge? Just like any other part > of HDFS, we can continue to _improve_ it after it is in trunk. There are > many features that we commit and then improve later on when someone comes > up with a way to simplify it. > One concern I have is, once this is merged into trunk I feel that any proposed improvements may be blocked. Given that why not just wait for these discussions to complete and do the work in this branch? What is the need for getting this into trunk immediately? If the problem is one of investment of your time, I have already offered to help out. > To be clear about the current status of the vote: you are officially > vetoing the merge? For now I am going to vote not to merge until the discussion completes. Regards, Suresh
