Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

2012-03-19 Thread Todd Lipcon
On Mon, Mar 19, 2012 at 9:17 AM, Suresh Srinivas wrote: > I am sure Jitendra understands what Todd meant, given he was quite involved > in the work. As Jitendra said, I would like to keep the wire type from the > implementation type. Even for internal protocols. Rolling upgrades is > important. H

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

2012-03-19 Thread Suresh Srinivas
I am sure Jitendra understands what Todd meant, given he was quite involved in the work. As Jitendra said, I would like to keep the wire type from the implementation type. Even for internal protocols. Rolling upgrades is important. I understand where Todd is coming from. We did this work for 10 pr

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

2012-03-19 Thread Todd Lipcon
Exactly. We still have wire compatibility, we just avoid the redundant work of wrapping the whole thing. -Todd On Mon, Mar 19, 2012 at 8:11 AM, Robert Evans wrote: > I think what we are talking about here is removing some of the extra layers > of abstraction in java.  The wire protocol used wil

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

2012-03-19 Thread Robert Evans
I think what we are talking about here is removing some of the extra layers of abstraction in java. The wire protocol used will be identical in either case. It is just that we would have to use the Protocol Buffer Builder APIs instead of wrapping them with our own custom getters/setters. I am

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

2012-03-19 Thread Jitendra Pandey
Wire compatibility in hdfs private protocols between different components is also important for rolling upgrades. We do want to support upgrading different components of a cluster independent of each other and wire compatibility is one of the essential prerequisites. Therefore, even if some prot

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

2012-03-18 Thread Aaron T. Myers
I'm also +1 to at least taking them out for the intra-cluster protocols. -- Aaron T. Myers Software Engineer, Cloudera On Sun, Mar 18, 2012 at 7:49 PM, Eli Collins wrote: > On Sat, Mar 17, 2012 at 3:31 PM, Todd Lipcon wrote: > > Hi all, > > > > I've been working on some patches recently that

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

2012-03-18 Thread Eli Collins
On Sat, Mar 17, 2012 at 3:31 PM, Todd Lipcon wrote: > Hi all, > > I've been working on some patches recently that required adding a new > protocol and some RPC calls that will be used entirely internally to > HDFS -- i.e the types and functions are never exposed to clients. The > process to do thi