Re: DataNodeRegistration Serialization

2014-03-23 Thread Andrew Wang
I think the code is the only documentation. A Writable is just something that has a readFrom and a writeTo method, the contents of these methods can be essentially arbitrary. Best, Andrew On Sun, Mar 23, 2014 at 7:43 PM, Dhaivat Pandya wrote: > Another question: is there some kind of documentat

Re: DataNodeRegistration Serialization

2014-03-23 Thread Dhaivat Pandya
Another question: is there some kind of documentation (other than the code) that specifies the kind of serialization algorithm Writables use? I think I may have to port the concept over to another language. Thanks, Dhaivat On Sun, Mar 23, 2014 at 9:11 AM, Dhaivat Pandya wrote: > Thank you very

Re: DataNodeRegistration Serialization

2014-03-23 Thread Dhaivat Pandya
Thank you very much, Andrew. On Sat, Mar 22, 2014 at 7:02 PM, Andrew Wang wrote: > Hi Dhaivat, > > Take a look at DatanodeRegistration and it's parent class DatanodeID. DR is > a Writable, meaning it's a custom serialization format. Hadoop 2 uses > protobuf for the RPC serialization rather than

Re: DataNodeRegistration Serialization

2014-03-22 Thread Andrew Wang
Hi Dhaivat, Take a look at DatanodeRegistration and it's parent class DatanodeID. DR is a Writable, meaning it's a custom serialization format. Hadoop 2 uses protobuf for the RPC serialization rather than writables (with the exception of DN data transfer). https://github.com/apache/hadoop-common/

DataNodeRegistration Serialization

2014-03-22 Thread Dhaivat Pandya
Hi everyone, I'm currently working on an application that requires some important details about the DataNode registration (w/ NameNode) procedure. Specifically, I have understood (after Wireshark-ing and looking through the Hadoop code) that the DataNode is registered with the NameNode using a si