0.20.2 is too old. The current stable release of hadoop is 1.2.1. Anyway, for 0.20.2, you probably need -Djava.net.preferIPv4Stack=true as well, which is already set for Hadoop 1+. BTW, you'll need to set all these in mapred.child.java.opts as well.
On Thu, Oct 10, 2013 at 5:12 AM, Yasin Celik <yasinceli...@gmail.com> wrote: > Hi Luke, > > I actually tested hadoop RPC using SDP. But it did not work for me. > I used hadoop-core 0.20.2 > Following is the error I received: > > java.io.IOException: Address family not supported by protocol > at sun.net.sdp.SdpSupport. > convert0(Native Method) > at sun.net.sdp.SdpSupport.convertSocket(SdpSupport.java:70) > at sun.net.sdp.SdpProvider.convertTcpToSdpIfMatch(SdpProvider.java:300) > at sun.net.sdp.SdpProvider.implBeforeTcpBind(SdpProvider.java:323) > at sun.net.NetHooks.beforeTcpBind(NetHooks.java:86) > at > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:198) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at org.apache.hadoop.ipc.Server.bind(Server.java:188) > at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:253) > at org.apache.hadoop.ipc.Server.<init>(Server.java:1026) > at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:488) > at org.apache.hadoop.ipc.RPC.getServer(RPC.java:450) > > My sdp.conf file is like: > bind * * > connect 10.xx.xx.0/24 <http://10.80.70.0/24> 50030-50050 > > I also tried this for one specific node: > > bind 10.80.70.13 50031 > connect 10.xx.xx.0/24 <http://10.80.70.0/24> 50030-50050 > > > Thank you very much. > > > On Thu, Oct 10, 2013 at 4:00 AM, Luke Lu <l...@apache.org> wrote: > > > Hadoop works with Java 7, which has builtin SDP support. You'll need to > > pass SDP related properties (-Dcom.sun.sdp.conf=your_sdp.conf) to hadoop > > JVMs via env var HADOOP_OPTS. > > > > One caveat of SDP is the 64KB message size (performance cliff beyond > 64KB), > > which _should_ work with default hadoop config, where dfs packet size is > > 64KB. > > > > __Luke > > > > > > On Tue, Oct 8, 2013 at 4:49 PM, Yasin Celik <yasinceli...@gmail.com> > > wrote: > > > > > Hello, > > > > > > I looked around but I think it seems there is no Sockets Direct > > > Protocol(SDP) support in Hadoop RPC. > > > I basically want to take advantage of Infiniband when transferring > data. > > > Is there any known support for this? Has anybody tried this before? > > > > > > Best > > > > > > --------- > > > Yasin Celik > > > > > > > > > -- > --------- > Yasin Celik >
