Have you checked out Mellanox's UDA ? There have been a lot of efforts from RoCE providers to get Hadoop optimized implementations with modern networking hardware. Even RedHat has announced HPN (High Performance Networking) interfaces to allow modern networks. It might take a few years to get there for Open Source Hadoop implementations, but believe me, every Hadoop vendor based off of open source Hadoop is incorporating new network architectures in their proprietary distro. We were there first. That's all.
- milind --- Milind Bhandarkar Chief Scientist Pivotal +1-650-523-3858 (W) +1-408-666-8483 (C) On Tue, Oct 8, 2013 at 6:13 PM, Yasin Celik <yasinceli...@gmail.com> wrote: > Hello > > How do you get hadoop worked with infiniband? Do you use SDP? > I only need to use RPC of hadoop for now. Any help on how to get it worked? > > Thanks > Yasin > > > On Tue, Oct 8, 2013 at 8:57 PM, Milind Bhandarkar < > mbhandar...@gopivotal.com > > wrote: > > > Yes, we have. It works very well, but it is considered too niche by folks > > who insist on buying the least capable hardware for their test clusters, > > and therefore, recommend such underpowered clusters to customers as well. > > > > - milind > > > > > On Oct 8, 2013, at 16:49, Yasin Celik <yasinceli...@gmail.com> wrote: > > > > > > Hello, > > > > > > I looked around but I think it seems there is no Sockets Direct > > > Protocol(SDP) support in Hadoop RPC. > > > I basically want to take advantage of Infiniband when transferring > data. > > > Is there any known support for this? Has anybody tried this before? > > > > > > Best > > > > > > --------- > > > Yasin Celik > > > > > > -- > --------- > Yasin Celik >
