Hi,
Following your comments, you should be using
address sdp 192.168.90.1:7778 <http://192.168.20.1:7778>;
Instead of:
address sdp 192.168.20.1:7778 <http://192.168.20.1:7778>;
We have IB working in centos 5.5 too, without problems. Let me know if
it's not this!
Have fun.
Em 17-09-2010 00:29, J. Ryan Earl escreveu:
Hello,
I've recently setup an InfiniBand 40Gbit interconnect between two
nodes to run DRBD on top of some pretty fast storage. I am able to
get DRBD to work over Ethernet and IPoIB, however, when I try to
enable SDP for the lower latency, lower overhead communication I'm
getting connection errors:
block drbd0: conn( Unconnected -> WFConnection )
block drbd0: connect failed, err = -22
block drbd0: connect failed, err = -22
block drbd0: connect failed, err = -22
I have the MLNX_OFED installed on CentOS5.5 with SDP active:
# rpm -qa|grep sdp
libsdp-devel-1.1.100-0.1.g920ea31
sdpnetstat-1.60-0.2.g8844f04
libsdp-1.1.100-0.1.g920ea31
libsdp-1.1.100-0.1.g920ea31
libsdp-devel-1.1.100-0.1.g920ea31
libsdp-debuginfo-1.1.100-0.1.g920ea31
[r...@node02 log]# netperf -f g -H 192.168.20.1 -c -C
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.20.1
(192.168.20.1) port 0 AF_INET
Recv Send Send Utilization Service
Demand
Socket Socket Message Elapsed Send Recv Send
Recv
Size Size Size Time Throughput local remote local
remote
bytes bytes bytes secs. 10^9bits/s % S % S us/KB
us/KB
87380 65536 65536 10.00 16.15 1.74 4.61 0.211
0.562
[r...@node02 log]# LD_PRELOAD="libsdp.so" netperf -f g -H 192.168.20.1
-c -C
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.20.1
(192.168.20.1) port 0 AF_INET
Recv Send Send Utilization Service
Demand
Socket Socket Message Elapsed Send Recv Send
Recv
Size Size Size Time Throughput local remote local
remote
bytes bytes bytes secs. 10^9bits/s % S % S us/KB
us/KB
87380 65536 65536 10.01 24.67 3.18 3.28 0.253
0.262
There is a significant (50-100%) increase in bandwidth and decrease in
latency using SDP instead of IPoIB, so even though IPoIB works I'd
like to use the SDP method.
Here's my node stanzas:
<snip>
# 90-subnet = Ethernet, 20-subnet = InfiniBand
on node01 {
address 192.168.20.1:7778 <http://192.168.20.1:7778>;
# address sdp 192.168.20.1:7778 <http://192.168.20.1:7778>;
# address 192.168.90.1:7778 <http://192.168.90.1:7778>;
}
on node02 {
address 192.168.20.2:7778 <http://192.168.20.2:7778>;
# address sdp 192.168.20.2:7778 <http://192.168.20.2:7778>;
# address 192.168.90.2:7778 <http://192.168.90.2:7778>;
}
</snip>
If I uncomment on the SDP-based address, I get connection error -22
and haven't been able to surmise what that means. Any ideas? Running
the lastest DRBD:
# rpm -qa|grep drbd
drbd83-8.3.8.1-1
kmod-drbd83-8.3.8.1-1
Any help would be appreciated,
-JR
_______________________________________________
drbd-user mailing list
[email protected]
http://lists.linbit.com/mailman/listinfo/drbd-user
--
Igor Neves<[email protected]>
3GNTW - Tecnologias de Informação, Lda
SIP: [email protected]
MSN: [email protected]
JID: [email protected]
PSTN: 00351 252377120
_______________________________________________
drbd-user mailing list
[email protected]
http://lists.linbit.com/mailman/listinfo/drbd-user
