On Jul 29, 2008, at 4:52 AM, Andy Georgi wrote:
The upcoming Open MPI v1.3 series will support iWARP, which gives
much
better latency than that. I don't know all the Chelsio models
offhand;
are those iWARP-capable cards?
Thanks for the fast answer. So is this latency normal for TCP
communications over MPI!?
There are multiple factors here:
- TCP stacks are not normally optimized for low latency
- We've spent much more time optimizing OMPI's communications over
other networks because we assume that TCP latencies will be high
So yes, when using "normal" Linux TCP, latencies in the tens of us are
not uncommon.
Could RDMA maybe reduce the latency?
It's not so much RDMA that reduces the latency as it is OS-bypass, NIC
offload, etc. RDMA can help as well, but I wouldn't call that the
driving factor.
It
should work with those cards but there are still problems with OFED.
iWARP is also one of the features they offer but if it works...
If you're having problems getting iWARP to work with your Chelsio
cards, you should take it up with their support channels. It seems to
work fine for me.
--
Jeff Squyres
Cisco Systems
