On Jul 28, 2008, at 2:53 PM, Andy Georgi wrote:
we use Chelsio S320E-CXA adapters (http://www.chelsio.com/assetlibrary/products/S320E%20Product%20Brief%20080424.pdf ) in one of our clusters. After tuning the kernel i measured the ping pong latency via NetPIPE and got ~12us which is pretty good for TCP i think. So i wrote a simple ping-pong-kernel and was really terrified about the ~45us i got with OpenMPI 1.2.6. Are there any hints how we can reduce the MPI latency? To increase the bandwidth we already set the buffer sizes but we couldn't find a parameter which can be relevant for the latency. Every hint is welcome.
The upcoming Open MPI v1.3 series will support iWARP, which gives much better latency than that. I don't know all the Chelsio models offhand; are those iWARP-capable cards?
-- Jeff Squyres Cisco Systems
