On Sat, Dec 02, 2006 at 10:31:30AM -0500, Jeff Squyres wrote: > FWIW, especially on NUMA machines (like AMDs), physical access to > network resources (such as NICs / HCAs) can be much faster on > specific sockets.
Yes, the penalty is actually 50 ns per hop, and you pay it on both sides. So our 1.28 usec latency rises to 1.38 if you are on the other socket on both ends, using a 2 socket motherboard. In our MPI we also have code that pays attention to the NUMA topology if you have 2 of our HTX HCAs in a system. -- greg
