Thanks Jeff and Andrew, those responses were well thought out and very informative.
I'll run the same test explicitly using TCP for a couple runs and then mvapi a couple times. The results should give the users a warm fuzzy that OpenMPI is in fact using the expensive faster network (rather than cheep gig-e). Thanks again. -----Original Message----- From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Jeff Squyres Sent: Wednesday, December 20, 2006 10:01 PM To: Jeff Squyres Cc: Open MPI Users Subject: Re: [OMPI users] Infiniband - Any suggestions on "How can you proveto me that OpenMPI is using it?" On Dec 20, 2006, at 7:04 PM, Jeff Squyres wrote: >> I've been asked by the owner of the cluster "How can you prove to me >> that this openmpi job is using the Infiniband network?" >> >> At first I thought a simple netstat -an on the compute nodes might >> tell >> me, however I don't see the Infiniband IP's in the list so I'm >> thinking >> maybe I need to be looking elsewhere. > > That's correct. I completely forgot to explain this comment, sorry. Open MPI does not use TCP over IB -- it uses the native "verbs" interface (i.e., a lower lever than TCP over IB), and therefore a) doesn't show up in tcpdump/netstat/etc., and b) gets much better performance (TCP demands much processing on the host CPU, whereas verbs can be offloaded to the processor on the HCA). -- Jeff Squyres Server Virtualization Business Unit Cisco Systems _______________________________________________ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
