Hi, I am trying to benchmark Open MPI performance on 10G Ethernet network between two hosts. The performance numbers of benchmarks are less than expected. The maximum bandwidth achieved by OMPI-C is 5678 Mbps and I was expecting around 9000+ Mbps. Moreover latency is also quite higher than expected, ranging from 37 to 59 us. Here is complete set of numbers.
*LatencyOpen MPI C Size Time (us)* 1 37.76 2 37.75 4 37.78 8 55.17 16 37.89 32 39.08 64 37.78 128 59.46 256 39.37 512 40.39 1024 47.18 2048 47.84 *BandwidthOpen MPI C Size (Bytes) Bandwidth (Mbps)* 2048 412.22 4096 539.59 8192 827.73 16384 1655.35 32768 3274.3 65536 1995.22 131072 3270.84 262144 4316.22 524288 5019.46 1048576 5236.17 2097152 5362.61 4194304 5495.2 8388608 5565.32 16777216 5678.32 My environments consists of two hosts having point-to-point (switch-less) 10Gbps Ethernet connection. Environment (OS, user, directory structure etc) on both hosts is exactly same. There is no NAS or shared file system between both hosts. Following are configuration and job launching commands that I am using. Moreover, I have attached output of script ompi_info --all. Configuration commmand: ./configure --enable-mpi-java --prefix=/home/mpj/installed/openmpi_installed CC=/usr/bin/gcc --disable-mpi-fortran Job launching command: mpirun -np 2 -hostfile machines -npernode 1 ./latency.out Are these numbers okay? If not then please suggest performance tuning steps... Thanks -- Ansar Javed HPC Lab SEECS NUST Contact: +92 334 438 9394 Email: muhammad.an...@seecs.edu.pk
ompi_info.tar.bz2
Description: BZip2 compressed data
