Hi Ralph, Yes, you are right. I should have also tested NetPipe-MPI version earlier. I ran NetPipe-MPI version on 10G Ethernet and maximum bandwidth achieved is 5872 Mbps. Moreover, maximum bandwidth achieved by osu_bw test is 6080 Mbps. I have used OSU-Micro-Benchmarks version 4.3.
On Wed, Apr 16, 2014 at 3:40 PM, Ralph Castain <r...@open-mpi.org> wrote: > I apologize, but I am now confused. Let me see if I can translate: > > * you ran the non-MPI version of the NetPipe benchmark and got 9.5Gps on a > 10Gps network > > * you ran iperf and got 9.61Gps - however, this has nothing to do with > MPI. Just tests your TCP stack > > * you tested your bandwidth program on a 1Gps network and got about 90% > efficiency. > > Is the above correct? If so, my actual suggestion was to run the MPI > version of NetPipe and to use the OSB benchmark program as well. Your > program might well be okay, but benchmarking is a hard thing to get right > in a parallel world, so you might as well validate it by cross-checking the > result. > > I suggest this mostly because your performance numbers are far worse than > anything we've measured using those standard benchmarks, and so we should > first ensure we aren't chasing a ghost. > > > > > > On Wed, Apr 16, 2014 at 1:41 AM, Muhammad Ansar Javed < > muhammad.an...@seecs.edu.pk> wrote: > >> Yes, I have tried NetPipe-Java and iperf for bandwidth and configuration >> test. NetPipe Java achieves maximum 9.40 Gbps while iperf achieves maximum >> 9.61 Gbps bandwidth. I have also tested my bandwidth program on 1Gbps >> Ethernet connection and it achieves 901 Mbps bandwidth. I am using the same >> program for 10G network benchmarks. Please find attached source file of >> bandwidth program. >> >> As far as --bind-to core is concerned, I think it is working fine. Here >> is output of --report-bindings switch. >> [host3:07134] MCW rank 0 bound to socket 0[core 0[hwt 0]]: [B/././.] >> [host4:10282] MCW rank 1 bound to socket 0[core 0[hwt 0]]: [B/././.] >> >> >> >> >> On Tue, Apr 15, 2014 at 8:39 PM, Ralph Castain <r...@open-mpi.org> wrote: >> >>> Have you tried a typical benchmark (e.g., NetPipe or OMB) to ensure the >>> problem isn't in your program? Outside of that, you might want to >>> explicitly tell it to --bind-to core just to be sure it does so - it's >>> supposed to do that by default, but might as well be sure. You can check by >>> adding --report-binding to the cmd line. >>> >>> >>> On Apr 14, 2014, at 11:10 PM, Muhammad Ansar Javed < >>> muhammad.an...@seecs.edu.pk> wrote: >>> >>> Hi, >>> I am trying to benchmark Open MPI performance on 10G Ethernet network >>> between two hosts. The performance numbers of benchmarks are less than >>> expected. The maximum bandwidth achieved by OMPI-C is 5678 Mbps and I was >>> expecting around 9000+ Mbps. Moreover latency is also quite higher than >>> expected, ranging from 37 to 59 us. Here is complete set of numbers. >>> >>> >>> >>> *LatencyOpen MPI C Size Time (us)* >>> 1 37.76 >>> 2 37.75 >>> 4 37.78 >>> 8 55.17 >>> 16 37.89 >>> 32 39.08 >>> 64 37.78 >>> 128 59.46 >>> 256 39.37 >>> 512 40.39 >>> 1024 47.18 >>> 2048 47.84 >>> >>> >>> >>> >>> *BandwidthOpen MPI C Size (Bytes) Bandwidth (Mbps)* >>> 2048 412.22 >>> 4096 539.59 >>> 8192 827.73 >>> 16384 1655.35 >>> 32768 3274.3 >>> 65536 1995.22 >>> 131072 3270.84 >>> 262144 4316.22 >>> 524288 5019.46 >>> 1048576 5236.17 >>> 2097152 5362.61 >>> 4194304 5495.2 >>> 8388608 5565.32 >>> 16777216 5678.32 >>> >>> >>> My environments consists of two hosts having point-to-point >>> (switch-less) 10Gbps Ethernet connection. Environment (OS, user, directory >>> structure etc) on both hosts is exactly same. There is no NAS or shared >>> file system between both hosts. Following are configuration and job >>> launching commands that I am using. Moreover, I have attached output of >>> script ompi_info --all. >>> >>> Configuration commmand: ./configure --enable-mpi-java >>> --prefix=/home/mpj/installed/openmpi_installed CC=/usr/bin/gcc >>> --disable-mpi-fortran >>> >>> Job launching command: mpirun -np 2 -hostfile machines -npernode 1 >>> ./latency.out >>> >>> Are these numbers okay? If not then please suggest performance tuning >>> steps... >>> >>> Thanks >>> >>> -- >>> Ansar Javed >>> HPC Lab >>> SEECS NUST >>> Contact: +92 334 438 9394 >>> Email: muhammad.an...@seecs.edu.pk >>> <ompi_info.tar.bz2>_______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >> >> >> >> -- >> Regards >> >> >> Ansar Javed >> HPC Lab >> SEECS NUST >> Contact: +92 334 438 9394 >> Email: muhammad.an...@seecs.edu.pk >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > -- Regards Ansar Javed HPC Lab SEECS NUST Contact: +92 334 438 9394 Email: muhammad.an...@seecs.edu.pk
