Can you re-run these tests with processor binding enabled? On Dec 16, 2013, at 6:36 PM, tmish...@jcity.maeda.co.jp wrote:
> > > Hi, > > I usually use infiniband network, where openmpi-1.7.3 and 1.6.5 works fine. > > The other days, I had a chance to use tcp network(1GbE) and I noticed that > my application with openmpi-1.7.3 was quite slower than openmpi-1.6.5. > So, I did OSU MPI Bandwidth Test v3.1.1 as shown below, which shows > bandwidth for smaller size(< 1024) is very slow compared with 1.6.5. > In addition, the latency for larger size( >65536 ) seems to be strange. > > Does this depend on our local environment or some mca parameter would be > necesarry? I'm afraid that something is wrong with tcp of openmpi-1.7.3. > > openmpi-1.7.3: > > [mishima@node07 OMB-3.1.1]$ mpirun -np 2 -host node07,node08 -mca tbl > ^openib osu_bw > # OSU MPI Bandwidth Test v3.1.1 > # Size Bandwidth (MB/s) > 1 0.00 > 2 0.01 > 4 0.01 > 8 0.03 > 16 0.05 > 32 0.10 > 64 0.32 > 128 0.37 > 256 0.87 > 512 5.97 > 1024 20.00 > 2048 182.87 > 4096 202.53 > 8192 215.14 > 16384 225.16 > 32768 228.58 > 65536 115.23 > 131072 198.24 > 262144 193.38 > 524288 233.03 > 1048576 227.31 > 2097152 233.07 > 4194304 233.25 > > [mishima@node07 OMB-3.1.1]$ mpirun -np 2 -host node07,node08 -mca btl > ^openib osu_latency > # OSU MPI Latency Test v3.1.1 > # Size Latency (us) > 0 19.23 > 1 19.57 > 2 19.52 > 4 19.88 > 8 20.44 > 16 20.38 > 32 20.78 > 64 21.14 > 128 21.75 > 256 23.20 > 512 26.12 > 1024 31.54 > 2048 41.72 > 4096 64.55 > 8192 107.52 > 16384 179.23 > 32768 251.58 > 65536 20689.68 > 131072 21179.79 > 262144 20168.56 > 524288 22984.83 > 1048576 25994.54 > 2097152 30929.55 > 4194304 38028.48 > > openmpi-1.6.5: > > [mishima@node07 OMB-3.1.1]$ mpirun -np 2 -host node07,node08 -mca tbl > ^openib osu_bw > # OSU MPI Bandwidth Test v3.1.1 > # Size Bandwidth (MB/s) > 1 0.22 > 2 0.45 > 4 0.89 > 8 1.77 > 16 3.57 > 32 7.15 > 64 14.28 > 128 28.58 > 256 57.17 > 512 96.44 > 1024 152.38 > 2048 182.84 > 4096 203.17 > 8192 215.13 > 16384 225.05 > 32768 100.58 > 65536 225.24 > 131072 182.92 > 262144 192.82 > 524288 212.92 > 1048576 233.35 > 2097152 233.72 > 4194304 233.89 > > [mishima@node07 OMB-3.1.1]$ mpirun -np 2 -host node07,node08 -mca btl > ^openib osu_latency > # OSU MPI Latency Test v3.1.1 > # Size Latency (us) > 0 17.24 > 1 17.30 > 2 17.29 > 4 17.30 > 8 24.32 > 16 17.24 > 32 17.80 > 64 17.91 > 128 19.08 > 256 20.81 > 512 22.83 > 1024 27.82 > 2048 39.54 > 4096 52.66 > 8192 97.70 > 16384 143.23 > 32768 215.02 > 65536 481.08 > 131072 800.64 > 262144 1475.12 > 524288 2698.62 > 1048576 4992.31 > 2097152 9558.96 > 4194304 20801.50 > > Regards, > Tetsuya Mishima > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
