On Wednesday 20 May 2009, Pavel Shamis (Pasha) wrote: > > With the file Pavel has provided things have changed to the following. > > (maybe someone can confirm) > > > > If message size < 8192 > > bruck > > else > > pairwise > > end > > You are right here. Target of my conf file is disable basic_linear for > medium message side.
Disabling basic_linear seems like a good idea but your config file sets the cut-off at 128 Bytes for 64-ranks (the field you set to 8192 seems to result in a message size of that value divided by the number of ranks). In my testing bruck seems to win clearly (at least for 64 ranks on my IB) up to 2048. Hence, the following line may be better: 131072 2 0 0 # switch to pair wise for size 128K/nranks Disclaimer: I guess this could differ quite a bit for nranks!=64 and different btls. Here are some figures for this part of the package size range: all_bruck bw for 10 x 10 B : 13.7 Mbytes/s time was: 922.0 �s bw for 10 x 500 B : 45.9 Mbytes/s time was: 13.7 ms bw for 10 x 1000 B : 122.7 Mbytes/s time was: 10.3 ms bw for 10 x 1500 B : 86.9 Mbytes/s time was: 21.8 ms bw for 10 x 2000 B : 120.1 Mbytes/s time was: 21.0 ms bw for 10 x 2047 B : 92.6 Mbytes/s time was: 27.9 ms bw for 10 x 2048 B : 107.3 Mbytes/s time was: 24.1 ms bw for 10 x 2400 B : 93.7 Mbytes/s time was: 32.3 ms bw for 10 x 2800 B : 73.0 Mbytes/s time was: 48.3 ms bw for 10 x 2900 B : 79.5 Mbytes/s time was: 45.9 ms bw for 10 x 2925 B : 89.3 Mbytes/s time was: 41.3 ms bw for 10 x 2950 B : 72.7 Mbytes/s time was: 51.1 ms bw for 10 x 2975 B : 75.2 Mbytes/s time was: 49.8 ms bw for 10 x 3000 B : 74.9 Mbytes/s time was: 50.5 ms bw for 10 x 3100 B : 95.9 Mbytes/s time was: 40.7 ms totaltime was: 479.5 ms all_pair bw for 10 x 10 B : 414.2 kbytes/s time was: 30.4 ms bw for 10 x 500 B : 19.8 Mbytes/s time was: 31.9 ms bw for 10 x 1000 B : 43.3 Mbytes/s time was: 29.1 ms bw for 10 x 1500 B : 63.3 Mbytes/s time was: 29.9 ms bw for 10 x 2000 B : 81.2 Mbytes/s time was: 31.0 ms bw for 10 x 2047 B : 82.3 Mbytes/s time was: 31.3 ms bw for 10 x 2048 B : 83.0 Mbytes/s time was: 31.1 ms bw for 10 x 2400 B : 93.6 Mbytes/s time was: 32.3 ms bw for 10 x 2800 B : 105.0 Mbytes/s time was: 33.6 ms bw for 10 x 2900 B : 107.7 Mbytes/s time was: 33.9 ms bw for 10 x 2925 B : 108.1 Mbytes/s time was: 34.1 ms bw for 10 x 2950 B : 109.6 Mbytes/s time was: 33.9 ms bw for 10 x 2975 B : 111.1 Mbytes/s time was: 33.7 ms bw for 10 x 3000 B : 112.1 Mbytes/s time was: 33.7 ms bw for 10 x 3100 B : 114.5 Mbytes/s time was: 34.1 ms totaltime was: 484.1 ms bruckto2k_pair bw for 10 x 10 B : 11.9 Mbytes/s time was: 1.1 ms bw for 10 x 500 B : 100.3 Mbytes/s time was: 6.3 ms bw for 10 x 1000 B : 115.9 Mbytes/s time was: 10.9 ms bw for 10 x 1500 B : 117.2 Mbytes/s time was: 16.1 ms bw for 10 x 2000 B : 95.7 Mbytes/s time was: 26.3 ms bw for 10 x 2047 B : 96.6 Mbytes/s time was: 26.7 ms bw for 10 x 2048 B : 82.2 Mbytes/s time was: 31.4 ms bw for 10 x 2400 B : 94.1 Mbytes/s time was: 32.1 ms bw for 10 x 2800 B : 105.6 Mbytes/s time was: 33.4 ms bw for 10 x 2900 B : 108.4 Mbytes/s time was: 33.7 ms bw for 10 x 2925 B : 108.3 Mbytes/s time was: 34.0 ms bw for 10 x 2950 B : 109.9 Mbytes/s time was: 33.8 ms bw for 10 x 2975 B : 111.5 Mbytes/s time was: 33.6 ms bw for 10 x 3000 B : 108.3 Mbytes/s time was: 34.9 ms bw for 10 x 3100 B : 114.7 Mbytes/s time was: 34.0 ms totaltime was: 388.4 ms These figures were run on a freshly compiled OpenMPI-1.3.2. The numbers for bruck at smalla package sizes vary a bit from run to run. /Peter > Pasha.
signature.asc
Description: This is a digitally signed message part.
