Hi, Mike! I have impi v 4.1.2 (- impi) I build ompi 1.8.5 with MXM and hcoll (- ompi_yalla) I build ompi 1.8.5 without MXM and hcoll (- ompi_clear) I start osu p2p: osu_mbr_mr test with this MPIs. You can find the result of benchmark in attached file(mvs10p_mpi.xls: list osu_mbr_mr)
On 64 nodes (and 1024 mpi processes) ompi_yalla get 2 time worse perf than ompi_clear. Is mxm with yalla reduces performance in p2p compared with ompi_clear(and impi)? Am I doing something wrong? P.S. My colleague Alexander Semenov is in CC Best regards, Timur Четверг, 28 мая 2015, 20:02 +03:00 от Mike Dubman <mi...@dev.mellanox.co.il>: >it is not apples-to-apples comparison. > >yalla/mxm is point-to-point library, it is not collective library. >collective algorithm happens on top of yalla. > >Intel collective algorithm for a2a is better than OMPI built-in collective >algorithm. > >To see benefit of yalla - you should run p2p benchmarks (osu_lat/bw/bibw/mr) > > >On Thu, May 28, 2015 at 7:35 PM, Timur Ismagilov < tismagi...@mail.ru > wrote: >>I compare ompi-1.8.5 (hpcx-1.3.3-icc) with impi v 4.1.4. >> >>I build ompi with MXM but without HCOLL and without knem (I work on it). >>Configure options are: >> ./configure --prefix=my_prefix >>--with-mxm=path/to/hpcx/hpcx-v1.3.330-icc-OFED-1.5.4.1-redhat6.2-x86_64/mxm >>--with-platform=contrib/platform/mellanox/optimized >> >>As a result of the IMB-MPI1 Alltoall test, I have got disappointing results: >>for the most message sizes on 64 nodes and 16 processes per node impi is >>much (~40%) better. >> >>You can look at the results in the file "mvs10p_mpi.xlsx", I attach it. >>System configuration is also there. >> >>What do you think about? Is there any way to improve ompi yalla performance >>results? >> >>I attach the output of "IMB-MPI1 Alltoall" for yalla and impi. >> >>P.S. My colleague Alexander Semenov is in CC >> >>Best regards, >>Timur > > >-- > >Kind Regards, > >M.
mvs10p_mpi.xlsx
Description: MS-Excel 2007 spreadsheet
