Mike Dubman <mi...@dev.mellanox.co.il> writes: > these flags available in master and v1.10 branches and make sure that ranks > to core allocation is done starting from cpu socket closer to the HCA.
I'm confused by the 1.8.8 below, then. I haven't tried 1.10 since it breaks binary compatibility and seemed to have core binding issues. I expected openmpi to bind next to the HCA anyhow, since hwloc finds it, and wonder why that would be different with MXM. Incidentally, the different results with 1.6 seem to be because it ignores MXM for low core counts. Maybe that's the thing to do. > Of course you can have same effect with taskset. > > On Mon, Oct 5, 2015 at 8:46 PM, Dave Love <d.l...@liverpool.ac.uk> wrote: > >> Mike Dubman <mi...@dev.mellanox.co.il> writes: >> >> > what is your command line and setup? (ofed version, distro) >> > >> > This is what was just measured w/ fdr on haswell with v1.8.8 and mxm and >> UD >> > >> > + mpirun -np 2 -bind-to core -display-map -mca rmaps_base_mapping_policy >> > dist:span -x MXM_RDMA_PORTS=mlx5_3:1 -mca rmaps_dist_device mlx5_3:1 -x >> > MXM_TLS=self,shm,ud osu_latency
