On Mon, 29 Apr 2019 15:47:49 +0000 Michael Robbert <mrobb...@mines.edu> wrote:
> I was curious what startup method other sites are using with Intel > MPI? We (nsc.liu.se nat. level academic HPC) have used IntelMPI+slurm as a primary platform for several generations. Our setup to this day is "mpiexec.hydra -bootstrap slurm". Pros: * it works well * it binds well including with slurm cgroups * it performs well For those of you not familiar with the approach it basically goes like this: site wrapper -> mpirun -> mpiexec.hydra -> srun pmi_proxy That is, the intel pmi_proxy ends up under the slurmstepd and then launches and pinns the ranks/threads. When I say it performs well I include experiences up to configs such as 3200 ranks with 16 threads each and 64000 ranks pure MPI. Disclaimer: IntelMPI-2019 is "problematic" and not yet successfully used here... /Peter K
