On May 23, 2014, at 7:14 AM, Albert Solernou <albert.soler...@oerc.ox.ac.uk> wrote:
> Well, > the problem is that I don't know how to do any of these things Ah! You might want to read this: http://www.open-mpi.org/faq/?category=tuning#mca-params > , so more explicitly: > - does OpenMPI accept any environment variable that prevents binding like > MV2_ENABLE_AFFINITY does on MVAPICH2? OMPI_MCA_hwloc_base_binding_policy=none > - what is the default mca param file? Is it a runtime file? or a > configuration file? how do I modify it to achieve that? When you install OMPI, an "etc" directory gets created under the prefix location. In that directory is a file "openmpi-mca-params.conf". This is your default MCA param file that mpirun (and every OMPI process) reads on startup. You can put any params in there that you want. In this case, you'd add a line: hwloc_base_binding_policy = none HTH Ralph > > Thanks, > Albert > > On 23/05/14 15:02, Ralph Castain wrote: >> >> On May 23, 2014, at 6:58 AM, Albert Solernou <albert.soler...@oerc.ox.ac.uk> >> wrote: >> >>> Hi, >>> thanks a lot for your quick answers, and I see my error, it is "--bind-to >>> none" instead of "--bind-to-none". >>> >>> However, I need to be able to run "mpirun -np 2" without any binding >>> argument and get a "--bind-to none" behaviour. I don't know if I can export >>> an environment variable to do that, and I don't mind to re-compile with >>> some flag I missed or to alter the code. >>> >>> Any suggestion? >> >> Obviously, you could set an variable in your environment, so I'm assuming >> there is some other limitation in play here? If so, you could always put the >> MCA param in the default mca param file - we'll pick it up from there. >> >>> >>> Albert >>> >>> On 23/05/14 14:32, Ralph Castain wrote: >>>> Note that the lama mapper described in those slides may not work as it >>>> hasn't been maintained in a while. However, you can use the map-by and >>>> bind-to options to do the same things. >>>> >>>> If you want to disable binding, you can do so by adding "--bind-to none" >>>> to the cmd line, or via the MCA param "hwloc_base_binding_policy=none" >>>> >>>> If you want to bind your process to multiple cores (say one per thread), >>>> then you can use "--map-by core:pe=N". Many hybrid users prefer to bind to >>>> a socket, with one process for each socket - that can be done with >>>> "--map-by socket --bind-to socket". This keeps all the threads in the same >>>> NUMA domain. If you aren't sure that each socket is its own NUMA domain, >>>> you can alternatively "--map-by numa --bind-to numa" - that'll keep you in >>>> your own NUMA domain regardless of whether that's at the socket level or >>>> elsewhere. >>>> >>>> I'm working on adding a full description of the mapping/binding system to >>>> our web site. >>>> >>>> HTH >>>> Ralph >>>> >>>> On May 23, 2014, at 6:22 AM, Brock Palen <bro...@umich.edu> wrote: >>>> >>>>> Albert, >>>>> >>>>> Actually doing affinity correctly for hybrid got easier in OpenMPI 1.7 >>>>> and newer, In the past you had to make a lot of assumptions, stride by >>>>> node etc, >>>>> >>>>> Now you can define a layout: >>>>> >>>>> http://blogs.cisco.com/performance/eurompi13-cisco-slides-open-mpi-process-affinity-user-interface/ >>>>> >>>>> Brock Palen >>>>> www.umich.edu/~brockp >>>>> CAEN Advanced Computing >>>>> XSEDE Campus Champion >>>>> bro...@umich.edu >>>>> (734)936-1985 >>>>> >>>>> >>>>> >>>>> On May 23, 2014, at 9:19 AM, Albert Solernou >>>>> <albert.soler...@oerc.ox.ac.uk> wrote: >>>>> >>>>>> Hi, >>>>>> after compiling and installing OpenMPI 1.8.1, I find that OpenMPI is >>>>>> pinning processes onto cores. Although this may be >>>>>> desirable on some cases, it is a complete disaster when runnning hybrid >>>>>> OpenMP-MPI applications. Therefore, I want to disable this behaviour, >>>>>> but don't know how. >>>>>> >>>>>> I configured OpenMPI with: >>>>>> ./configure \ >>>>>> --prefix=$OPENMPI_HOME \ >>>>>> --with-psm \ >>>>>> --with-tm=/system/software/arcus/lib/PBS/11.3.0.121723 \ >>>>>> --enable-mpirun-prefix-by-default \ >>>>>> --enable-mpi-thread-multiple >>>>>> >>>>>> and: >>>>>> ompi_info | grep paffinity >>>>>> does not report anything. However, >>>>>> mpirun -np 2 --report-bindings ./wrk >>>>>> reports bindings: >>>>>> [login3:04574] MCW rank 1 bound to socket 0[core 1[hwt 0-1]]: >>>>>> [../BB/../../../../../..][../../../../../../../..] >>>>>> [login3:04574] MCW rank 0 bound to socket 0[core 0[hwt 0-1]]: >>>>>> [BB/../../../../../../..][../../../../../../../..] >>>>>> but they cannot be disabled as: >>>>>> mpirun -np 2 --bind-to-none ./wrk >>>>>> returns: >>>>>> mpirun: Error: unknown option "--bind-to-none" >>>>>> >>>>>> Any idea on what went wrong? >>>>>> >>>>>> Best, >>>>>> Albert >>>>>> >>>>>> -- >>>>>> --------------------------------- >>>>>> Dr. Albert Solernou >>>>>> Research Associate >>>>>> Oxford Supercomputing Centre, >>>>>> University of Oxford >>>>>> Tel: +44 (0)1865 610631 >>>>>> --------------------------------- >>>>>> _______________________________________________ >>>>>> users mailing list >>>>>> us...@open-mpi.org >>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>> >>>>> _______________________________________________ >>>>> users mailing list >>>>> us...@open-mpi.org >>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> >>> >>> -- >>> --------------------------------- >>> Dr. Albert Solernou >>> Research Associate >>> Oxford Supercomputing Centre, >>> University of Oxford >>> Tel: +44 (0)1865 610631 >>> --------------------------------- >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> > > -- > --------------------------------- > Dr. Albert Solernou > Research Associate > Oxford Supercomputing Centre, > University of Oxford > Tel: +44 (0)1865 610631 > --------------------------------- > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
