I'm having problems running OpenMPI jobs
(using a hostfile) on an HPC cluster running
ROCKS on CentOS 6.3. I'm running OpenMPI
outside of Sun Grid Engine (i.e. it is not submitted
as a job to SGE). The program being run is a LAPACK
benchmark. The commandline parameter I'm
using to run the jobs is:
$MPI_DIR/bin/mpirun -np $NSLOTS -bind-to-core -report-bindings --hostfile
hostfile --mca btl_tcp_if_include eth0 --prefix $MPI_DIR
$BENCH_DIR/$APP_DIR/$APP_BIN
Where MPI_DIR=/hpc/apps/mpi/openmpi/1.8.2/
NSLOTS=128
I'm getting errors of the form and OpenMPI never runs the LAPACK benchmark:
--------------------------------------------------------------------------
WARNING: a request was made to bind a process. While the system
supports binding the process itself, at least one node does NOT
support binding memory to the process location.
Node: csclprd3-0-11
This usually is due to not having the required NUMA support installed
on the node. In some Linux distributions, the required support is
contained in the libnumactl and libnumactl-devel packages.
This is a warning only; your job will continue, though performance may be
degraded.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
A request was made to bind to that would result in binding more
processes than cpus on a resource:
Bind to: CORE
Node: csclprd3-0-11
#processes: 2
#cpus: 1
You can override this protection by adding the "overload-allowed"
option to your binding directive.
--------------------------------------------------------------------------
The only installed numa packages are:
numactl.x86_64 2.0.7-3.el6
@centos6.3-x86_64-0/$
When I search for the available NUMA packages I find:
yum search numa | less
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
============================== N/S Matched: numa
===============================
numactl-devel.i686 : Development package for building Applications that
use numa
numactl-devel.x86_64 : Development package for building Applications
that use
: numa
numad.x86_64 : NUMA user daemon
numactl.i686 : Library for tuning for Non Uniform Memory Access machines
numactl.x86_64 : Library for tuning for Non Uniform Memory Access
machines
Do I need to install additional and/or different NUMA packages in order to get
OpenMPI to work
on this cluster?
-Bill Lane
IMPORTANT WARNING: This message is intended for the use of the person or entity
to which it is addressed and may contain information that is privileged and
confidential, the disclosure of which is governed by applicable law. If the
reader of this message is not the intended recipient, or the employee or agent
responsible for delivering it to the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this information is
strictly prohibited. Thank you for your cooperation.
