Hi,
If I run "mpirun --mca btl tcp,self --mca allow_ib 0 -n 12 ./my_program", I get
to disable some "extra" info in the output file like:
The OpenFabrics (openib) BTL failed to initialize while trying to
allocate some locked memory. This typically can indicate that the
memlock limits are set too low. For most HPC installations, the
memlock limits should be set to "unlimited". The failure occured
here:
Local host: clus11
OMPI source: btl_openib.c:757
Function: opal_free_list_init()
Device: qib0
Memlock limit: 65536
You may need to consult with your system administrator to get this
problem fixed. This FAQ entry on the Open MPI web site may also be
helpful:
http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
--------------------------------------------------------------------------
[clus11][[33029,1],0][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],1][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],9][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],8][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],2][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],6][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],10][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],11][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],5][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],3][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],4][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
[clus11][[33029,1],7][btl_openib.c:1062:mca_btl_openib_add_procs] could not
prepare openib device for use
or like
By default, for Open MPI 4.0 and later, infiniband ports on a device
are not used by default. The intent is to use UCX for these devices.
You can override this policy by setting the btl_openib_allow_ib MCA parameter
to true.
Local host: clus11
Local adapter: qib0
Local port: 1
--------------------------------------------------------------------------
--------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.
Local host: clus11
Local device: qib0
--------------------------------------------------------------------------
so, now, I would like to force that parameters in file
$OMPI/etc/openmpi-mca-params.conf. I have run "ompi_info --param all all
--level 9" to get all parameters, but I don't know exactly what parameters I
need to add to $OMPI/etc/openmpi-mca-params.conf and what is the correcty
syntax of them to force always "--mca btl tcp,self --mca allow_ib 0". I have
already added "btl_openib_allow_ib = " and it works, but for parametes "--mca
btl tcp,self", what would be the correct syntax in
$OMPI/etc/openmpi-mca-params.conf file?
Thanks!!
