Jeff
I found the solution - rdma needs significant memory so the limits on
the shell have to be increased. I needed to add the lines
* soft memlock unlimited
* hard memlock unlimited
to the end of the file /etc/security/limits.conf. After that the openib
driver loads and everything is fine -
Charles,
What is the machine configuration you're running on? It seems that there
are two MCA parameter for the tcp btl: btl_tcp_if_include and
btl_tcp_if_exclude (see ompi_info for details). There may be other knobs
I'm not aware of. If you're using UCX then my guess is that UCX has its
own
3.1.2 was a long time ago, but I'm pretty sure that Open MPI v3.1.2 has
btl_tcp_if_include / btl_tcp_if_exclude.
Try running: "ompi_info --all --parsable | grep btl_tcp_if_"
I believe that those options will both take a CIDR notation of which network(s)
to use/not use. Note: the _if_include op
Yes. It is not unusual to have multiple network interfaces on each host of a
cluster. Usually there is a preference to use only one network interface on
each host due to higher speed or throughput, or other considerations. It would
be useful to be able to explicitly specify the interface to use
Charles, I recall using the I_MPI_NETMASK to choose which interface for MPI
to use.
I guess you are asking the same question for OpenMPI?
On Tue, 1 Sep 2020 at 17:03, Charles Doland via users <
users@lists.open-mpi.org> wrote:
> Is there a way to limit the IP addresses or network interfaces used
Is there a way to limit the IP addresses or network interfaces used for
communication by OpenMPI? I am looking for something similar to the
I_MPI_TCP_NETMASK or I_MPI_NETMASK environment variables for Intel MPI.
The OpenMPI documentation mentions the btl_tcp_if_include and
btl_tcp_if_exclude MC
