Howdy,
When running a Gromacs job using OpenMPI 1.4.1 on Infiniband enabled nodes, I'm
seeing the following process listing:
\_ -bash /opt/gridengine/default/spool/compute-0-3/job_scripts/97037
\_ mpirun -np 4 mdrun_mpi -v -np 4 -s production-Npt-323K_4CPU -o
production-Npt-323K_4CPU -c production-Npt-323K_4CPU -x
production-Npt-323K_4CPU -g production-Npt-323K_4CPU.log
\_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V
compute-0-4.local orted -mca ess env -mca orte_ess_jobid 945881088
-mca orte_ess_vpid 1 -mca orte_ess_num_procs 4 --hnp-uri
"945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
\_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V
compute-0-2.local orted -mca ess env -mca orte_ess_jobid 945881088
-mca orte_ess_vpid 2 -mca orte_ess_num_procs 4 --hnp-uri
"945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
\_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V
compute-0-1.local orted -mca ess env -mca orte_ess_jobid 945881088
-mca orte_ess_vpid 3 -mca orte_ess_num_procs 4 --hnp-uri
"945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
\_ mdrun_mpi -v -np 4 -s production-Npt-323K_4CPU -o
production-Npt-323K_4CPU -c production-Npt-323K_4CPU
-x production-Npt-323K_4CPU -g production-Npt-323K_4CPU.log
Is it normal for these tcp addresses to be listed if the job is using
Infiniband?
The 192.168.20.x subnet is the eth0 GigE network
And the 192.168.21.x subnet is the ib0 IPoverIB network
Or is this job actually using TCPIP over Infiniband / GigE?
I'm running mpirun without any special fabric includes / excludes.
ompi_info lists openib as a valid fabric:
$ ompi_info |grep openib
MCA btl: openib (MCA v2.0, API v2.0, Component v1.4.1)
Thanks for any insight,
Mike
=================================
Mike Hanby
mha...@uab.edu
Information Systems Specialist II
IT HPCS / Research Computing
