[OMPI users] mpirun hangs: "hello" test in single machine

[Rodrigo Gómez Vázquez]

Hi,

I am having troubles with the program in a simulation server.

The system consists of several processors but all in the same node (more information of the specs. is in the attachments). The system is quite new (few months) and a user reported me that it was not possible to run simulations on multiple processors in parallel. We are using it for CFD-Simulations with OpenFOAM, which comes along with an own 1.5.3-version of OpenMPI (for more details you can look inside the "ThirdParty software folder" following this link: http://www.openfoam.org/archive/2.1.1/download/source.php). The OS is an Ubuntu 12.04 Server distro (see uname.out in the attachments).

He tried to start a simulation in parallel using the following command:

~: mpirun -np 4 <solver-with-its-corresponding-parameters>





:~$mpicc hello_c.c -o hello
:~$mpirun -np 4 hello
... and here it hangs again.



:~$mpirun -np 4 hostname
simserver
simserver
simserver
simserver
:~$

Works, as well as "ompi_info" does.





It would be wonderful if anyone could give me a hint.



Thanks in advance, guys

Rodrigo (Europe, GMT+2:00)

As a result the simulation does not start and there is no error message. It looks like the program is just waiting/looking for something. We can see shortly the 4 processes with their PIDs in the "top" processes list, but only for few tenths of second and with 0% use of CPU and 0.0% use of memory as well. In order to recover the command terminal we have to kill the process. The same happens with the "hello" scripts that come along with the OpenMPI's sources: I tried to execute other simpler processes, as recommended to check the installation. Let's see: Since we use the same OpenFOAM version without problems in several computers over ubuntu-based distros, I supposed that there must be any kind of incompatibility problem, due to the hardware, but... Anyway, I repeated the tests with the OpenMPI version from the ubuntu repositories (1.4.3) and got the same result. I am afraid, it may result a complicated issue, so please, let me know whatever relevant information missing.

openmpi1.4.3_ompi_info.out.bz2
Description: application/bzip

Linux simserver 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 
x86_64 x86_64 x86_64 GNU/Linux

cat_-proc-cpuinfo.out.bz2
Description: application/bzip