"김건홍(KIM GEON HONG)"<geonhong....@hhi.co.kr> writes: > Hello, > > I tried to run a parallel computation (OpenFOAM) using Open MPI on a > HPC connected with infiniband. > When I ran a job using mpirun over a couple of nodes (20 cpus per > node), the computation was not accelerated as I expected.
The usual problem with OpenFOAM is building it against the bundled openmpi, not the system one, or even failing to build the MPI version at all, and not noticing because of the horrible build system. I've known plenty of people fail to get it right, including me. > For example, I ran the job over 40 cpus on 2 nodes, and I checked cpu > usages and processes via top command. > I expected 20 processes would be running on each node but I found that > only 19 processes were running and a cpu was in idle while others were > used. Was the example set up to use them all? Anyhow, this doesn't sound like an openmpi problem. > 2. Linux/kernel info: RHEL6.5 2.6.32-431.el6.x86_64 > - Linux distro/version : Red hat Enterprise Linux Server release 6.5 > (Santiago) [For what it's worth, there's a packaged version of OpenFOAM, built against current EPEL6 system libraries, along with the source, under <https://copr.fedoraproject.org/coprs/loveshack/livhpc/>.]
