Could you please tell us what version of OpenMPI you are using, and the cmd line you used to execute the job?
Thanks Ralph > On Nov 23, 2015, at 11:05 PM, 김건홍(KIM GEON HONG) <geonhong....@hhi.co.kr> > wrote: > > Hello, > > I tried to run a parallel computation (OpenFOAM) using Open MPI on a HPC > connected with infiniband. > When I ran a job using mpirun over a couple of nodes (20 cpus per node), the > computation was not accelerated as I expected. > > For example, I ran the job over 40 cpus on 2 nodes, and I checked cpu usages > and processes via top command. > I expected 20 processes would be running on each node but I found that only > 19 processes were running and a cpu was in idle while others were used. > Following is a capture of top result. > > As you can see, Cpu1 is in idle and there are only 19 simpleFoam processes! > > <image002.png> > > I have no idea why this is happened. > > Sometimes, a cpu is in idle while 20 processes are running but, in that case > two processes running with 50% of cpu usage. > That is, those two different processes are assigned to the same cpu. > > Please refer to the attached file for required information of the cluster and > its environment. > The output of “ulimit –l“ command on both nodes is “unlimited”. > > Additional information for OpenFabrics-based network is as following: > 1. OpenFabrics version : MLNX_OFED_LINUX-2.4.1.0.0 > 2. Linux/kernel info: RHEL6.5 2.6.32-431.el6.x86_64 > - Linux distro/version : Red hat Enterprise Linux Server release 6.5 > (Santiago) > - Kernel version : 2.6.32-431.el6.x86_64 > 3. Subnet manager : infiniband B class > > Thank you. > > Best regards, > Geon-Hong Kim. > > ----------------------------------- > <image001.png> > Geon-Hong Kim > Engineer, Ph.D. > > Performance Evaluation Research Department > Hyundai Maritime Research Institute > Hyundai Heavy Industries Co., Ltd. > > Office +82-52-203-8053 > Fax +82-52-250-9675 > Mobile +82-10-3084-1357 > ----------------------------------- > > <system_info.tar.bz2>_______________________________________________ > users mailing list > us...@open-mpi.org <mailto:us...@open-mpi.org> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > <http://www.open-mpi.org/mailman/listinfo.cgi/users> > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/11/28100.php > <http://www.open-mpi.org/community/lists/users/2015/11/28100.php>
