If normal users can't write to /tmp (or if /tmp is an NFS-mounted filesystem), that's the underlying problem.
@Vinson -- you should probably try to get that fixed. On Oct 23, 2014, at 10:35 AM, Joshua Ladd <jladd.m...@gmail.com> wrote: > It's not coming from OSHMEM but from the OPAL "shmem" framework. You are > going to get terrible performance - possibly slowing to a crawl having all > processes open their backing files for mmap on NSF. I think that's the error > that he's getting. > > > Josh > > On Thu, Oct 23, 2014 at 6:06 AM, Vinson Leung <lwhvinson1...@gmail.com> wrote: > HI, Thanks for your reply:) > I really run an MPI program (compile with OpenMPI and run with "mpirun -n 8 > ......"). My OpenMPI version is 1.8.3 and my program is Gromacs. BTW, what is > OSHMEM ? > > Best > Vincent > > On Thu, Oct 23, 2014 at 12:21 PM, Ralph Castain <r...@open-mpi.org> wrote: > From your error message, I gather you are not running an MPI program, but > rather an OSHMEM one? Otherwise, I find the message strange as it only would > be emitted from an OSHMEM program. > > What version of OMPI are you trying to use? > >> On Oct 22, 2014, at 7:12 PM, Vinson Leung <lwhvinson1...@gmail.com> wrote: >> >> Thanks for your reply:) >> Follow your advice I tried to set the TMPDIR to /var/tmp and /dev/shm and >> even reset to /tmp (I get the system permission), the problem still occur >> (CPU utilization still lower than 20%). I have no idea why and ready to give >> up OpenMPI instead of using other MPI library. >> >> --------Old Message------------- >> >> Date: Tue, 21 Oct 2014 22:21:31 -0400 >> From: Brock Palen <bro...@umich.edu> >> To: Open MPI Users <us...@open-mpi.org> >> Subject: Re: [OMPI users] low CPU utilization with OpenMPI >> Message-ID: <cc54135d-0cfe-440a-8df2-06b587e17...@umich.edu> >> Content-Type: text/plain; charset=us-ascii >> >> Doing special files on NFS can be weird, try the other /tmp/ locations: >> >> /var/tmp/ >> /dev/shm (ram disk careful!) >> >> Brock Palen >> www.umich.edu/~brockp >> CAEN Advanced Computing >> XSEDE Campus Champion >> bro...@umich.edu >> (734)936-1985 >> >> >> >> > On Oct 21, 2014, at 10:18 PM, Vinson Leung <lwhvinson1...@gmail.com> wrote: >> > >> > Because of permission reason (OpenMPI can not write temporary file to the >> > default /tmp directory), I change the TMPDIR to my local directory (export >> > TMPDIR=/home/user/tmp ) and then the MPI program can run. But the CPU >> > utilization is very low under 20% (8 MPI rank running in Intel Xeon 8-core >> > CPU). >> > >> > And I also got some message when I run with OpenMPI: >> > [cn3:28072] 9 more processes have sent help message >> > help-opal-shmem-mmap.txt / mmap on nfs >> > [cn3:28072] Set MCA parameter "orte_base_help_aggregate" to 0 to see all >> > help / error messages >> > >> > Any idea? >> > Thanks >> > >> > VIncent >> > _______________________________________________ >> > users mailing list >> > us...@open-mpi.org >> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> > Link to this post: >> > http://www.open-mpi.org/community/lists/users/2014/10/25548.php >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/10/25555.php > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25556.php > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25558.php > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25560.php -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
