It's not coming from OSHMEM but from the OPAL "shmem" framework. You are going to get terrible performance - possibly slowing to a crawl having all processes open their backing files for mmap on NSF. I think that's the error that he's getting.
Josh On Thu, Oct 23, 2014 at 6:06 AM, Vinson Leung <lwhvinson1...@gmail.com> wrote: > HI, Thanks for your reply:) > I really run an MPI program (compile with OpenMPI and run with "mpirun -n > 8 ......"). My OpenMPI version is 1.8.3 and my program is Gromacs. BTW, > what is OSHMEM ? > > Best > Vincent > > On Thu, Oct 23, 2014 at 12:21 PM, Ralph Castain <r...@open-mpi.org> wrote: > >> From your error message, I gather you are not running an MPI program, but >> rather an OSHMEM one? Otherwise, I find the message strange as it only >> would be emitted from an OSHMEM program. >> >> What version of OMPI are you trying to use? >> >> On Oct 22, 2014, at 7:12 PM, Vinson Leung <lwhvinson1...@gmail.com> >> wrote: >> >> Thanks for your reply:) >> Follow your advice I tried to set the TMPDIR to /var/tmp and /dev/shm and >> even reset to /tmp (I get the system permission), the problem still occur >> (CPU utilization still lower than 20%). I have no idea why and ready to >> give up OpenMPI instead of using other MPI library. >> >> --------Old Message------------- >> >> Date: Tue, 21 Oct 2014 22:21:31 -0400 >> From: Brock Palen <bro...@umich.edu> >> To: Open MPI Users <us...@open-mpi.org> >> Subject: Re: [OMPI users] low CPU utilization with OpenMPI >> Message-ID: <cc54135d-0cfe-440a-8df2-06b587e17...@umich.edu> >> Content-Type: text/plain; charset=us-ascii >> >> Doing special files on NFS can be weird, try the other /tmp/ locations: >> >> /var/tmp/ >> /dev/shm (ram disk careful!) >> >> Brock Palen >> www.umich.edu/~brockp >> CAEN Advanced Computing >> XSEDE Campus Champion >> bro...@umich.edu >> (734)936-1985 >> >> >> >> > On Oct 21, 2014, at 10:18 PM, Vinson Leung <lwhvinson1...@gmail.com> >> wrote: >> > >> > Because of permission reason (OpenMPI can not write temporary file to >> the default /tmp directory), I change the TMPDIR to my local directory >> (export TMPDIR=/home/user/tmp ) and then the MPI program can run. But the >> CPU utilization is very low under 20% (8 MPI rank running in Intel Xeon >> 8-core CPU). >> > >> > And I also got some message when I run with OpenMPI: >> > [cn3:28072] 9 more processes have sent help message >> help-opal-shmem-mmap.txt / mmap on nfs >> > [cn3:28072] Set MCA parameter "orte_base_help_aggregate" to 0 to see >> all help / error messages >> > >> > Any idea? >> > Thanks >> > >> > VIncent >> > _______________________________________________ >> > users mailing list >> > us...@open-mpi.org >> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> > Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/10/25548.php >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/10/25555.php >> >> >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/10/25556.php >> > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/10/25558.php >
