Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp for OpenMPI usage
On 11/05/2011 09:11 AM, Blosch, Edwin L wrote: .. I know where you're coming from, and I probably didn't title the post correctly because I wasn't sure what to ask. But I definitely saw it, and still see it, as an OpenMPI issue. Having /tmp mounted over NFS on a stateless cluster is not a broken configuration, broadly speaking. The vendors made those decisions and presumably that's how they do it for other customers as well. There are two other (Platform/HP) MPI applications that apparently work normally. But OpenMPI doesn't work normally. So it's deficient. I'm also concerned that there is a bit of an over-reaction to network filesystems. Stores to mmap'd files do not instantly turn into filesystem writes - there are dirty_writeback parameters to control how often writes occur and its typically 5-20 seconds. Ideally, memory or a local disk is used for session directories but, in many cases, you just wont notice a performance hit from network filesystems - we didn't when we tested session directories on Lustre. If your app is one of those handful that is slowed by OS jitter at megascale, then you may well notice. Obviously, its something to test. For our 1.5 install, I removed Lustre from the list of filesystem types that generate the warning message about network filesystems. It would be nice if it was a site choice whether or not to produce that message and when. David
[OMPI users] Problem with openmpi-default-hostfile
Hello together! I've built v1.4.3 (which was in OFED-1.5.3.2) and v1.4.4 (from you website). But in both versions I've the following problem: If I write some hosts into '/usr/mpi/gcc/openmpi-1.4.4/etc/openmpi-default-hostfile': cluster1 cluster2 cluster3 cluster4 and execute 'mpirun -np 4 " then only the local host is used. If I execute 'mpirun -hostfile /.../openmpi-default-hostfile -np 4 ' all works fine. So have I done something wrong or is there another possibility to set a system-wide list of default hosts (i.e. for all users)? Many thanks for any help! Regards, Lukas
Re: [OMPI users] Problem with openmpi-default-hostfile
Where did you install OMPI? If you check "which mpirun", does it point to the same installation where you edited the default hostfile? On Nov 6, 2011, at 6:16 PM, Lukas Razik wrote: > Hello together! > > I've built v1.4.3 (which was in OFED-1.5.3.2) and v1.4.4 (from you website). > But in both versions I've the following problem: > > If I write some hosts into > '/usr/mpi/gcc/openmpi-1.4.4/etc/openmpi-default-hostfile': > cluster1 > cluster2 > cluster3 > cluster4 > and execute 'mpirun -np 4 " then only the local host is used. > > If I execute 'mpirun -hostfile /.../openmpi-default-hostfile -np 4 ' all > works fine. > So have I done something wrong or is there another possibility to set a > system-wide list of default hosts (i.e. for all users)? > > Many thanks for any help! > > Regards, > Lukas > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
