Hi, Yes I did, in fact the master node is an NFS server and the other 9 nodes auto-mount it on startup. The NFS partition on the master is also a RAMdisk. The initial method was to create the OpenFOAM case in the NFS partition and then just point the nodes at it using wdir on the mpirun command line. It worked, but wow that lit up the network switch! So I changed the method so that it actually copied the case process folders to each node prior to running mpirun. It improved things, but still enormous amounts of IP traffic.
From: John Hearns [mailto:hear...@googlemail.com] Sent: 24 January 2016 09:28 To: Open MPI Users <us...@open-mpi.org> Subject: Re: [OMPI users] Raspberry Pi 2 Beowulf Cluster for OpenFOAM Hi Steve. Regarding Step 3, have you thought of using some shared storage? NFS shared drive perhaps, or there are many alternatives! On 23 January 2016 at 20:47, Steve O'Hara <soh...@pivotal-solutions.co.uk<mailto:soh...@pivotal-solutions.co.uk>> wrote: Hi, I’m afraid I’m pretty new to both OpenFOAM and openMPI so please excuse me if my questions are either stupid or badly framed. I’ve created a 10 Raspberry pi beowulf cluster for testing out MPI concepts and see how they are harnessed in OpenFOAM. After a helluva lot of hassle, I’ve got the thing running using OpneMPI to run a solver in parallel. The problem I have is that if I switch the server node to not use the cluster (still use 3 cores in an MPI job) the job finishes in x minutes. If I tell it to use the 9 other members of the cluster, the same job takes x times 3! This is what I’m doing; 1. Create a mesh, adjust it with some other OF stuff 2. Run the process to split the job into processes for each node 3. Copy the process directories to each of the affected nodes using scp 4. Run mpirun with a hosts file 5. Re-constitute the case directory by copying back the processor folders 6. Re-construct the case Only step 4 Uses MPI and the other steps have a reasonably linear response time. Step 4 is characterised by a flurry of network activity, followed by all the Pis lighting up with CPU activity followed a long time of no CPU activity but huge network action. It’s this last bit that is consuming all the time – is this a tear-down phase of MPI? Each of the Pi nodes is set up as slots=4 max_slots=4 What is all the network activity? It seems to happen after the solver has completed its job so I’m guessing it has to be MPI. The network interface on the Pi is not a stellar performer so is there anything I can do to minimise the network traffic? Thanks, Steve _______________________________________________ users mailing list us...@open-mpi.org<mailto:us...@open-mpi.org> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2016/01/28349.php
