On Nov 14, 2005, at 8:21 PM, Allan Menezes wrote:
I think the confusion was my fault because --mca pml teg did not
produce errors and gave almost the same performance as Mpich2 v 1.02p1.
The reason why I cannot do what you suggest below is because the
.openmpi/mca-params.conf file if I am not mistaken would reside in my
home NFS share directory.
If I understand your setup, this should not be a problem. See the Open
MPI FAQ about setting MCA parameters -- this is just a different
mechanism to do it instead of passing them in on the mpirun command
line. Having this file available on all nodes means that all nodes
will get the same MCA parameters (which is typically what you want).
I have installed a new 5.01 beta version of
Oscar and /home/allan is a shared directory of my head node where the
openmpi installation resides.[/home/allan/openmpi with paths in the
.bash_profile and .bashrc files] I would have to do an individual 16
installations of open mpi on each node for /opt/openmpi and the
mca-params file residing in there.
You can install Open MPI on NFS or on each local disk. We haven't
migrated this information to the Open MPI FAQ yet, but the issues are
the same as are discussed on the LAM/MPI FAQ --
http://www.lam-mpi.org/faq/. See the "Typical setup of LAM" section,
and the question "Do I need a common filesystem on each node?".
Tell me if I am wrong. I might have
to do this as this is a heterogenous cluster with different brands of
ethernet cards and CPU's.
You might actually be ok -- Open MPI won't care what TCP cards you have
because we just use the OS TCP stack. Different CPUs *might* be a
problem, but it depends on how you compile Open MPI. Having different
Linux distos/versions can definitely be a problem because you may have
different versions of glibc across your nodes, etc. I'm guessing
that's homogeneous, though, since you're using OSCAR. So having an
NFS-install Open MPI might be ok.
Check out the LAM/MPI FAQ questions on heterogeneity, too -- the issues
are pretty much the same for Open MPI.
But it's a good test bed and I have no problems installing Oscar 4.2
on it.
See my later post Hpl and TCP today where I tried 0b1 without mca pml
teg and so on and get a good performance with 15 nodes and open mpi
rc6.
Thank you very much,
Regards,
Allan
--
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/
