On Jul 23, 2008, at 5:35 PM, Bill Broadley wrote:
My understanding is that -mca btl foo should fail since there isn't
a transport layer called foo.
It should, but it's getting trumped. See below.
So OFED-1.3.1 (or an openmpi build from source) ./install.pl works
with TCP, but not infinipath (because of a missing psm library).
All the "-mca btl" functionality works as expected.
OFED-1.3.1 (or an openmpi build from source) when I add "--with-psm"
works with infinipath, but all -mca parameters are ignored. Is
there a way to get openmpi working with infinipath without the psm
library? Or a suggestion on how to get the -mca functionality
working?
Ah -- here's what's happening: Open MPI has 2 different MPI point-to-
point engines:
1. "ob1", which uses the BTL components
2. "cm", which uses the MTL components
The two different engines reflect two different models for network
APIs. For the most part, users don't care what the difference is.
Usually, when an MTL is available, it will give better performance
than the corresponding BTL (lots of caveats here -- there's always
cases where the BTL will give better performance than the MTL, but in
general, the MTL will perform better).
So what's happening in your case is that "cm" sees that it can be
activated with the "psm" MTL (i.e., the native infinipath API), and
does so. Therefore, anything you specify about BTLs is ignored,
because cm doesn't see/know/use BTLs at all.
So you can force the use of specific BTLs by forcing the use of ob1,
like so:
mpirun --mca pml ob1 --mca btl tcp,self ...
If you don't specify the pml, the auto-selection will take over and
default to cm because you have the psm MTL available and able to be
used at run-time.
Make sense?
--
Jeff Squyres
Cisco Systems
