This probably shows my lack of understanding as to how OpenMPI negotiates the connectivity between nodes when given a choice of interfaces but anyway:
does dasher have any network interfaces that vixen does not? The scenario I am imgaining would be that you ssh into dasher from vixen using a "network" that both share and similarly, when you mpirun from vixen, the network that OpenMPI uses is constrained by the interfaces that can be seen from vixen, so you are fine. However when you are on dasher, mpirun sees another interface which it takes a liking to and so tries to use that, but that interface is not available to vixen so the OpenMPI processes spawned there terminate when they can't find that interface so as to talk back to dasher's controlling process. I know that you are no longer working with VMs but it's along those lines that I was thinking: extra network interfaces that you assume won't be used but which are and which could then be overcome by use of an explicit --mca btl_tcp_if_exclude virbr0 or some such construction (virbr0 used as an example here). Kevin -- Kevin M. Buckley Room: CO327 School of Engineering and Phone: +64 4 463 5971 Computer Science Victoria University of Wellington New Zealand
