> you need to specify both the transport and self, such as: > mpirun -mca btl self,tcp
I found that the reason why I was no-longer able to run without openib was that I had some openib-specific tunables on the command line. I expected the params would get ignored, but instead it just sat there. The other funny thing I found was that on the cmd line I do not need to specify self; just openib, or tcp will do. But in the param file I must specify tcp,openib, or self,openib; self,tcp would not work. Oh well, I do not care for tcp anyway :-). But should I understand from all this that the "direct" mode will never actually work ? It seems that if you need at least two transports, then none of them can be the hardwired unique one, right ? Unless there's a built-in switch between a built-in self and the built-in other transport. > For Heroic latencies on IB we would need to use small message RDMA and > poll each peers dedicated memory region for completion. Well, I tried to play around with the eager_limit, min_rdma, etc. I did not see the latency of messages of a given size be lowered by changing the tresholds to make them rdma'd. Rather the opposite (which was my initial expectation, actually). May be I just misunderstood the whole set of tunables. My understanding was that messages under the eager limit would never be rdma'd by definition, and that the others would or would not be, depending on the min_rdma_size. -- Jean-Christophe Hugly <j...@pantasys.com> PANTA
