Please see inline. (Liran, can you please comment) > >> >> So, is this setting required if there are multiple IB interfaces (as >> when there are multiple eth interfaces)? What is curious is that >> there is only one interface visible from the container. Does the >> openib btl look deeper and find all that exist in the host? >> >> Pasha > Not really. We use Verbs driver to fetch the list of devices on the >> "node" >> >> Is there something about the openib implementations in 1.8 and >> 1.10 that may handle this differently since we do not set >> OMPI_MCA_btl_openib_if_include but our tests seem to work? Or, >> is it a fluke? >> >> Pasha > I was not involved that much in 1.8 and 1.10 so it is a bit hard to >> comment. >> I would suspect that this might be somehow related to the locality feature >> and openib btl selects and creates only one btl instance and ignores all the >> rest. > > So if I understand correctly, we do not need to worry for 1.8 and 1.10.
Probably. The fact that it works well actually might be a bug :) > > Since it is possible to see many hca_id entries, even in the container, what > do we need to do under 1.6.5? Can we use a single mlx4_# (e.g., mlx4_0) for > all or do we need to select one based on the ib# interface? We expect to > run multiple containers on a single host where each container gets a > unique/dedicated ib# interface. You may find this useful: http://lists.openfabrics.org/pipermail/ofvwg/attachments/20150512/245833e1/attachment-0001.pdf Best, Pasha
