Re: [OMPI users] MPI with RoCE

2022-09-06 Thread Jeff Squyres (jsquyres) via users
You can think of RoCE as "IB over IP" -- RoCE is essentially the IB protocol over IP packets (which is different than IPoIB, which is emulating IP and TCP over the InfiniBand protocol). You'll need to consult the docs for your Mellanox cards, but if you have Ethernet cards, you'll want to set t

Re: [OMPI users] MPI with RoCE

2022-09-06 Thread Harutyun Umrshatyan via users
Guys, I actually could make it work! I had to change Mellanox configuration from Ethernet to Infiniband and set up IPoIB. That was in fact a good experience, but the issue is that not all my Mellanoxes can be configured to Infiniband. My final destination is to make it work without Mellanox OFED o