Honestly, I don't know - I haven't looked into the source. OFED 1.5.2 has a version of ibv_rc_pingpong that's been modified to work with RoCE; you can pass the gid_index and SL as command-line arguments. I'm not sure how that's handled at the IB layer, but the source may be a good place to start.
-- Mike Shuey On Tue, Mar 1, 2011 at 9:14 AM, Jeff Squyres <jsquy...@cisco.com> wrote: > I thought you mentioned in a prior email that you had gotten one or two other > OFED sample applications to work properly. How are they setting the SL? Are > they not using the RDMA CM? > > > On Mar 1, 2011, at 7:35 AM, Michael Shuey wrote: > >> So, since RoCE has no SM, and setting an SL is required to get >> lossless ethernet on Cisco switches (and possibly others), does this >> mean that RoCE will never work correctly with OpenMPI on Cisco >> hardware? >> >> -- >> Mike Shuey >> >> >> >> On Tue, Mar 1, 2011 at 3:42 AM, Doron Shoham <dor...@mellanox.com> wrote: >>> Hi, >>> >>> Regarding to using a specific SL with RDMA CM, I've checked in the code and >>> it seems that RDMA_CM uses the SL from the SA. >>> So if you want to configure a specific SL, you need to do it via the SM. >>> >>> Doron >>> >>> -----Original Message----- >>> From: Jeff Squyres [mailto:jsquy...@cisco.com] >>> Sent: Thursday, February 24, 2011 3:45 PM >>> To: Michael Shuey >>> Cc: Open MPI Users , Mike Dubman >>> Subject: Re: [OMPI users] RoCE (IBoE) & OpenMPI >>> >>> On Feb 24, 2011, at 8:00 AM, Michael Shuey wrote: >>> >>>> Late yesterday I did have a chance to test the patch Jeff provided >>>> (against 1.4.3 - testing 1.5.x is on the docket for today). While it >>>> works, in that I can specify a gid_index, >>> >>> Great! I'll commit that to the trunk and start the process of moving it to >>> the v1.5.x series (I know you haven't tested it yet, but it's essentially >>> the same patch, just slightly adjusted for each of the 3 branches). >>> >>>> it doesn't do everything >>>> required - my traffic won't match a lossless CoS on the ethernet >>>> switch. Specifying a GID is only half of it; I really need to also >>>> specify a service level. >>> >>> RoCE requires the use of the RDMA CM (I think?), and I didn't think there >>> was a way to request a specific SL via the RDMA CM...? (I could certainly >>> be wrong here) >>> >>> I think Mellanox will need to follow up with these questions... >>> >>> -- >>> Jeff Squyres >>> jsquy...@cisco.com >>> For corporate legal information go to: >>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
