Hi Doug, Attaching here a response from Ophir Maor (from Mellanox community)
> > Read your own guides ;-). > > I'm using this one for your switches: > https://community.mellanox.com/docs/DOC-1417 > > And these to try and get the linux machines configured properly: > https://community.mellanox.com/docs/DOC-1414 > https://community.mellanox.com/docs/DOC-1415 > https://community.mellanox.com/docs/DOC-2311 > https://community.mellanox.com/docs/DOC-2474 > http://www.mellanox.com/related-docs/prod_software/RoCE_with_Priority_Flow_Control_Application_Guide.pdf > > The guides are helpful if your setup allows you to follow their exact > example. But, they are shy on information about how to modify the > examples to your specific situation. For instance, I have to use vlan > priority 5 as my no-drop priority for RoCE traffic. I can't reliably > tell which portions of the guide I must switch the 3s to 5s in order to > get the new priority, and which uses of 3s in the guides relate to other > things that could be mapped to 5. On a separate note, it's unclear to > me if your switches and cards support more than one no-drop priority > (other vendor's RoCE cards I'm using here don't, they only allow one > no-drop priority for RoCE traffic and it must be 5). If it does support > more than one, I'd actually like both 3 and 5 to be no-drop and for one > vlan to use 3 and another to use 5. There are two flows to configure egress mapping - flow that pass via the kernel. Then you need to use kernel commands (e.g. vconfig set_egress_map, or other commands) to make the kernel set the egress priority. - flows that bypass the kernel such as RoCE, then you need to use tc_wrap to set the egress mapping. This post explains it very nicely for ConnectX-4. https://community.mellanox.com/docs/DOC-2474 Thanks, Ophir.
