On Oct 11, 2024, at 6:21 PM, Navdeep Parhar <n...@freebsd.org> wrote:
I’m running a FreeBSD 14-STABLE host with a Chelstio T520. I have a bhyve VM (also running 14-STABLE) to which I have assigned a VF of the NIC. That is all working as expected; the host can pass traffic using the PF cxl0 and the guest can pass traffic using the VF cxlv0. However the host cannot communicate with the guest. I am looking in to the possibility of enabling 802.1qbg / VEPA / reflective relay on the switch port but I’d like to know if the T5 can do that switching itself without sending the packets over the wire. The marketing material says the card "integrates a high performance packet switch” but I don’t know how to configure that functionality on FreeBSD or if this use case is supported. Can anyone shed some light on that?
The PF driver's tx bypasses the internal switch by default and is not visible to the VFs because of that. Set this knob to force it go through the switch.
hw.cxgbe.tx_vm_wr Setting this to 1 instructs the driver to use VM work requests to transmit data. This lets PF interfaces transmit frames to VF interfaces over the internal switch in the ASIC. Note that the cxgbev(4) VF driver always uses VM work requests and is not affected by this tunable. The default value is 0 and should be changed only if PF and VF interfaces need to communicate with each other. Different interfaces can be assigned different values using the dev.<port>.X.tx_vm_wr sysctl when the interface is administratively down.
Thank you! That indeed works as described. I did scan the man page before posting but obviously could have benefited from closer study. I appreciate you pointing me in the right direction.
Out of curiosity is there any practical difference (performance; security, etc) between setting this knob and just using an additional VF for the host rather than the PF (aside from being one less VF available for use elsewhere)?
JN |
