Hi Stephen,
Thanks. Ok, so we should implement the VF control protocol on the netsvc
device. That said, it looks to me that we could use the AF_PACKET for this, no
need for an optimized, poll-mode netsvc driver in VPP.
To implement this control protocol, I suppose studying the DPDK implementation
should be enough.
The usecase would be eg. to deploy VNF in Azure with VPP without having to pull
DPDK as a dependency and improve performance quite noticeably.
What do you think?
Best
ben
> -----Original Message-----
> From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Stephen
> Hemminger
> Sent: jeudi 11 avril 2019 22:32
> To: Stephen Hemminger via Lists.Fd.Io
> <stephen=networkplumber....@lists.fd.io>
> Cc: step...@networkplumber.org; Benoit Ganne (bganne) <bga...@cisco.com>;
> vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Mellanox dependency changes
>
> On Thu, 11 Apr 2019 10:24:45 -0700
> "Stephen Hemminger via Lists.Fd.Io"
> <stephen=networkplumber....@lists.fd.io> wrote:
>
> > On Thu, 11 Apr 2019 09:30:12 +0000
> > "Benoit Ganne (bganne)" <bga...@cisco.com> wrote:
> >
> > > Hi Stephen,
> > >
> > > > The rdma-core stuff is likely to be a problem on Azure. The
> > > > Mellanox device is always hidden as a secondary device behind a
> > > > synthetic virtual device based on VMBus. There are two DPDK
> > > > different ways this is used. One is with vdev_netvsc/failsafe/tap
> and the other is with netvsc PMD.
> > > > In either case, the virtual device expects to see the MLX device
> > > > show up as a VF if enabled.
> > > > So unless you want to write native VPP drivers for VMBus as well,
> > > > I don't see how having native rdma-core driver will help. It will
> only get
> > > > confused.
> > >
> > > I'd be interested to better understand how it works. My current
> understanding:
> > > - the Mellanox device shows up as a PCI SR-IOV VF in the Linux VM
> > > ("the VF")
> > > - the VF is associated with a VMBUS (not PCI) netvsc virtual device
> > > - the netvsc and the VF shares the same MAC. The netvsc is guaranteed
> to be always there whereas the VF can disappear between migrations etc.
> > > - control plane traffic (multicast eg. ARP, other?) always go
> > > through the netvsc
> > > - if the VF is present is gets all the traffic minus control plane.
> > > If it is not present, all traffic gets rerouted through netvsc as a
> > > fallback
> > >
> > > Am I correct? Could you elaborate a little bit more about what is the
> control plane traffic (basically what we should miss on the VF)?
> > > If so, I see at least 2 problems we need to tackle:
> > > - we need to receive control plane traffic from netvsc because of
> > > eg. ARP. However, being control-plane/fallback, it could be slow and
> > > use eg. TAP or even AF_PACKET. We will also need to use it to
> > > populate the neighbor entries of the VF
> > > - the VF can appear/disappear w/o notice in case of migration etc.
> That could be handled in different ways, eg. by the control plane agent
> (removing the rdma iface from VPP prior to migration, adding it after post
> migration) or by leveraging bonding or...
> > >
> > > Thanks for your help,
> > > Ben
> >
> > Very close.
> > - In Azure the first packet of every flow arrives on the netvsc (slow
> path).
> > Packets arrive to FPGA, anything with non-matching flow goes to
> Azure host flow control (VFP) which evaluates
> > it against rules. On success, it programs FPGA and forwards packet
> over netvsc path.
> > - The presence of VF is reported as an event on VMBus
> > - VF is not enabled until netvsc sends message to host, acts as ack
> that VF is ok.
> > - VF removal is reported to netvsc, and it switches datapath back
> > - the VF should not be manipulated directly, it is hidden in DPDK;
> > in Linux/FreeBSD it is marked as slave
>
> Additionally, Windows and recent versions of Linux associate netvsc with
> VF from the VMBus serial number.
> This is simpler and more reliable than the MAC address.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#12776): https://lists.fd.io/g/vpp-dev/message/12776
Mute This Topic: https://lists.fd.io/mt/30795618/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
