On 10/10/2017 11:18 AM, Jasvinder Singh wrote: > The SoftNIC PMD is intended to provide SW fall-back options for specific > ethdev APIs in a generic way to the NICs not supporting those features. > > Currently, the only implemented ethdev API is Traffic Management (TM), > but other ethdev APIs such as rte_flow, traffic metering & policing, etc > can be easily implemented. > > Overview: > * Generic: The SoftNIC PMD works with any "hard" PMD that implements the > ethdev API. It does not change the "hard" PMD in any way. > * Creation: For any given "hard" ethdev port, the user can decide to > create an associated "soft" ethdev port to drive the "hard" port. The > "soft" port is a virtual device that can be created at app start-up > through EAL vdev arg or later through the virtual device API. > * Configuration: The app explicitly decides which features are to be > enabled on the "soft" port and which features are still to be used from > the "hard" port. The app continues to explicitly configure both the > "hard" and the "soft" ports after the creation of the "soft" port. > * RX/TX: The app reads packets from/writes packets to the "soft" port > instead of the "hard" port. The RX and TX queues of the "soft" port are > thread safe, as any ethdev. > * Execution: The "soft" port is a feature-rich NIC implemented by the CPU, > so the run function of the "soft" port has to be executed by the CPU in > order to get packets moving between "hard" port and the app. > * Meets the NFV vision: The app should be (almost) agnostic about the NIC > implementation (different vendors/models, HW-SW mix), the app should not > require changes to use different NICs, the app should use the same API > for all NICs. If a NIC does not implement a specific feature, the HW > should be augmented with SW to meet the functionality while still > preserving the same API. > > Traffic Management SW fall-back overview: > * Implements the ethdev traffic management API (rte_tm.h). > * Based on the existing librte_sched DPDK library. > > Example: Create "soft" port for "hard" port "0000:04:00.1", enable the TM > feature with default settings: > --vdev 'net_softnic0,hard_name=0000:04:00.1,soft_tm=on' > > Q1: Why generic name, if only TM is supported (for now)? > A1: The intention is to have SoftNIC PMD implement many other (all?) > ethdev APIs under a single "ideal" ethdev, hence the generic name. > The initial motivation is TM API, but the mechanism is generic and can > be used for many other ethdev APIs. Somebody looking to provide SW > fall-back for other ethdev API is likely to end up inventing the same, > hence it would be good to consolidate all under a single PMD and have > the user explicitly enable/disable the features it needs for each > "soft" device. > > Q2: Are there any performance requirements for SoftNIC? > A2: Yes, performance should be great/decent for every feature, otherwise > the SW fall-back is unusable, thus useless. > > Q3: Why not change the "hard" device (and keep a single device) instead of > creating a new "soft" device (and thus having two devices)? > A3: This is not possible with the current librte_ether ethdev > implementation. The ethdev->dev_ops are defined as constant structure, > so it cannot be changed per device (nor per PMD). The new ops also > need memory space to store their context data structures, which > requires updating the ethdev->data->dev_private of the existing > device; at best, maybe a resize of ethdev->data->dev_private could be > done, assuming that librte_ether will introduce a way to find out its > size, but this cannot be done while device is running. Other side > effects might exist, as the changes are very intrusive, plus it likely > needs more changes in librte_ether. > > Q4: Why not call the SW fall-back dev_ops directly in librte_ether for > devices which do not support the specific feature? If the device > supports the capability, let's call its dev_ops, otherwise call the > SW fall-back dev_ops. > A4: First, similar reasons to Q&A3. This fixes the need to change > ethdev->dev_ops of the device, but it does not do anything to fix the > other significant issue of where to store the context data structures > needed by the SW fall-back functions (which, in this approach, are > called implicitly by librte_ether). > Second, the SW fall-back options should not be restricted arbitrarily > by the librte_ether library, the decision should belong to the app. > For example, the TM SW fall-back should not be limited to only > librte_sched, which (like any SW fall-back) is limited to a specific > hierarchy and feature set, it cannot do any possible hierarchy. If > alternatives exist, the one to use should be picked by the app, not by > the ethdev layer. > > Q5: Why is the app required to continue to configure both the "hard" and > the "soft" devices even after the "soft" device has been created? Why > not hiding the "hard" device under the "soft" device and have the > "soft" device configure the "hard" device under the hood? > A5: This was the approach tried in the V2 of this patch set (overlay > "soft" device taking over the configuration of the underlay "hard" > device) and eventually dropped due to increased complexity of having > to keep the configuration of two distinct devices in sync with > librte_ether implementation that is not friendly towards such > approach. Basically, each ethdev API call for the overlay device > needs to configure the overlay device, invoke the same configuration > with possibly modified parameters for the underlay device, then resume > the configuration of overlay device, turning this into a device > emulation project. > V2 minuses: increased complexity (deal with two devices at same time); > need to implement every ethdev API, even those not needed for the scope > of SW fall-back; intrusive; sometimes have to silently take decisions > that should be left to the app. > V3 pluses: lower complexity (only one device); only need to implement > those APIs that are in scope of the SW fall-back; non-intrusive (deal > with "hard" device through ethdev API); app decisions taken by the app > in an explicit way. > > Q6: Why expose the SW fall-back in a PMD and not in a SW library? > A6: The SW fall-back for an ethdev API has to implement that specific > ethdev API, (hence expose an ethdev object through a PMD), as opposed > to providing a different API. This approach allows the app to use the > same API (NFV vision). For example, we already have a library for TM > SW fall-back (librte_sched) that can be called directly by the apps > that need to call it outside of ethdev context (use-cases exist), but > an app that works with TM-aware NICs through the ethdev TM API would > have to be changed significantly in order to work with different > TM-agnostic NICs through the librte_sched API. > > Q7: Why have all the SW fall-backs in a single PMD? Why not develop > the SW fall-back for each different ethdev API in a separate PMD, then > create a chain of "soft" devices for each "hard" device? Potentially, > this results in smaller size PMDs that are easier to maintain. > A7: Arguments for single ethdev/PMD and against chain of ethdevs/PMDs: > 1. All the existing PMDs for HW NICs implement a lot of features under > the same PMD, so there is no reason for single PMD approach to break > code modularity. See the V3 code, a lot of care has been taken for > code modularity. > 2. We should avoid the proliferation of SW PMDs. > 3. A single device should be handled by a single PMD. > 4. People are used with feature-rich PMDs, not with single-feature > PMDs, so we change of mindset? > 5. [Configuration nightmare] A chain of "soft" devices attached to > single "hard" device requires the app to be aware that the N "soft" > devices in the chain plus the "hard" device refer to the same HW > device, and which device should be invoked to configure which > feature. Also the length of the chain and functionality of each > link is different for each HW device. This breaks the requirement > of preserving the same API while working with different NICs (NFV). > This most likely results in a configuration nightmare, nobody is > going to seriously use this. > 6. [Feature inter-dependecy] Sometimes different features need to be > configured and executed together (e.g. share the same set of > resources, are inter-dependent, etc), so it is better and more > performant to do them in the same ethdev/PMD. > 7. [Code duplication] There is a lot of duplication in the > configuration code for the chain of ethdevs approach. The ethdev > dev_configure, rx_queue_setup, tx_queue_setup API functions have to > be implemented per device, and they become meaningless/inconsistent > with the chain approach. > 8. [Data structure duplication] The per device data structures have to > be duplicated and read repeatedly for each "soft" ethdev. The > ethdev device, dev_private, data, per RX/TX queue data structures > have to be replicated per "soft" device. They have to be re-read for > each stage, so the same cache misses are now multiplied with the > number of stages in the chain. > 9. [rte_ring proliferation] Thread safety requirements for ethdev > RX/TXqueues require an rte_ring to be used for every RX/TX queue > of each "soft" ethdev. This rte_ring proliferation unnecessarily > increases the memory footprint and lowers performance, especially > when each "soft" ethdev ends up on a different CPU core (ping-pong > of cache lines). > 10.[Meta-data proliferation] A chain of ethdevs is likely to result > in proliferation of meta-data that has to be passed between the > ethdevs (e.g. policing needs the output of flow classification), > which results in more cache line ping-pong between cores, hence > performance drops. > > Cristian Dumitrescu (4): > Jasvinder Singh (4): > net/softnic: add softnic PMD > net/softnic: add traffic management support > net/softnic: add TM capabilities ops > net/softnic: add TM hierarchy related ops > > Jasvinder Singh (1): > app/testpmd: add traffic management forwarding mode
Series applied to dpdk-next-net/master, thanks. (Was getting same build error from previous version, fixed while applying please confirm the pushed commit. Also waiting for testpmd document to squash this set later)
