The main goal of the following commits is to be able to load a PMD
virtual device driver as a shared library (DPDK extension). Today it is
already possible to load an external PMD PCI driver (ex: memnic,
virtio-net-pmd, ...), but the DPDK framework does not allow to load a
virtual device driver in this way. For instance, adding the support of
host-side in memnic would require these patches.
How it worked before the patch
------------------------------
Example of what occurs when we started testpmd:
testpmd -c 0x15 -n 3 \
--proc-type=primary --huge-dir=/mnt/huge --use-device=eth_ring0 \
--use-device=eth_ring1 \
--use-device=02:00.0 \
-- -i --port-topology=chained
For each "--use-device" option, we call eal_dev_whitelist_add_entry():
main()
rte_eal_init()
eal_parse_args()
eal_dev_whitelist_add_entry()
Each device is appended in a global string. After parsing all the
"--use-device" argumentss, we have dev_list_str="eth_ring0,eth_ring1,".
Then, after all eal arguments are parsed, eal_dev_whitelist_parse() is
called:
main()
rte_eal_init()
eal_parse_args()
eal_dev_whitelist_parse()
is_valid_wl_entry()
It removes the extra ',' at the end of dev_list_str. Then, it splits
each device name and its argument (separated by ';'). Each device is
checked by is_valid_wl_entry(). It checks that the PCI identifier is
correct or that the name of the device starts with a known prefix
("eth_ring", "eth_pcap" or "eth_xenvirt", defined statically in eal code).
Then, rte_eal_pci_init() is called, it scans the PCI bus:
main()
rte_eal_init()
rte_eal_pci_init()
After that, rte_eal_non_pci_ethdev_init() tries all combination:
"eth_ring0", "eth_ring1", ..., "eth_ring31", "eth_pcap0", ...,
"eth_pcap31", ..., "-nodev-0", ..., "-nodev-31". For each, test if it
is in whitelist or not:
main()
rte_eal_init()
rte_eal_non_pci_ethdev_init()
eal_dev_is_whitelisted(name, ¶ms) # fills params
dev_types[i].init_fn(name, params) [ex: rte_pmd_ring_init()]
...
rte_eth_dev_allocate()
To check that, eal_dev_is_whitelisted() browse the list of devices in
the whitelist. Then, the init function of the device allocates the
ethernet device structure (attributing a port_id).
The main application then calls rte_pmd_init_all(). It initializes all
poll-mode drivers. Each driver calls rte_eth_driver_register(), like in
this example:
main()
rte_pmd_init_all()
rte_igb_pmd_init()
rte_eth_driver_register()
The probing of the PCI bus is done using rte_eal_pci_probe(), which
calls pci_probe_all_drivers(dev) for each PCI device:
main()
rte_eal_pci_probe()
pcidev_is_whitelisted(dev)
pci_probe_all_drivers(dev)
rte_eal_pci_probe_one_driver(dev, driver)
driver->devinit(driver, dev) [rte_eth_dev_init()]
rte_eth_dev_allocate()
eth_drv->eth_dev_init(eth_drv, eth_dev) [ex: eth_em_dev_init()]
List of problems
----------------
- pmd_ring, pmd_pcap and pmd_xenvirt are referenced in eal code:
- it's not possible to add a new virtual pmd dynamically
- eal compilation depends on these pmd... but the pmd depends on eal.
- it's referenced twice with duplicated values: eal_common_nonpci_devs.c
and eal_common_whitelist.c
- the parsing of virtual devices arguments is complex:
- they are all appended in a global string then split again
- there are 2 ways to do the same thing, like in the following example:
- --use-device="eth_ring0,eth_pcap0;iface=ixgbe0"
- --use-device="eth_ring0" --use-device="eth_pcap0;iface=ixgbe0"
- the same --use-device option is used for both pci whitelist and
virtual devices
- files and functions related to virtual devices are called 'non_pci',
'vdev' would be clearer (all non_pci devices aren't virtual devices).
- using ";" to separate a device and its argument is not a good idea in
a command line argument as it can be used in shell to separate
commands.
- it is not possible to use the blacklist mode (bind all devices) while
we add virtual devices.
Summary of the changes introduced by the patchset
-------------------------------------------------
- allow to register a virtual device driver from a dpdk extension
provided as a shared library
- embed all library symbols in dpdk library.
- remove references to rte_pmd_ring, rte_pmd_pcap and rte_pmd_xenvirt in
eal code
- add a new rte_devargs file in eal that unifies the code storing the
user arguments pci-blacklist, pci-whitelist, and virtual devices
arguments in one file
- rework eal user arguments
- "--use-device" becomes "--pci-whitelist" and "--vdev"
- replace ";" by "," when parsing device args
- support start-up arguments for PCI devices
Olivier Matz (11):
mk: use whole-archive option when creating dpdk binaries
devices-args: introduce rte_devargs in eal
devices-args: use rte_devargs and remove old whitelist code
devices-args: add a dump_devargs command in basic test application
pci: rename device_list as pci_device_list
vdev: rename eal_common_nonpci_devs.c as eal_common_vdev.c
vdev: allow external registration of virtual device drivers
device-args: use a comma instead of semicolon to separate key/values
device-args: replace use-device eal option by pci-whitelist and vdev
device-args: allow to provide per pci device command line arguments
testpmd: add several dump commands, useful for debug
app/test-pmd/cmdline.c | 114 +++++++++++++
app/test/Makefile | 1 +
app/test/commands.c | 9 +-
app/test/test.h | 1 +
app/test/test_devargs.c | 132 ++++++++++++++
app/test/test_eal_flags.c | 70 ++------
app/test/test_kvargs.c | 14 +-
app/test/test_pci.c | 55 +++---
app/test/test_pmd_ring.c | 6 +-
lib/librte_eal/common/Makefile | 2 +-
lib/librte_eal/common/eal_common_devargs.c | 153 +++++++++++++++++
lib/librte_eal/common/eal_common_nonpci_devs.c | 93 ----------
lib/librte_eal/common/eal_common_pci.c | 98 +++++------
lib/librte_eal/common/eal_common_vdev.c | 107 ++++++++++++
lib/librte_eal/common/eal_common_whitelist.c | 227 -------------------------
lib/librte_eal/common/include/eal_private.h | 42 +----
lib/librte_eal/common/include/rte_devargs.h | 140 +++++++++++++++
lib/librte_eal/common/include/rte_pci.h | 20 +--
lib/librte_eal/common/include/rte_vdev.h | 90 ++++++++++
lib/librte_eal/linuxapp/eal/Makefile | 4 +-
lib/librte_eal/linuxapp/eal/eal.c | 114 +++++++------
lib/librte_eal/linuxapp/eal/eal_ivshmem.c | 2 +-
lib/librte_eal/linuxapp/eal/eal_pci.c | 16 +-
lib/librte_kvargs/rte_kvargs.h | 6 +-
lib/librte_pmd_pcap/rte_eth_pcap.c | 16 +-
lib/librte_pmd_pcap/rte_eth_pcap.h | 8 -
lib/librte_pmd_ring/rte_eth_ring.c | 17 +-
lib/librte_pmd_ring/rte_eth_ring.h | 6 +-
lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 14 +-
lib/librte_pmd_xenvirt/rte_eth_xenvirt.h | 4 +-
mk/rte.app.mk | 5 +
31 files changed, 987 insertions(+), 599 deletions(-)
create mode 100644 app/test/test_devargs.c
create mode 100644 lib/librte_eal/common/eal_common_devargs.c
delete mode 100644 lib/librte_eal/common/eal_common_nonpci_devs.c
create mode 100644 lib/librte_eal/common/eal_common_vdev.c
delete mode 100644 lib/librte_eal/common/eal_common_whitelist.c
create mode 100644 lib/librte_eal/common/include/rte_devargs.h
create mode 100644 lib/librte_eal/common/include/rte_vdev.h
--
1.8.5.3
