Link to v8: [18]
:: Introduction ::
DPDK has been inherently a PCI inclined framework. Because of this, the
design of device tree (or list) within DPDK is also PCI inclined. A
non-PCI device doesn't have a way of being expressed without using hooks
started from EAL to PMD.
(Check 'Version Changes' section for changes)
:: Overview of the Proposed Changes ::
Assuming the below graph for a computing node:
device A1
|
+==.===='==============.============+ Bus A.
| `--> driver A11 \
device A2 `-> driver A12 \______
|CPU |
/`````
device B1 /
| /
+==.===='==============.============+ Bus B`
| `--> driver B11
device B2 `-> driver B12
- One or more buses are connected to a CPU (or core)
- One or more devices are conneted to a Bus
- Drivers are running instances which manage one or more devices
- Bus is responsible for identifying devices (and interrupt propogation)
- Driver is responsible for initializing the device
In [15], model assumes that rte_bus would be the base class using which
all the bus implementations would instantiate the objects. This patches
takes a much more basic approach, on the same lines of rte_device/
rte_driver and rte_pci_device/rte_pci_driver.
This is based on various review comments as well as offline (IRC)
discussions.
- rte_bus is an abstract class which includes basic methods supported by
all buses.
- specific implementation, for example for PCI rte_pci_bus, would extend
this class to form their own buses, with their own bus specific device
and driver list.
-
+-----------------+
|rte_pci_bus |
|+---------------+|
||rte_bus ||
|+---------------+|
+-----------------+
And example implementation would look like:
.--------------->+-------------------+
| |rte_pci_bus |
| | +----------------+|
| | |rte_bus <------.
| | | name || |
| | | scan || |
| | | probe || |
| | | attach || |
| | | detach || |
| | +----------------+| |
| | pci_driver_list | |
| .-------------->pci_device_list | |
| | | ... | |
| | +-------------------+ |
| | |
| +-------------------+ |
| |rte_pci_device | |
'----bus | |
| +----------------+| |
| |rte_device || |
| | bus --------------------------------'
| | ... ||
| +----------------+|
| ... |
+-------------------+
:: Brief about Patch Layout ::
0001~0002: Introducing the basic Bus model and associated test case
0003 : Split the PCI match into a separate function
0004 : Remove an unnecessary loop over drivers in device detach
0005~0006: Introduce bus->scan and bus->probe APIs
0007 : Integrate the bus with EAL, without removing existing device
driver init/scan model.
0008 : Add PCI Bus over Bus patches introduced earlier
0009~0010: Update the Bus and PCI test cases for scanning/probing
0011 : Enable PCI bus and remove code for direct PCI scan/probe from
EAL
0012 : Add device hotplugging over the bus and introduce PCI helpers
:: Pending Changes/Caveats ::
1. This patchset only moves the PCI into a bus. And, that movement is also
currently part of the EAL (lib/librte_eal/linux)
Eventual aim is the PCI bus reside in driver/bus/pci
2. Though the implementation for bus is common for Linux and BSD, the PCI
bus implementation has been done/tested only for Linux.
3. RTE_REGISTER_BUS has been declared with contructor priority of 101
It is important that Bus is registered *before* drivers are registered.
Only way I could find to assure that was via
__attribute(contructor(priority)) of GCC. I am not sure how it would
behave on other compilers. Any suggestions?
- One suggestion from David Marchand was to use global bus object
handles, which I have not implemented for now. If that is common
choice, I will change in v3.
4. Hotplugging has been introduced over Bus with a caveat - it would only
work for ports/NICs already scanned and available in the bus list.
:: ToDo list ::
- Bump to librte_eal version
- Documentation continues to have references to some _old_ PCI symbols
:: References ::
[1] http://dpdk.org/ml/archives/dev/2016-November/050186.html
[2] http://dpdk.org/ml/archives/dev/2016-November/050622.html
[3] http://dpdk.org/ml/archives/dev/2016-November/050416.html
[4] http://dpdk.org/ml/archives/dev/2016-November/050567.html
[5] http://dpdk.org/ml/archives/dev/2016-November/050628.html
[6] http://dpdk.org/ml/archives/dev/2016-November/050415.html
[7] http://dpdk.org/ml/archives/dev/2016-November/050443.html
[8] http://dpdk.org/ml/archives/dev/2016-November/050624.html
[9] http://dpdk.org/ml/archives/dev/2016-November/050296.html
[10] http://dpdk.org/ml/archives/dev/2016-December/051349.html - v1
[12] http://dpdk.org/ml/archives/dev/2016-December/052092.html - v2
[13] http://dpdk.org/ml/archives/dev/2016-December/052381.html - v3
[14] http://dpdk.org/ml/archives/dev/2016-December/053302.html - v4
[15] http://dpdk.org/ml/archives/dev/2016-December/053315.html - v5
[16] http://dpdk.org/ml/archives/dev/2017-January/055120.html - v6
[17] http://dpdk.org/ml/archives/dev/2017-January/055320.html - v7
[18] http://dpdk.org/ml/archives/dev/2017-January/055398.html - v8
:: Version Changes ::
v9:
- Removed comments over function definitions
- Documentation fixes as per review comments
- Split the test_pci and test_bus patches
- Split the patches for generic bus changes and PCI level bus changes
- Add bus_autotest as entry into autotest_data.py
v8:
- fix return value bug in rte_eal_pci_attach and rte_eal_pci_detach
v7:
- update to rte_pci_match for const parameters
- remove unnecessary log messages in probe; moved _after_ matching
of device and driver
- bug fixes in attach/detach methods
- PCI disable for debugging was missed (from rte_eal_pci_init) in v6
v6:
- Rearchitecture to bring bus object parallel to rte_device/driver
This majorly includes:
-- rte_pci_bus class and pci_bus as its object for PCI
-- bus->attach/detach (hotplugging)
-- removing bus->match as that is local to an implementation
- rename symbols rte_eal_bus_* to rte_bus_*
- restructuring patches (order) for simplicity
- update to test_pci
v5:
- Fix checkpatch error in Patch 0003
v4:
- rebase over master (eac901ce)
- Fix a bug in test_bus while setup and cleanup
- rename rte_eal_get_bus to rte_eal_bus_get
- Add helper (iterator) macros for easy bus,device,driver traversal
- removed 0001 patch as it is already merged in master
- fix missing rte_eal_bus_insert_device symbol in map file
v3:
- rebase over master (c431384c8f)
- revert patch 0001 changes for checkpatch (container_of macro)
- qat/rte_qat_cryptodev update for rte_driver->probe
- test_pci update for using a test_pci_bus for verification
- some bug fixes based on internal testing.
-- rte_eal_dev_attach not handling devargs
-- blacklisting not working
v2:
- No more bus->probe()
Now, rte_eal_bus_probe() calls rte_driver->probe based on match output
- new functions, rte_eal_pci_probe and rte_eal_pci_remove have been
added as glue code between PCI PMDs and PCI Bus
`-> PMDs are updated to use these new functions as callbacks for
rte_driver
- 'default' keyword has been removed from match and scan
- Fix for incorrect changes in mlx* and nicvf*
- Checkpatch fixes
- Some variable checks have been removed from internal functions;
functions which are externally visible continue to have such checks
- Some rearrangement of patches:
-- changes to drivers have been separated from EAL changes (but this
does make PCI PMDs non-working for a particular patch)
Shreyansh Jain (12):
eal/bus: introduce bus abstraction
test: add basic bus infrastructure tests
pci: split match and probe function
eal: remove loop over drivers in device detach
eal/bus: support for scanning of bus
eal/bus: introduce support for bus probing
eal: integrate bus scan and probe with EAL
eal/pci: add support for PCI bus
test: add test cases for scan and probe on BUS
test: add Bus based scan and probe test cases for PCI
eal: enable PCI bus
eal: enable hotplugging of devices on bus
app/test/Makefile | 2 +-
app/test/autotest_data.py | 6 +
app/test/test.h | 2 +
app/test/test_bus.c | 686 ++++++++++++++++++++++++
app/test/test_pci.c | 164 ++++--
lib/librte_eal/bsdapp/eal/Makefile | 1 +
lib/librte_eal/bsdapp/eal/eal.c | 13 +-
lib/librte_eal/bsdapp/eal/eal_pci.c | 17 +
lib/librte_eal/bsdapp/eal/rte_eal_version.map | 15 +-
lib/librte_eal/common/Makefile | 2 +-
lib/librte_eal/common/eal_common_bus.c | 138 +++++
lib/librte_eal/common/eal_common_dev.c | 56 +-
lib/librte_eal/common/eal_common_pci.c | 374 ++++++++-----
lib/librte_eal/common/eal_private.h | 10 -
lib/librte_eal/common/include/rte_bus.h | 191 +++++++
lib/librte_eal/common/include/rte_dev.h | 1 +
lib/librte_eal/common/include/rte_pci.h | 148 ++++-
lib/librte_eal/linuxapp/eal/Makefile | 1 +
lib/librte_eal/linuxapp/eal/eal.c | 13 +-
lib/librte_eal/linuxapp/eal/eal_pci.c | 57 +-
lib/librte_eal/linuxapp/eal/rte_eal_version.map | 15 +-
21 files changed, 1636 insertions(+), 276 deletions(-)
create mode 100644 app/test/test_bus.c
create mode 100644 lib/librte_eal/common/eal_common_bus.c
create mode 100644 lib/librte_eal/common/include/rte_bus.h
--
2.7.4
