On Thu, Mar 09, 2017 at 09:15:14AM +0000, Bruce Richardson wrote:
On Wed, Mar 08, 2017 at 11:54:02AM -0500, Neil Horman wrote:
On Wed, Mar 08, 2017 at 04:15:33PM +0100, Gaetan Rivet wrote:
> This PMD intercepts and manages Ethernet device removal events issued by
> slave PMDs and re-initializes them transparently when brought back so that
> existing applications do not need to be modified to benefit from true
> hot-plugging support.
>
> The stacked PMD approach shares many similarities with the bonding PMD but
> with a different purpose. While bonding provides the ability to group
> several links into a single logical device for enhanced throughput and
> supports fail-over at link level, this one manages the sudden disappearance
> of the underlying device; it guarantees applications face a valid device in
> working order at all times.
>
Why not just add this feature to the bonding pmd then? A bond is perfectly
capable of handling the trivial case of a single underlying device, and adding
an option to make the underly slave 'persistent' seem both much simpler in terms
of implementation and code size, than adding an entire new pmd, along with its
supporting code.
Neil
@Neil
I don't know if you saw my answer to Bruce on the matter [1], it
partially adresses your point.
+1
I don't like the idea of having multiple PMDs in DPDK to handle
combining multiple other devices into one.
/Bruce
I understand the concern. Let's first put aside for the moment the link
grouping, which is only part of the fail-safe PMD function.
The fail-safe PMD at its core, provides an alternative paradigm, a new
proposal for a hot-plug functionality in a lightweight form-factor from a
user standpoint.
The central question that I would like to tackle is this: why should we
require from our users declaring a bonding device to have hot-plug support?
I took some time to illustrate a few modes of operation:
Fig. 1
.-------------.
| application |
`------.------'
|
.----'-----.---------. <------ init, conf, Rx/Tx
| | |
| .---|--.------|--. <--- conf, link check, Rx/Tx
| | | | | |
v | v v v v
.---------. | .-------. .------.
| bonding | | | ixgbe | | mlx4 |
`----.----' | `-------' `------'
| |
`------'
Typical link fail-over.
Fig. 2
.-------------.
| application |
`------.------'
| <-------- init, conf, Rx/Tx
v
.-----------.
| fail-safe |
`-----.-----'
|
.---'----. <--- init, conf, dev check, Rx/Tx
| |
v v
.-------. .------.
| ixgbe | | mlx4 |
`-------' `------'
Typical automatic hot-plug handling with device fail-over.
Fig. 3
.-------------.
| application |
`------.------'
|
.----'-----.-------------. <---------- init, conf, Rx/Tx
| | |
| .---|--.----------|--. <------- conf, link check, Rx/Tx
| | | | | |
v | v v v v
.---------. | .-----------. .-----------.
| bonding | | | fail-safe | | fail-safe |
`----.----' | `-----.-----' `-----.-----'
| | | | <------ init, conf, dev check, Rx/Tx
`------' v v
.-------. .------.
| ixgbe | | mlx4 |
`-------' `------'
Combination to provide link fail-over with automatic hot-plug handling.
Fig. 4
.-------------.
| application |
`------.------'
|
.----'-----.-------------. <---------- init, conf, Rx/Tx
| | |
| .---|--.----------|--. <------- conf, link check, Rx/Tx
| | | | | |
v | v v v v
.---------. | .-----------. .-----------.
| bonding | | | fail-safe | | fail-safe |
`----.----' | `-----.-----' `-----.-----'
| | | | <------- init, conf, dev check, Rx/Tx
`------' | |
.------'---. .---'------.
| | | |
v v v v
.--------. .--------. .--------. .--------.
| mlx4 1 | | mlx4 2 | | mlx4 1 | | mlx4 2 |
| port 1 | | port 1 | | port 2 | | port 2 |
`--------' `--------' `--------' `--------'
Complex use case with link fail-over at port level and automatic hot-plug
handling with device fail-over.
1. LSC vs. RMV
A link status change is a valid state for a device. It calls for
specific responses, e.g. a link switch in a bonding device, without
losing the general configuration of the port.
The removal of a device calls for more than pausing operations and
switching an active device. The party responsible for initializing the
device should take care of closing it properly. If this party also
wants to be able to restore the device if it was plugged back in, it
would need be able to initialize it back and reconfigure its previous
state.
As we can see that in [Fig. 1], this responsibility lies upon the
application.
2. Bonding and link availability
The hot-plug functionality is not a core function of the bonding PMD.
It is only interested in knowing if the link is active or not.
Adding the device persistence to the bonding PMD would mean adding the
ability to flexibly parse device definitions to cope with plug-ins in
evolving busses (PCI hot-plug could mean changing bus addresses), being
able to emulate the EAL and the ether layer and to properly store the
device configuration. This means formally describing the life of a
device in a DPDK application from start to finish.
All of this hot-plug support, in order for the bonding to be aware of
the status of some of its links. This seems like scope-creep.
3. Fail-safe and hot-plug support
An attach / detach (pseudo-hotplug) API exists in DPDK. The main
problem of this API is that it does not offer reacting to a device
plug-out, only triggering a device detaching from an application. This
is a completely different approach from an application standpoint.
The fail-safe PMD offers an out-of-the-box implementation of a newly
proposed hot-plug API [2]. It allows a seamless integration to users
for device removal support in their applications, without requiring
evolutions [Fig. 2]. This flexibility allows it to be used as part of
a bond [Fig. 3], [Fig. 4], while current bonding does not allow for
detaching devices, even if it were to be considered hot-plug. There is
no reason however to require declaring a bonding device to be able to
use it, from a user perspective this seems backward.
Both bonding and fail-safe PMDs deal with enslaving devices and
acting upon their state changing. The bonding function performs at
link level, while the hot-plug function deals with the device itself.
I do not think this similarity justify merging both functions.
Maintenance would be easier with clear, simpler functions implemented
in separate PMDs.
[1]: http://dpdk.org/ml/archives/dev/2017-March/059446.html
[2]: http://dpdk.org/ml/archives/dev/2017-March/059217.html
--
Gaëtan Rivet
6WIND
