Hi--
On 12/7/20 2:13 PM, Parav Pandit wrote:
> Added documentation for devlink port and port function related commands.
>
> Signed-off-by: Parav Pandit <pa...@nvidia.com>
> Reviewed-by: Jiri Pirko <j...@nvidia.com>
> Reviewed-by: Jacob Keller <jacob.e.kel...@intel.com>
> ---
> Changelog:
> v4->v5:
> - described logically ingress and egress point of devlink port
> - removed networking from devlink port description
> - rephrased port type description
> - introdue PCI controller section and description
introduce
> - rephrased controller, device, function description
> - removed confusing eswitch to system wording
> - rephrased port function description
> - added example of mac address in port function attribute description
> ---
> .../networking/devlink/devlink-port.rst | 116 ++++++++++++++++++
> Documentation/networking/devlink/index.rst | 1 +
> 2 files changed, 117 insertions(+)
> create mode 100644 Documentation/networking/devlink/devlink-port.rst
>
> diff --git a/Documentation/networking/devlink/devlink-port.rst
> b/Documentation/networking/devlink/devlink-port.rst
> new file mode 100644
> index 000000000000..dce87d2c07ac
> --- /dev/null
> +++ b/Documentation/networking/devlink/devlink-port.rst
> @@ -0,0 +1,116 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +============
> +Devlink Port
> +============
> +
> +``devlink-port`` is a port that exists on the device. It has a logically
> +separate ingress/egress point of the device. A devlink port can be of one
port can be any one
of many flavours.
> +among many flavours. A devlink port flavour along with port attributes
> +describe what a port represents.
> +
> +A device driver that intends to publish a devlink port sets the
> +devlink port attributes and registers the devlink port.
> +
> +Devlink port flavours are described below.
> +
> +.. list-table:: List of devlink port flavours
> + :widths: 33 90
> +
> + * - Flavour
> + - Description
> + * - ``DEVLINK_PORT_FLAVOUR_PHYSICAL``
> + - Any kind of physical networking port. This can be an eswitch physical
> + port or any other physical port on the device.
> + * - ``DEVLINK_PORT_FLAVOUR_DSA``
> + - This indicates a DSA interconnect port.
> + * - ``DEVLINK_PORT_FLAVOUR_CPU``
> + - This indicates a CPU port applicable only to DSA.
> + * - ``DEVLINK_PORT_FLAVOUR_PCI_PF``
> + - This indicates an eswitch port representing a networking port of
> + PCI physical function (PF).
> + * - ``DEVLINK_PORT_FLAVOUR_PCI_VF``
> + - This indicates an eswitch port representing a networking port of
> + PCI virtual function (VF).
> + * - ``DEVLINK_PORT_FLAVOUR_VIRTUAL``
> + - This indicates a virtual port for the virtual PCI device such as PCI
> VF.
> +
> +Devlink port can have a different type based on the link layer described
> below.
> +
> +.. list-table:: List of devlink port types
> + :widths: 23 90
> +
> + * - Type
> + - Description
> + * - ``DEVLINK_PORT_TYPE_ETH``
> + - Driver should set this port type when a link layer of the port is
> + Ethernet.
> + * - ``DEVLINK_PORT_TYPE_IB``
> + - Driver should set this port type when a link layer of the port is
> + InfiniBand.
> + * - ``DEVLINK_PORT_TYPE_AUTO``
> + - This type is indicated by the user when driver should detect the port
> + type automatically.
> +
> +PCI controllers
> +---------------
> +In most cases PCI device has only one controller. A controller consists of
either PCI devices have
or a PCI device has
> +potentially multiple physical and virtual functions. Such PCI function
> consists
> +of one or more ports. This port of the function is represented by the devlink
> +eswitch port.
> +
> +A PCI Device connected to multiple CPUs or multiple PCI root complex or
complexes
> +SmartNIC, however, may have multiple controllers. For a device with multiple
> +controllers, each controller is distinguished by a unique controller number.
> +An eswitch on the PCI device may suppport ports of multiple controllers.
support
> +
> +An example view of two controller systems::
I think that this is
An example view of a two-controller system::
instead of 2 controller systems. ?
> +
> +In this example, external controller (identified by controller number = 1)
> +doesn't have eswitch. Local controller (identified by controller number = 0)
> +has the eswitch. Devlink instance on local controller has eswitch devlink
> +ports representing ports for both the controllers.
> +
> + ---------------------------------------------------------
> + | |
> + | --------- --------- ------- ------- |
> + ----------- | | vf(s) | | sf(s) | |vf(s)| |sf(s)| |
> + | server | | ------- ----/---- ---/----- ------- ---/--- ---/--- |
> + | pci rc |=== | pf0 |______/________/ | pf1 |___/_______/ |
> + | connect | | ------- ------- |
> + ----------- | | controller_num=1 (no eswitch) |
> + ------|--------------------------------------------------
> + (internal wire)
> + |
> + ---------------------------------------------------------
> + | devlink eswitch ports and reps |
> + | ----------------------------------------------------- |
> + | |ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 |ctrl-0 | |
> + | |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | |
> + | ----------------------------------------------------- |
> + | |ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 |ctrl-1 | |
> + | |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | |
> + | ----------------------------------------------------- |
> + | |
> + | |
> + ----------- | --------- --------- ------- ------- |
> + | smartNIC| | | vf(s) | | sf(s) | |vf(s)| |sf(s)| |
> + | pci rc |==| ------- ----/---- ---/----- ------- ---/--- ---/--- |
> + | connect | | | pf0 |______/________/ | pf1 |___/_______/ |
> + ----------- | ------- ------- |
> + | |
> + | local controller_num=0 (eswitch) |
> + ---------------------------------------------------------
> +
> +Port function configuration
> +===========================
> +
> +A user can configure the port function attribute before enumerating the
> +PCI function. Usually it means, user should configure port function attribute
> +before a bus specific device for the function is created. However, when
> +SRIOV is enabled, virtual function devices are created on the PCI bus.
> +Hence, function attribute should be configured before binding virtual
> +function device to the driver.
> +
> +User may set the hardware address of the function represented by the devlink
> +port function. For Ethernet port function this means a MAC address.
--
~Randy
