Hi Anatoly,
On Monday 04 September 2017 08:38 PM, Burakov, Anatoly wrote: >> From: Santosh Shukla [mailto:santosh.shu...@caviumnetworks.com] >> Sent: Thursday, August 31, 2017 4:26 AM >> To: dev@dpdk.org >> Cc: tho...@monjalon.net; jerin.ja...@caviumnetworks.com; >> hemant.agra...@nxp.com; olivier.m...@6wind.com; >> maxime.coque...@redhat.com; Gonzalez Monroy, Sergio >> <sergio.gonzalez.mon...@intel.com>; Richardson, Bruce >> <bruce.richard...@intel.com>; shreyansh.j...@nxp.com; >> gaetan.ri...@6wind.com; Burakov, Anatoly <anatoly.bura...@intel.com>; >> step...@networkplumber.org; acon...@redhat.com; Santosh Shukla >> <santosh.shu...@caviumnetworks.com> >> Subject: [PATCH v7 3/9] linuxapp/eal_pci: get iommu class >> >> Get iommu class of PCI device on the bus and returns preferred iova >> mapping mode for that bus. >> >> Patch also introduces RTE_PCI_DRV_IOVA_AS_VA drv flag. >> Flag used when driver needs to operate in iova=va mode. >> >> Algorithm for iova scheme selection for PCI bus: >> 0. If no device bound then return with RTE_IOVA_DC mapping mode, else >> goto 1). >> 1. Look for device attached to vfio kdrv and has .drv_flag set to >> RTE_PCI_DRV_IOVA_AS_VA. >> 2. Look for any device attached to UIO class of driver. >> 3. Check for vfio-noiommu mode enabled. >> >> If 2) & 3) is false and 1) is true then select mapping scheme as RTE_IOVA_VA. >> Otherwise use default mapping scheme (RTE_IOVA_PA). >> >> Signed-off-by: Santosh Shukla <santosh.shu...@caviumnetworks.com> >> Signed-off-by: Jerin Jacob <jerin.ja...@caviumnetworks.com> >> Reviewed-by: Maxime Coquelin <maxime.coque...@redhat.com> >> Acked-by: Hemant Agrawal <hemant.agra...@nxp.com> >> --- >> v6 --> v7: >> - squashed v6 series patch no [01/12] & [05/12].. >> i.e.. moved RTE_PCI_DRV_IOVA_AS_VA flag into this patch. (Aaron >> comment). >> >> lib/librte_eal/common/include/rte_pci.h | 2 + >> lib/librte_eal/linuxapp/eal/eal_pci.c | 95 >> +++++++++++++++++++++++++ >> lib/librte_eal/linuxapp/eal/eal_vfio.c | 19 +++++ >> lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ >> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 + >> 5 files changed, 121 insertions(+) >> >> diff --git a/lib/librte_eal/common/include/rte_pci.h >> b/lib/librte_eal/common/include/rte_pci.h >> index 0e36de093..a67d77f22 100644 >> --- a/lib/librte_eal/common/include/rte_pci.h >> +++ b/lib/librte_eal/common/include/rte_pci.h >> @@ -202,6 +202,8 @@ struct rte_pci_bus { #define >> RTE_PCI_DRV_INTR_RMV 0x0010 >> /** Device driver needs to keep mapped resources if unsupported dev >> detected */ #define RTE_PCI_DRV_KEEP_MAPPED_RES 0x0020 >> +/** Device driver supports iova as va */ #define >> RTE_PCI_DRV_IOVA_AS_VA >> +0X0040 >> >> /** >> * A structure describing a PCI mapping. >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c >> b/lib/librte_eal/linuxapp/eal/eal_pci.c >> index 8951ce742..9725fd493 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c >> @@ -45,6 +45,7 @@ >> #include "eal_filesystem.h" >> #include "eal_private.h" >> #include "eal_pci_init.h" >> +#include "eal_vfio.h" >> >> /** >> * @file >> @@ -487,6 +488,100 @@ rte_pci_scan(void) >> return -1; >> } >> >> +/* >> + * Is pci device bound to any kdrv >> + */ >> +static inline int >> +pci_device_is_bound(void) >> +{ >> + struct rte_pci_device *dev = NULL; >> + int ret = 0; >> + >> + FOREACH_DEVICE_ON_PCIBUS(dev) { >> + if (dev->kdrv == RTE_KDRV_UNKNOWN || >> + dev->kdrv == RTE_KDRV_NONE) { >> + continue; >> + } else { >> + ret = 1; >> + break; >> + } >> + } >> + return ret; >> +} >> + >> +/* >> + * Any one of the device bound to uio >> + */ >> +static inline int >> +pci_device_bound_uio(void) >> +{ >> + struct rte_pci_device *dev = NULL; >> + >> + FOREACH_DEVICE_ON_PCIBUS(dev) { >> + if (dev->kdrv == RTE_KDRV_IGB_UIO || >> + dev->kdrv == RTE_KDRV_UIO_GENERIC) { >> + return 1; >> + } >> + } >> + return 0; >> +} >> + >> +/* >> + * Any one of the device has iova as va */ static inline int >> +pci_device_has_iova_va(void) >> +{ >> + struct rte_pci_device *dev = NULL; >> + struct rte_pci_driver *drv = NULL; >> + >> + FOREACH_DRIVER_ON_PCIBUS(drv) { >> + if (drv && drv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) { >> + FOREACH_DEVICE_ON_PCIBUS(dev) { >> + if (dev->kdrv == RTE_KDRV_VFIO && >> + rte_pci_match(drv, dev)) >> + return 1; >> + } >> + } >> + } >> + return 0; >> +} >> + >> +/* >> + * Get iommu class of PCI devices on the bus. >> + */ >> +enum rte_iova_mode >> +rte_pci_get_iommu_class(void) >> +{ >> + bool is_bound; >> + bool is_vfio_noiommu_enabled = true; >> + bool has_iova_va; >> + bool is_bound_uio; >> + >> + is_bound = pci_device_is_bound(); >> + if (!is_bound) >> + return RTE_IOVA_DC; >> + >> + has_iova_va = pci_device_has_iova_va(); >> + is_bound_uio = pci_device_bound_uio(); #ifdef VFIO_PRESENT >> + is_vfio_noiommu_enabled = vfio_noiommu_is_enabled() == 1 ? 1 : >> 0; > If you specify is_vfio_noiommu_enabled as bool, you should probably treat it > as such, and assign true/false. queued for v8. > Other than that, I'm curious why is it always set to "true" by default? If we > don't have VFIO compiled, it seems like the error message would always > complain about vfio-noiommu mode being enabled, which is confusing. Set to 'true' for case when VFIO_PRESENT unset.. meaning platform doesn't support VFIO (linux versioned < 3.6) i.e.. using UIO - In that case, flag makes sure _pa policy selected. On error message: It won't come in non-vfio case, as 'has_iova_va' will set to 0. Error message will show for those case where few device out of many bind to uio, so message will pop-up and iova policy would be _pa in that case. Thanks. > Thanks, > Anatoly
