Hi, A little bit of self-advertising: I recently pushed patches that will make DPDK default to RTE_IOVA_VA when physical addresses were not explicitly requested and are not available, e.g. when running as a non-privileged user. It shouldn't cause any conflicts with the changes you're proposing here, but any review is welcome.
pci/linux: use RTE_IOVA_VA whenever possible http://patches.dpdk.org/patch/44392/ [v2] eal/bus: use RTE_IOVA_PA only if phys addresses are available http://patches.dpdk.org/patch/44420/ As for the --iova-mode=<pa/va>, I agree it could help DPDK use cases where most drivers or devices are hotplugged/hotattached at runtime - e.g. SPDK, it could certainly make use of such param. D. > -----Original Message----- > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Eric Zhang > Sent: Tuesday, September 11, 2018 7:22 PM > To: Burakov, Anatoly <anatoly.bura...@intel.com>; Jerin Jacob > <jerin.ja...@caviumnetworks.com> > Cc: santosh <santosh.shu...@caviumnetworks.com>; > hemant.agra...@nxp.com; Gaëtan Rivet <gaetan.ri...@6wind.com>; > Richardson, Bruce <bruce.richard...@intel.com>; dev@dpdk.org; > allain.leg...@windriver.com; matt.pet...@windriver.com > Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical > > > > On 09/07/2018 04:13 PM, Eric Zhang wrote: > > > > > > On 09/07/2018 05:26 AM, Burakov, Anatoly wrote: > >> On 06-Sep-18 8:34 AM, Jerin Jacob wrote: > >>> -----Original Message----- > >>>> Date: Tue, 4 Sep 2018 23:40:36 -0400 > >>>> From: Eric Zhang <eric.zh...@windriver.com> > >>>> To: santosh <santosh.shu...@caviumnetworks.com>, > >>>> hemant.agra...@nxp.com, > >>>> Gaëtan Rivet <gaetan.ri...@6wind.com>, "Burakov, Anatoly" > >>>> <anatoly.bura...@intel.com> > >>>> CC: bruce.richard...@intel.com, dev@dpdk.org, > >>>> allain.leg...@windriver.com, > >>>> matt.pet...@windriver.com > >>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical > >>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 > >>>> Thunderbird/52.9.1 > >>>> > >>>> On 08/30/2018 08:59 AM, santosh wrote: > >>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote: > >>>>>> External Email > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote: > >>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote: > >>>>>>>>> This patch adds a configuration option to force the IOVA mode > >>>>>>>>> to physical address (PA). There exists virtual devices that > >>>>>>>>> are not directly attached to the PCI bus, and therefore the > >>>>>>>>> auto detection of the IOVA mode based on probing the PCI bus > >>>>>>>>> and IOMMU configuration may not report the required > addressing > >>>>>>>>> mode. Having the configuration option permits the mode to be > >>>>>>>>> explicitly configured in this scenario. > >>>>>>>>> > >>>>>>>>> Signed-off-by: eric zhang <eric.zh...@windriver.com> > >>>>>>>>> --- > >>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't > >>>>>>>> it be better to just add an EAL command-line option to force > >>>>>>>> IOVA mode to a particular value? > >>>>>> That is a good suggestion. > >>>>>>>> -- > >>>>>>>> Thanks, > >>>>>>>> Anatoly > >>>>>>> What is the bus of these devices and why not implement > >>>>>>> get_iommu_class in it? > >>>>>> There are cases, where you are using dpdk libraries with external > >>>>>> libraries and you need to change the default behavior DPDK lib to > >>>>>> use physical address instead of virtual address. > >>>>>> Providing an option to user will help. > >>>>>> > >>>>>> > >>>>> More appropriate solution could be: > >>>>> * Either fix it at bus layer .. i.e.. get_iommu_class().. > >>>>> * Or introduce something like [1] --iova-mode=<pa/va> param. > >>>>> > >>>>> Former is better solution than latter if autodetection is a key > >>>>> criteria. > >>>>> Thanks. > >>>>> > >>>>> [1] http://patchwork.dpdk.org/patch/25192/ > >>>>> > >>>> It's not generic which couldn't be fixed at bus layer. > >>>> So what's the preference of EAL option or compile time solution? > >>>> Adding --iova-mode as patch [1] will overrivde auto-detection > >>>> rte_bus_get_iommu_class() > >>>> make it no use; compile time solution will align with upstream and > >>>> keep new atuodetection solution in #ifndef. > >>> > >>> If it is for vdev devices, why not introduce something like > >>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality. > >>> And based on the devices(flags) on vdev bus, > >>> rte_bus_get_iommu_class() of vdev can decide the mode just like PCI > bus. > >>> > >> > >> That seems like a better option to me, +1. As far as i know, at the > >> moment if there are no devices attached at all, or if there are only > >> vdev devices attached, DPDK will default to IOVA as PA mode for no > >> good reason; such a change would certainly fix this. > > Thanks for the suggestions however our virtual device doesn't run dpdk > > vdev code so we can't use the flag. > > Notice that in eal.c there is one workaround that force iova to be PA > > per virtual device is not directly attached to pci. That case is > > checking kni module. Ours is a similar case that virtual device not > > attach pci directly. > > So we have to turn to force iova to PA either 1. compilation option 2. > > eal option. Which one should be the preference by taking into > > consideration that align with upstream? > > > > Thanks > > > Any comments? > >> > >>> > >>>> > >>>> Thanks > >>>> Eric > >>>> > >>> > >> > >> > >
