On 09-Jul-19 2:30 PM, Burakov, Anatoly wrote:
On 09-Jul-19 1:11 PM, Jerin Jacob Kollanukkaran wrote:
-----Original Message-----
From: Burakov, Anatoly <anatoly.bura...@intel.com>
Sent: Tuesday, July 9, 2019 5:10 PM
To: Jerin Jacob Kollanukkaran <jer...@marvell.com>; David Marchand
<david.march...@redhat.com>
Cc: dev <dev@dpdk.org>; Thomas Monjalon <tho...@monjalon.net>; Ben
Walker <benjamin.wal...@intel.com>
Subject: Re: [EXT] Re: [dpdk-dev] [PATCH] bus/pci: fix IOVA as VA mode
selection
________________________________________
On Mon, Jul 8, 2019 at 4:25 PM <mailto:jer...@marvell.com> wrote:
From: Jerin Jacob <mailto:jer...@marvell.com>
Existing logic fails to select IOVA mode as VA if driver request to
enable IOVA as VA.
IOVA as VA has more strict requirement than other modes, so enabling
positive logic for IOVA as VA selection.
This patch also updates the default IOVA mode as PA for PCI devices
as it has to deal with DMA engines unlike the virtual devices that
may need only IOVA as DC.
We have three cases:
- driver/hw supports IOVA as PA only
[Jerin] It is not driver cap, it is more of system cap(IOMMU vs non
IOMMU). We are already addressing that case
I don't get how this works. How does "system capability" affect what
the device itself supports? Are we to assume that *all* hardware
support IOVA as VA by default? "System capability" is more of a bus
issue than an individual device issue, is it not?
What I meant is, supporting VA vs PA is function of IOMMU(not the
device
attribute).
Ie. Device makes the bus master request, if IOMMU available and
enabled in the SYSTEM , It goes over IOMMU and translate the IOVA to
physical address.
Another way to put is, Is there any _PCIe_ device which need/requires
RTE_PCI_DRV_NEED_IOVA_AS_PA in rte_pci_driver.drv_flags
Previously, as far as i can tell, the flag was used to indicate
support for IOVA
as VA mode, not *requirement* for IOVA as VA mode. For example, there
are multiple patches [1][2][3][4] (i'm sure i can find more!) that
added IOVA
as VA support to various drivers, and they all were worded it in this
exact way
- "support for IOVA as VA mode", not "require IOVA as VA mode". As
far as i
can tell, none of these drivers *require* IOVA as VA mode - they
merely use
this flag to indicate support for it.
Some class of devices NEED IOVA as VA for performance reasons.
Specially the devices has HW mempool allocators. On those devices If
we don’t use IOVA as VA,
Upon getting packet from device, It needs to go over
rte_mem_iova2virt() per
packet see driver/net/dppa2. Which has real performance issue.
I wouldn't classify this as "needing" IOVA. "Need" implies it cannot
work without it, whereas in this case it's more of a "highly
recommended" rather than "need".
Now suddenly it turns out that someone somewhere "knew" that "IOVA as
VA" flag in PCI drivers is supposed to indicate *requirement* and not
support, and it appears that this knowledge was not communicated nor
documented anywhere, and is now treated as common knowledge.
I think, the confusion here is, I was under impression that
# If device supports IOVA as VA and system runs with IOMMU then
the dpdk should run in IOVA as VA mode.
If above statement true then we don’t really need a new flag.
Exactly. And the flag used to indicate that the device *supports* IOVA
as VA, not that it *requires* it.
...unless the driver itself is written in such a way as to simply not
support VA to PA lookups - in that case, the above suggested way of
simply not indicating IOVA as PA support would fix the issue in that it
will require the device to either work in IOVA as VA mode, or fail to
initialize. Current semantics of only having one flag do not distinguish
between "can do both PA and VA" and "can only do VA" - hence the
suggestion of adding an additional flag indicating IOVA as PA support.
--
Thanks,
Anatoly
