On Monday 17 September 2018 06:30 PM, Burakov, Anatoly wrote:
On 17-Sep-18 1:16 PM, Shahaf Shuler wrote:
Monday, September 17, 2018 1:07 PM, Burakov, Anatoly:
Subject: Re: [dpdk-dev] [PATCH 00/16] Support externally allocated
memory
in DPDK
On 13-Sep-18 8:44 AM, Shahaf Shuler wrote:
[...]
The responsibility to ensure memory is accessible before using it is
on the shoulders of the user - there is no checking done with regards
to validity of the memory (nor could there be...).
That makes sense. However who should be in-charge of mapping this
memory for dma access?
The user or internally be the PMD when encounter the first packet or
while
traversing the existing mempools?
Hi Shahaf,
There are two ways this can be solved. The first way is to perform VFIO
mapping automatically on adding/attaching memory. The second is to force
user to do it manually. For now, the latter is chosen because user
knows best
if they intend to do DMA on that memory, but i'm open to suggestions.
I agree with that approach, and will add not only if the mempool is
for dma or not but also which ports will use this mempool (this can
effect on the mapping).
That is perhaps too hardware-specific - this should probably be handled
inside the driver callbacks.
However I don't think this is generic enough to use only VFIO. As you
said, there are some devices not using VFIO for mapping rather some
proprietary driver utility.
IMO DPDK should introduce generic and device agnostic APIs to the user.
My suggestion is instead of doing vfio_dma_map that or vfio_dma_unmap
that have a generic dma_map(uint8_t port, address, len). Each driver
will register with its own mapping callback (can be vfio_dma_map).
It can be outside of this series, just wondering the people opinion on
such approach.
I don't disagree. I don't like bus/net/etc drivers doing their own thing
with regards to mapping, and i would by far prefer generic way to set up
DMA maps, to which VFIO will be a subscriber.
There is an issue with some devices and buses (i.e. bus/fslmc)
bypassing EAL
VFIO infrastructure and performing their own VFIO/DMA mapping magic, but
solving that problem is outside the scope of this patchset. Those
devices/buses should fix themselves :)
DMA mapping is a very common principle and can be easily be a candidate
for lets-make-generic-movement, but, being close to hardware (or
hardware specific), it does require the driver to have some flexibility
in terms of its eventual implementation.
I maintain one of those drivers (bus/fslmc) in DPDK which needs to have
special VFIO layer - and from that experience, I can say that VFIO
mapping does require some flexibility. SoC semantics are sometimes too
complex to pin to general-universally-agreed-standard concept. (or, one
can easily call it a 'bug', while it is a 'feature' for others :D)
In fact, NXP has another driver (bus/dpaa) which doesn't even work with
VFIO - loves to work directly with Phys_addr. And, it is not at a lower
priority than one with VFIO.
Thus, I really don't think a strongly controlled VFIO mapping should be
EAL's responsibility. Failure because of lack of mapping is a driver's
problem.
When not using VFIO, it's out of our hands anyway.
Why?
VFIO is not a must requirement for devices in DPDK.
When i say "out of our hands", what i mean to say is, currently as far
as EAL API is concerned, there is no DMA mapping outside of VFIO.
--
Thanks,
Anatoly
