On 4/4/2019 10:57 AM, Burakov, Anatoly wrote: > On 03-Apr-19 5:29 PM, Ferruh Yigit wrote: >> On 4/1/2019 10:51 AM, Kiran Kumar Kokkilagadda wrote: >>> From: Kiran Kumar K <kirankum...@marvell.com> >>> >>> With current KNI implementation kernel module will work only in >>> IOVA=PA mode. This patch will add support for kernel module to work >>> with IOVA=VA mode. >> >> Thanks Kiran for removing the limitation, I have a few questions, can you >> please >> help me understand. >> >> And when this patch is ready, the restriction in 'linux/eal/eal.c', in >> 'rte_eal_init' should be removed, perhaps with this patch. I assume you >> already >> doing it to be able to test this patch. >> >>> >>> The idea is to maintain a mapping in KNI module between user pages and >>> kernel pages and in fast path perform a lookup in this table and get >>> the kernel virtual address for corresponding user virtual address. >>> >>> In IOVA=VA mode, the memory allocated to the pool is physically >>> and virtually contiguous. We will take advantage of this and create a >>> mapping in the kernel.In kernel we need mapping for queues >>> (tx_q, rx_q,... slow path) and mbuf memory (fast path). >> >> Is it? >> As far as I know mempool can have multiple chunks and they can be both >> virtually >> and physically separated. >> >> And even for a single chunk, that will be virtually continuous, but will it >> be >> physically continuous? > > Just to clarify. > > Within DPDK, we do not make a distinction between physical address and > IOVA address - we never need the actual physical address, we just need > the DMA addresses, which can either match the physical address, or be > completely arbitrary (in our case, they will match VA addresses, but it > doesn't have to be the case - in fact, 17.11 will, under some > circumstances, populate IOVA addresses simply starting from address 0). > > However, one has to remember that IOVA address *is not a physical > address*. The pages backing a VA chunk may be *IOVA*-contiguous, but may > not necessarily be *physically* contiguous. Under normal circumstances > we really don't care, because the VFIO/IOMMU takes care of the mapping > between IOVA and PA transparently for the hardware. > > So, in IOVA as VA mode, the memory allocated to the mempool will be > (within a given chunk) both VA and IOVA contiguous - but not necessarily > *physically* contiguous! In fact, if you try calling rte_mem_virt2phy() > on the mempool pages, you'll likely find that they aren't (i've seen > cases where pages were mapped /backwards/!). > > Therefore, unless by "physically contiguous" you mean "IOVA-contiguous", > you *cannot* rely on memory in mempool being *physically* contiguous > merely based on the fact that it's IOVA-contiguous.
Thanks for clarification.
