On Mon, Nov 27, 2017 at 4:30 AM, Zhang, Qi Z <qi.z.zh...@intel.com> wrote:
> Hi William: > > > -----Original Message----- > > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Chas Williams > > Sent: Saturday, November 25, 2017 7:57 AM > > To: dev@dpdk.org > > Cc: skh...@vmware.com; Chas Williams <3ch...@gmail.com>; Chas > > Williams <ch...@att.com> > > Subject: [dpdk-dev] [PATCH] vfio: try physical address if virtual > address fails > > > > Some machines appear to have buggy DMAR mappings. A typical mapping > > error looks like: > > > > DMAR: intel_iommu_map: iommu width (39) is not sufficient for the > > mapped address (7fc4fa800000) > > DMAR: intel_iommu_map: iommu width (39) is not sufficient for the > > mapped address (7fc4fa800000) > > DMAR: intel_iommu_map: iommu width (39) is not sufficient for the > > mapped address (7fc4fa800000) > > DMAR: intel_iommu_map: iommu width (39) is not sufficient for the > > mapped address (7fc4fa800000) > > > I met the same issue on some intel atom platform, the root cause is IOMMU > only support 39 bit virtual address. > Not sure retry with physical address will be the right fix. I saw > rte_eal_iova_mode is called at other place, it still take the virtual > address as the mapped result, does that break something? > So far the workaround may works by using --virtbase-addr to assign a > address in range explicitly (for example 0x70,0000,0000) > Regards > Qi > I don't think the IOVA usage elsewhere is an issue since I limited my changes to reworking what was done in commit e85a919286d2 which appears to be what broken things for me. It's not clear that passing --base-virtaddr is a workable solution. First, it's only a hint to mmap(). The kernel really can do whatever it wants. And then, what values do I pick? If one value fails, do I just restart and keep trying new values until it succeeds? I just want to fall back to the previous behavior. I know the physical mapping will succeed since the DMAR tables are limited to 36 bits in width. And yes, I should update the commit to blame the IOMMU width. > > > To work around this, attempt to do a physical address mapping if the > virtual > > address mapping fails. > > > > Fixes: e85a919286d2 ("vfio: honor IOVA mode before mapping") > > > > Signed-off-by: Chas Williams <ch...@att.com> > > --- > > lib/librte_eal/linuxapp/eal/eal_vfio.c | 8 ++++++++ > > 1 file changed, 8 insertions(+) > > > > diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c > > b/lib/librte_eal/linuxapp/eal/eal_vfio.c > > index 58f0123..6250676 100644 > > --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c > > +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c > > @@ -35,6 +35,7 @@ > > #include <fcntl.h> > > #include <unistd.h> > > #include <sys/ioctl.h> > > +#include <stdbool.h> > > > > #include <rte_log.h> > > #include <rte_memory.h> > > @@ -702,6 +703,7 @@ vfio_type1_dma_map(int vfio_container_fd) > > /* map all DPDK segments for DMA. use 1:1 PA to IOVA mapping */ > > for (i = 0; i < RTE_MAX_MEMSEG; i++) { > > struct vfio_iommu_type1_dma_map dma_map; > > + int retried = false; > > > > if (ms[i].addr == NULL) > > break; > > @@ -716,9 +718,15 @@ vfio_type1_dma_map(int vfio_container_fd) > > dma_map.iova = ms[i].iova; > > dma_map.flags = VFIO_DMA_MAP_FLAG_READ | > > VFIO_DMA_MAP_FLAG_WRITE; > > > > +retry: > > ret = ioctl(vfio_container_fd, VFIO_IOMMU_MAP_DMA, > > &dma_map); > > > > if (ret) { > > + if (!retried && rte_eal_iova_mode() == > RTE_IOVA_VA) { > > + dma_map.iova = ms[i].iova; > > > > + retried = true; > > + goto retry; > > + } > > RTE_LOG(ERR, EAL, " cannot set up DMA remapping, " > > "error %i (%s)\n", errno, > > strerror(errno)); > > -- > > 2.9.5 > Regards > Qi >
