On Tue, Feb 13, 2018 at 6:31 AM, Burakov, Anatoly <anatoly.bura...@intel.com > wrote:
> On 12-Feb-18 10:00 PM, Ravi Kerur wrote: > >> >> Let me just give you what has been tested and working/nonworking >> scenarios. Some of your questions might get answered as well. >> Test bed is very simple with 2 VF's created under IXGBE PF on >> host with one VF interface added to ovs-bridge on host and >> another VF interface given to guest. Test connectivity between >> VF's via ping. >> >> Host and guest -- Kernel 4.9 >> Host -- Qemu 2.11.50 (tried both released 2.11 and tip of the >> git (2.11.50)) >> DPDK -- 17.05.1 on host and guest >> Host and guest -- booted with GRUB intel_iommu=on (which enables >> IOMMU). Have tried with "iommu=on and intel_iommu=on" as well, >> but iommu=on is not needed when intel_iommu=on is set. >> >> Test-scenario-1: Host -- ixgbe_vf driver, Guest ixgbe_vf driver >> ping works >> Test-scenario-2: Host -- DPDK vfio-pci driver, Guest ixgbe_vf >> driver ping works >> Test-scenario-3: Host -- DPDK vfio-pci driver, Guest DPDK >> vfio-pci driver, DMAR errors seen on host, ping doesn't work >> >> >> OK, that makes it clearer, thanks. Does the third scenario work in >> other DPDK versions? >> >> >> No. Tried 16.11 same issue on guest and works fine on host. >> >> >> So now we've moved from "this worked on 16.11" to "this never worked". > > It would be good to see output of rte_dump_physmem_layout() on both host > and guest, and check which address triggers the DMAR error (i.e. if the > physical address is present in mappings for either DPDK process). > > -- > Earlier I was focusing only on DMAR errors and I might have said 'it worked' when I didn't notice them on host when dpdk was started on guest. When trying to send packets out of that interface from guest I did see DMAR errors. I am attaching information you requested. I have enabled log-level=8 and files contain dpdk EAL/PMD logs as well. Snippets below on host, DMAR fault address from dmesg [351576.998109] DMAR: DRHD: handling fault status reg 702 [351576.998113] DMAR: [DMA Read] Request device [04:10.0] fault addr 257617000 [fault reason 06] PTE Read access is not set on guest (dump phys_mem_layout) Segment 235: phys:0x257600000, len:2097152, virt:0x7fce87e00000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 ... PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fce87e0f4c0 sw_sc_ring=0x7fce87e07380 hw_ring=0x7fce87e17600 dma_addr=0x257617600 PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7fce89c67d40 sw_sc_ring=0x7fce89c5fc00 hw_ring=0x7fce89c6fe80 dma_addr=0x25406fe80 ... Thanks, Ravi > Thanks, > Anatoly >
