On Wed, Nov 25, 2015 at 11:47 PM, Younghwan Go <yhwan at ndsl.kaist.edu> wrote: > Hello, > > Thank you all for helping us understand on issues with no hugepage option. > > As Konstantin mentioned at the end, I tried using VFIO module instead of IGB > UIO module. I enabled all necessary parameters (IOMMU, virtualization, > vfio-pci, VFIO permission) and ran my code with no hugepage option. > > At first, it seemed to receive packets fine, but after a while, it stopped > receiving packets. I could temporarily remove this issue by not calling > rte_eth_tx_burst(). Also, when I looked at the received packets, they all > contained 0s instead of actual data.
In my experience, if there's all zeroes it's because the NIC didn't actually write any packet data. In my case, to prove that, I initialized the mbuf data to a well-known pattern, and saw that after packet reception, the mbuf data was still that pattern, not the data I actually sent in the packet. From my analysis, I concluded that the ixgbe NIC was failing to actually write the data to memory because the corresponding PCI transaction was failing. The PCI transaction failed because something was going wrong with how IOMMU was mapping my VM's guest physical address to the host's physical address, and how that interacted with the PCI tree. I don't know if that's what you're seeing. However, it may be that the physical address being given to the NIC for the mbuf is invalid, or cannot be properly mapped to a real physical address if you're using IOMMU. When this happens, can you check the pcie AER registers on the NIC, if they exist? Maybe they'll show that something happened. Thanks, Kyle > Was there anything that I missed in > running with VFIO? I'm curious if no hugepage with no hugepage option was > confirmed to run with VFIO. > > Thank you, > Younghwan > > 2015-11-25 ?? 11:12? Ananyev, Konstantin ?(?) ? ?: >
