On 7/4/2023 7:39 PM, Thomas Monjalon wrote:
04/07/2023 11:23, Gupta, Nipun:
On 7/4/2023 1:36 PM, Ding, Xuan wrote:
From: Gupta, Nipun <nipun.gu...@amd.com>
From: Ding, Xuan <xuan.d...@intel.com>
From: Ding, Xuan
From: Nipun Gupta <nipun.gu...@amd.com>
Hi Xuan,
Thanks for pointing out the issue and figuring out the patch which
introduced this. If you have answers to below queries, please let me know:
Is there any other test cases which tests "--no-huge" which pass?
Yes, there are test cases adding "--no-huge" option to validate 4k
page size in async vhost.
Actually, the page size is decided by front-end, so I think this
case can be removed.
Previously, testpmd can start with "--no-huge" options (not sure if
there are test cases).
Cmd: ./build/app/dpdk-testpmd -l 5-6 -n 4 --no-huge -m 1024 -- -i
Also, if we change the "-m" option to provide lower memory, does
the test pass?
"-m" option is also added and does not work.
When you mention too many pages exceed the capability of IOMMU,
you are referring to HW capability to create multiple pages? Here
it seems in case of 4K page size we need 256K pages which is limiting the
capacity?
Yes, this is the result of my initial debugging.
The direct impact is that this kind of testpmd cases cannot start now.
If this is expected, I think we can close this defect and ignore the "--no-
huge"
option when start.
Any insights? Should we just ignore the "--no-huge" option and close this
defect?
Now we did this as a workaround. Seems no one uses the "--no-huge"
option in testpmd now.
VFIO supports dma_entry_limit as a module parameter, which has a default
value of U16_MAX i.e. 64K, most likely which is limiting creation of 256K
entries for 4K pages here. This can be modified while inserting vfio module:
modprobe vfio_iommu_type1 dma_entry_limit=1000000
Thanks for your suggestion. I tried it on ubuntu 22.04 but it does not work.
The reason I think is vfio-pci is build-in in kernel driver (since 20.04) and
it does not support dynamic insmod/rmmod.
Does this command need to rmmod vfio first and then modprobe again?
If it is inserted as a module then you can remove using rmmod and then
modprobe again with the dma_entry_limit parameter. Also note,
vfio_iommu_type1 is the module which is limiting the entries to 64K, so
this module needs to be inserted again providing the dma_entry_limit
module param.
In case the module is built-in you can provide via kernel command line
parameter (ref:
https://www.kernel.org/doc/html/v4.12/admin-guide/kernel-parameters.html).
As per this ref document, "vfio_iommu_type1.dma_entry_limit=1000000"
should be used in the bootargs to set the module parameters.
FYI.. DPDK documentation also mentions the limitation at:
https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html
Yes the parameter is discussed in
https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html#vfio-memory-mapping-limits
but it does not mention we may need to decrease it with --no-huge.
Please could you add this to the documentation?
sure! Ill send out a patch for this.
