On 2025/2/26 21:45, Stephen Hemminger wrote:
Caution: This is an external email. Please be very careful when clicking links
or opening attachments. See http://nok.it/nsb for additional information.
On Wed, 22 Jan 2025 16:15:03 +0800
Yang Ming <ming.1.y...@nokia-sbell.com> wrote:
On 2025/1/18 00:47, Stephen Hemminger wrote:
Caution: This is an external email. Please be very careful when clicking links
or opening attachments. See http://nok.it/nsb for additional information.
On Fri, 17 Jan 2025 15:28:47 +0800
Yang Ming <ming.1.y...@nokia-sbell.com> wrote:
DPDK detect vfio container according the existence of vfio
module. But for container with non-privileged mode, there is
possibility that no VFIO_DIR(/dev/vfio) mapping from host to
container when host have both Intel NIC and Mellanox NIC but
this conntainer only allocate VFs from Mellanox NIC.
In this case, vfio kernel module has already been loaded from
the host.
This scenario will cause the error log occurs in DPDK primary
process as below:
'EAL: cannot open VFIO container, error 2 (No such file or
directory)'
'EAL: VFIO support could not be initialized'
Because `rte_vfio_enable()` call `rte_vfio_get_container_fd()`
to execute `vfio_container_fd = open(VFIO_CONTAINER_PATH,
O_RDWR);` but VFIO_CONTAINER_PATH(/dev/vfio/vfio) doesn't exist
in this container.
This scenario will also lead to the delay of DPDK secondary
process because `default_vfio_cfg->vfio_enabled = 0` and
`default_vfio_cfg->vfio_container_fd = -1`, socket error will
be set in DPDK primary process when it sync this info to
the secondary process.
This patch use to skip this kind of useless detection for this
scenario.
Signed-off-by: Yang Ming <ming.1.y...@nokia-sbell.com>
---
lib/eal/linux/eal_vfio.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/lib/eal/linux/eal_vfio.c b/lib/eal/linux/eal_vfio.c
index 7132e24cba..1679d29263 100644
--- a/lib/eal/linux/eal_vfio.c
+++ b/lib/eal/linux/eal_vfio.c
@@ -7,6 +7,7 @@
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
+#include <dirent.h>
#include <rte_errno.h>
#include <rte_log.h>
@@ -1083,6 +1084,7 @@ rte_vfio_enable(const char *modname)
/* initialize group list */
int i, j;
int vfio_available;
+ DIR *dir;
const struct internal_config *internal_conf =
eal_get_internal_configuration();
@@ -1119,6 +1121,15 @@ rte_vfio_enable(const char *modname)
return 0;
}
+ /* return 0 if VFIO directory not exist for container with non-privileged mode */
+ dir = opendir(VFIO_DIR);
+ if (dir == NULL) {
+ EAL_LOG(DEBUG,
+ "VFIO directory not exist, skipping VFIO support...");
+ return 0;
+ }
+ closedir(dir);
You need to test the non-container cases.
If vfio is loaded /dev/vfio is a character device (not a directory)
Also looks suspicious that VFIO_DIR is defined but never used currently.
Hi Stephen,
For non-container test, /dev/vfio/vfio will be character device, not
/dev/vfio.
Here is the command result on my testing environment with Intel NIC.
[root@computer-1 testuser]# ls -l /dev/vfio
total 0
crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
[root@computer-1 testuser]# dpdk-devbind.py -b vfio-pci 0000:04:10.2
[root@computer-1 testuser]# ls -l /dev/vfio
total 0
crw-------. 1 root root 239, 0 Jan 22 01:52 59
crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
[root@computer-1 testuser]# dpdk-devbind.py -b ixgbevf 0000:04:10.2
[root@computer-1 testuser]# ls -l /dev/vfio
total 0
crw-rw-rw-. 1 root root 10, 196 Jan 22 01:50 vfio
Can you confirm your test scenario?
When vfio-pci is loaded but no device bound:
$ ls -l /dev/vfio
total 0
crw-rw-rw- 1 root root 10, 196 Feb 26 05:39 vfio
After binding device
$ ls -l /dev/vfio
total 0
crw------- 1 root root 511, 0 Feb 26 05:42 15
crw-rw-rw- 1 root root 10, 196 Feb 26 05:39 vfio
So testing for /dev/vfio is good indication that module is loaded.
Not sure what I was thinking earlier.
Hi Stephen,
Thank you very much for your explanation. It's very clear.
Can you help to accept this patch, or we need more comments?
Brs,
Yang Ming
