From: David Marchand <david.march...@redhat.com>
Current KNI implementation only operates in IOVA_PA mode
patch adds required functionality to enable KNI in
IOVA_VA mode.
Due to IOVA to KVA address translations, based on the KNI
use case there can be a performance impact.
To mitigate the impact on performance,forcing IOVA to PA
via eal "--iova-mode=pa" option can be used, IOVA_DC bus
iommu scheme can also result in IOVA as PA.
During KNI creation, app's iova_mode details are passed to
the KNI kernel module, accordingly kernel module translates
PA/IOVA addresses to KVA and vice-versa.
Signed-off-by: David Marchand <david.march...@redhat.com>
Signed-off-by: Vamsi Attunuru <vattun...@marvell.com>
Signed-off-by: Kiran Kumar K <kirankum...@marvell.com>
---
doc/guides/prog_guide/kernel_nic_interface.rst | 14 ++++++++++++++
doc/guides/rel_notes/release_19_11.rst | 13 ++++++++++++-
lib/librte_eal/linux/eal/eal.c | 11 +++++++++--
lib/librte_kni/rte_kni.c | 5 +++++
4 files changed, 40 insertions(+), 3 deletions(-)
diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst
b/doc/guides/prog_guide/kernel_nic_interface.rst
index 2fd58e1..e688efc 100644
--- a/doc/guides/prog_guide/kernel_nic_interface.rst
+++ b/doc/guides/prog_guide/kernel_nic_interface.rst
@@ -300,6 +300,20 @@ The sk_buff is then freed and the mbuf sent in the tx_q
FIFO.
The DPDK TX thread dequeues the mbuf and sends it to the PMD via
``rte_eth_tx_burst()``.
It then puts the mbuf back in the cache.
+IOVA = VA: Support
+------------------
+
+KNI operates in IOVA_VA scheme when
+
+- LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0) and
+- eal option `iova-mode=va` is passed or bus IOVA scheme in the DPDK is
selected
+ as RTE_IOVA_VA.
+
+Due to IOVA to KVA address translations, based on the KNI use case there
+can be a performance impact. For mitigation, forcing IOVA to PA via eal
+"--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also
+result in IOVA as PA.
+
Ethtool
-------
diff --git a/doc/guides/rel_notes/release_19_11.rst
b/doc/guides/rel_notes/release_19_11.rst
index 682c1bd..bed3344 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -292,8 +292,19 @@ New Features
compilers store their internal representation of the source code that
the linker uses at the final stage of compilation process.
- See :doc:`../prog_guide/lto` for more information:
+* **Added IOVA as VA support for KNI.**
+
+ * Added IOVA = VA support for KNI, KNI can operate in IOVA = VA mode when
+ `iova-mode=va` eal option is passed to the application or when bus IOVA
+ scheme is selected as RTE_IOVA_VA. This mode only works on Linux Kernel
+ versions 4.6.0 and above.
+ * Due to IOVA to KVA address translations, based on the KNI use case there
+ can be a performance impact. For mitigation, forcing IOVA to PA via eal
+ "--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also
+ result in IOVA as PA.
+
+ See :doc:`../prog_guide/lto` for more information:
Removed Items
diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c
index 9e2d50c..b5b7150 100644
--- a/lib/librte_eal/linux/eal/eal.c
+++ b/lib/librte_eal/linux/eal/eal.c
@@ -1073,6 +1073,11 @@ rte_eal_init(int argc, char **argv)
*/
iova_mode = RTE_IOVA_VA;
RTE_LOG(DEBUG, EAL, "Physical addresses are
unavailable, selecting IOVA as VA mode.\n");
+#if defined(RTE_LIBRTE_KNI) && LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0)
+ } else if (rte_eal_check_module("rte_kni") == 1) {
+ iova_mode = RTE_IOVA_PA;
+ RTE_LOG(DEBUG, EAL, "KNI is loaded, selecting
IOVA as PA mode for better KNI perfomance.\n");
+#endif
} else if (is_iommu_enabled()) {
/* we have an IOMMU, pick IOVA as VA mode */
iova_mode = RTE_IOVA_VA;
@@ -1085,8 +1090,10 @@ rte_eal_init(int argc, char **argv)
RTE_LOG(DEBUG, EAL, "IOMMU is not available,
selecting IOVA as PA mode.\n");
}
}
-#ifdef RTE_LIBRTE_KNI
- /* Workaround for KNI which requires physical address to work */
+#if defined(RTE_LIBRTE_KNI) && LINUX_VERSION_CODE < KERNEL_VERSION(4, 6, 0)
+ /* Workaround for KNI which requires physical address to work
+ * in kernels < 4.6
+ */
if (iova_mode == RTE_IOVA_VA &&
rte_eal_check_module("rte_kni") == 1) {
if (phys_addrs) {
diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index 7fbcf22..0507a48 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -10,6 +10,7 @@
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
+#include <linux/version.h>
#include <rte_spinlock.h>
#include <rte_string_fns.h>
@@ -97,10 +98,12 @@ static volatile int kni_fd = -1;
int
rte_kni_init(unsigned int max_kni_ifaces __rte_unused)
{
+#if KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE
if (rte_eal_iova_mode() != RTE_IOVA_PA) {
RTE_LOG(ERR, KNI, "KNI requires IOVA as PA\n");
return -1;
}
+#endif
/* Check FD and open */
if (kni_fd < 0) {
@@ -302,6 +305,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool,
kni->group_id = conf->group_id;
kni->mbuf_size = conf->mbuf_size;
+ dev_info.iova_mode = (rte_eal_iova_mode() == RTE_IOVA_VA) ? 1 : 0;
+
ret = ioctl(kni_fd, RTE_KNI_IOCTL_CREATE, &dev_info);
if (ret < 0)
goto ioctl_fail;
--
2.8.4
