Hi Lingyu,
On 7/4/2023 10:27 AM, Lingyu Liu wrote:
Describe iavf live migration steps.
Signed-off-by: Lingyu Liu <lingyu....@intel.com>
---
v2: Fixed CI.
Added brief introduction about live migration.
Clarified this is iavf feature.
---
doc/guides/nics/intel_vf.rst | 101 +++++++++++++++++++++++++
doc/guides/rel_notes/release_23_07.rst | 3 +
2 files changed, 104 insertions(+)
diff --git a/doc/guides/nics/intel_vf.rst b/doc/guides/nics/intel_vf.rst
index d365dbc185..dd75aadf51 100644
--- a/doc/guides/nics/intel_vf.rst
+++ b/doc/guides/nics/intel_vf.rst
@@ -622,6 +622,107 @@ which belongs to the destination VF on the VM.
Inter-VM Communication
+Live Migrating a VM running DPDK
+--------------------------------
+
+Live migration refers to the process of moving a running virtual machine (VM)
or application
+between different physical machines without disconnecting the client or
application
+(see https://en.wikipedia.org/wiki/Live_migration for more information).
+
Please also add some introduction about the vfio migration and the link
for qemu vfio migration doc:
https://qemu.readthedocs.io/en/latest/devel/vfio-migration.html
+This part describes how to migrate a Virtual Machine which has a iavf device.
+
+The following describes a target environment:
+
+* Host Operating System: Ubuntu 20.04.5
+
+* Guest Operating System: Ubuntu 20.04.5
+
+* Linux Kernel Version: 5.15.0-72-generic
+
+* Target Applications: dpdk-testpmd
+
+* Ice Kernel Driver Version: 1.11.17.1
`<https://www.intel.com/content/www/us/en/download/19630>`_
+
+* Qemu Version: 7.2
+
+The setup procedure is as follows:
+
+#. Before booting the Host OS, open **BIOS setup** and enable **Intel® VT
features**.
+
Can you add some description about kernel boot parameters?
+#. In the Host OS
+
+ Install the ice driver and migration driver:
+
+ .. code-block:: console
+
+ insmod ice.ko
+ insmod ice-vfio-pci.ko
+
+ Create 2 VFs and bind them to vfio pci driver:
+
+ .. code-block:: console
+
+ echo 2 > /sys/bus/pci/devices/0000:ca:00.1/sriov_numvfs
+ echo "8086 1889" > /sys/bus/pci/drivers/ice-vfio-pci/new_id
+ dpdk-devbind.py -b ice-vfio-pci 0000:ca:11.0
+ dpdk-devbind.py -b ice-vfio-pci 0000:ca:11.1
ice-vfio-pci binding is not supported on dpdk-devbind.py yet. Please use
sysfs to bind device and driver
+
+ .. note::
+
+ The command above creates two vfs for device 0000:ca:00.1:
+
+ .. code-block:: console
+
+ 0000:ca:11.0 'Ethernet Adaptive Virtual Function 1889' if=
drv=ice-vfio-pci unused=iavf
+ 0000:ca:11.1 'Ethernet Adaptive Virtual Function 1889' if=
drv=ice-vfio-pci unused=iavf
+
+#. Now, start the migration source Virtual Machine by running the following
command:
+
+ .. code-block:: console
+
+ qemu/build/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cpu host -m
4G -smp 1 -device
vfio-pci,host=0000:ca:11.0,x-enable-migration=true,x-pre-copy-dirty-page-tracking=off
-drive file=ubuntu-2004.qcow2 -nic user,hostfwd=tcp::5555-:22 -monitor stdio
+
+ .. note::
+ The vfio-pci,host=0000:ca:11.0 value indicates that you want to attach
a vfio PCI device
+ to a Virtual Machine and the respective (Bus:Device.Function) numbers
should be passed,
+ x-enable-migration=true indicates that this VF supports migration.
Dirty page tracking
+ is not supported, so set x-pre-copy-dirty-page-tracking=off.
+
+#. In VM, install iavf driver and vfio-pci driver
+
+ .. code-block:: console
+
+ insmod iavf.ko
+ modprobe vfio enable_unsafe_noiommu_mode=1
+ moodprobe vfio-pci
+
+#. Bind net device to vfio-pci driver and launch dpdk-testpmd
+
+ .. code-block:: console
+
We need to ensure dpdk-testpmd should be running independently of ssh
console and suggest to put it in a background process like tmux/screen
so that migration will not causing ssh console exit and dpdk-testpmd
killed. Please add related description.
+ dpdk-testpmd -l 0-1 -- -i
+ testpmd> set txpkts 64
+ testpmd> start tx_first
+
+#. Start the migration destination Virtual Machine
+
+ .. code-block:: console
+
+ qemu/build/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cpu host -m
4G -smp 1 -device
vfio-pci,host=0000:ca:11.1,x-enable-migration=true,x-pre-copy-dirty-page-tracking=off
-drive file=ubuntu-2004.qcow2 -nic user,hostfwd=tcp::5556-:22 -monitor stdio
-incoming tcp:127.0.0.1:4444
+
+ #. Start migration by issuing the command in qemu console
+
+ .. code-block:: console
+
+ migrate -d tcp:127.0.0.1:4444
+
+#. Log in the destination VM, and dpdk-testpmd is not interrupt
+
+.. note::
+
+ Host kernel version above 5.17 is currently not supported due to kAPI
change.
+
+
Windows Support
---------------
diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst
index 4459144140..a187cb6146 100644
--- a/doc/guides/rel_notes/release_23_07.rst
+++ b/doc/guides/rel_notes/release_23_07.rst
@@ -200,6 +200,9 @@ New Features
Enhanced the GRO library to support TCP packets over IPv6 network.
+* **Updated Intel iavf driver.**
+
+ Added the document for iavf driver live migration.
Suggest to highlight iavf driver live migration based on KVM vfio migration.
Removed Items
-------------
