Re: [RFC PATCH v1 00/19] Factor out HVF's instruction emulator

2025-02-21 Thread Wei Liu
On Fri, Feb 21, 2025 at 04:53:26PM +, Peter Maydell wrote: > On Fri, 21 Feb 2025 at 14:02, Wei Liu wrote: > > > > Hi, > > > > Microsoft's Linux Systems Group developed a Linux driver for the Microsoft > > Hypervisor (MSHV for short). The driver is being u

Re: [RFC PATCH v1 00/19] Factor out HVF's instruction emulator

2025-02-21 Thread Wei Liu
On Fri, Feb 21, 2025 at 05:36:39PM +0100, Paolo Bonzini wrote: > On 2/21/25 09:36, Wei Liu wrote: > > This patch series attempts to make the instruction emulator in HVF a common > > component for the i386 target. It removes HVF specific code by either using > > a > > s

[RFC PATCH v1 14/19] target/i386/hvf: drop unused headers

2025-02-21 Thread Wei Liu
Signed-off-by: Wei Liu --- target/i386/hvf/x86_decode.c | 3 --- target/i386/hvf/x86_emu.c| 4 2 files changed, 7 deletions(-) diff --git a/target/i386/hvf/x86_decode.c b/target/i386/hvf/x86_decode.c index 31285952ad..ffece4773b 100644 --- a/target/i386/hvf/x86_decode.c +++ b/target

[RFC PATCH v1 05/19] target/i386/hvf: remove HVF specific calls from x86_decode.c

2025-02-21 Thread Wei Liu
Use the newly defined emul_ops. This allows the module to be reused by other accelerator in the future. No functional change intended. Signed-off-by: Wei Liu --- target/i386/hvf/x86_decode.c | 19 ++- 1 file changed, 6 insertions(+), 13 deletions(-) diff --git a/target/i386

[RFC PATCH v1 09/19] target/i386/hvf: use emul_ops->read_mem in x86_emu.c

2025-02-21 Thread Wei Liu
No functional change. Signed-off-by: Wei Liu --- target/i386/hvf/x86_emu.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/target/i386/hvf/x86_emu.c b/target/i386/hvf/x86_emu.c index d0a8e221ea..f1244640e6 100644 --- a/target/i386/hvf/x86_emu.c +++ b/target/i386

[RFC PATCH v1 02/19] target/i386/hvf: fix the declaration of hvf_handle_io

2025-02-21 Thread Wei Liu
-i386.h. Take the chance to change the first argument's type to be CPUState. Signed-off-by: Wei Liu --- target/i386/hvf/hvf-i386.h | 2 +- target/i386/hvf/hvf.c | 6 +++--- target/i386/hvf/x86_emu.c | 4 +--- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/target/i3

[RFC PATCH v1 16/19] target/i386/hvf: rename some include guards

2025-02-21 Thread Wei Liu
These headers will be moved out to its own component. Signed-off-by: Wei Liu --- target/i386/hvf/x86.h| 4 ++-- target/i386/hvf/x86_decode.h | 4 ++-- target/i386/hvf/x86_flags.h | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/target/i386/hvf/x86.h b/target

[RFC PATCH v1 07/19] target/i386/hvf: provide and use handle_io in emul_ops

2025-02-21 Thread Wei Liu
This drops the calls to hvf_handle_io from x86_emu.c. Signed-off-by: Wei Liu --- target/i386/hvf/hvf.c | 1 + target/i386/hvf/x86_emu.c | 30 +++--- target/i386/hvf/x86_emu.h | 2 ++ 3 files changed, 18 insertions(+), 15 deletions(-) diff --git a/target/i386/hvf

[RFC PATCH v1 11/19] target/i386/hvf: move and rename simulate_{rdmsr, wrmsr}

2025-02-21 Thread Wei Liu
This requires making raise_exception non-static. That function needs to be renamed to avoid clashing with a function in TCG. Mostly code movement. No functional change. Signed-off-by: Wei Liu --- target/i386/hvf/hvf-i386.h | 2 + target/i386/hvf/hvf.c | 216

[RFC PATCH v1 03/19] target/i386/hvf: use x86_segment in x86_decode.c

2025-02-21 Thread Wei Liu
Make the code to rely on the segment definition for checking cs.db. This allows removing HVF specific VMX related definition from the decoder. Introduce a function for retrieving the CS descriptor. No functional change intended. Signed-off-by: Wei Liu --- target/i386/hvf/x86_decode.c | 20

[RFC PATCH v1 12/19] target/i386/hvf: provide and use simulate_{wrmsr, rdmsr} in emul_ops

2025-02-21 Thread Wei Liu
Change the first argument's type to be CPUState to match other hooks. Signed-off-by: Wei Liu --- target/i386/hvf/hvf-i386.h | 4 ++-- target/i386/hvf/hvf.c | 18 ++ target/i386/hvf/x86_emu.c | 4 ++-- target/i386/hvf/x86_emu.h | 2 ++ 4 files changed, 16 inser

[RFC PATCH v1 13/19] target/i386: rename lazy flags field and its type

2025-02-21 Thread Wei Liu
The same structure and code can be used by other accelerators. Drop the hvf prefix in the type and field name. No functional change. Signed-off-by: Wei Liu --- target/i386/cpu.h | 6 ++-- target/i386/hvf/x86_flags.c | 56 ++--- 2 files changed, 31

[RFC PATCH v1 18/19] target/i386/x86-insn-emul: add a panic.h

2025-02-21 Thread Wei Liu
The macros will be used by the instruction emulator. The code is the same as the one under hvf. Signed-off-by: Wei Liu --- target/i386/x86-insn-emul/panic.h | 45 +++ 1 file changed, 45 insertions(+) create mode 100644 target/i386/x86-insn-emul/panic.h diff --git a

[RFC PATCH v1 06/19] target/i386/hvf: move and rename {load, store}_regs

2025-02-21 Thread Wei Liu
They contain HVF specific code. Move them to a better location and add "hvf_" prefix. Fix up all the call sites. No functional change. Signed-off-by: Wei Liu --- target/i386/hvf/hvf.c | 71 +++--- target/i386/hvf/x86_e

[RFC PATCH v1 19/19] target/i386: move x86 instruction emulator out of hvf

2025-02-21 Thread Wei Liu
Move x86_decode, x86_emu, x86_flags and some headers to the new location. Fix up all the inclusion sites in hvf. Signed-off-by: Wei Liu --- target/i386/hvf/hvf.c | 8 target/i386/hvf/meson.build | 3 --- target/i386/hvf/vmx.h

[RFC PATCH v1 01/19] target/i386/hvf: fix a typo in a type name

2025-02-21 Thread Wei Liu
The prefix x68 is wrong. Change it to x86. Signed-off-by: Wei Liu --- target/i386/hvf/hvf.c | 2 +- target/i386/hvf/x86.c | 4 ++-- target/i386/hvf/x86.h | 8 target/i386/hvf/x86_descr.c | 8 target/i386/hvf/x86_descr.h | 6 +++--- target/i386/hvf

[RFC PATCH v1 15/19] target/i386/hvf: drop some dead code

2025-02-21 Thread Wei Liu
Signed-off-by: Wei Liu --- target/i386/hvf/x86_emu.c | 4 1 file changed, 4 deletions(-) diff --git a/target/i386/hvf/x86_emu.c b/target/i386/hvf/x86_emu.c index 84f97ed386..44ef068bef 100644 --- a/target/i386/hvf/x86_emu.c +++ b/target/i386/hvf/x86_emu.c @@ -1240,10 +1240,6 @@ static void

[RFC PATCH v1 00/19] Factor out HVF's instruction emulator

2025-02-21 Thread Wei Liu
elerator. That looks to be a worse option. Thanks, Wei. Wei Liu (19): target/i386/hvf: fix a typo in a type name target/i386/hvf: fix the declaration of hvf_handle_io target/i386/hvf: use x86_segment in x86_decode.c target/i386/hvf: introduce x86_emul_ops target/i386/hvf: remove HVF spec

[RFC PATCH v1 04/19] target/i386/hvf: introduce x86_emul_ops

2025-02-21 Thread Wei Liu
This will be used to remove HVF specific code from the instruction emulator. For now we only introduce two hooks for x86_decode.c. More hooks will be added when the code is refactored. The emulator initialization function now takes in a pointer to the ops structure. Signed-off-by: Wei Liu

[RFC PATCH v1 08/19] target/i386: rename hvf_mmio_buf to mmio_buf

2025-02-21 Thread Wei Liu
We want to refactor HVF's instruction emulator to a common component. Renaming hvf_mmio_buf removes the association between HVF and the instruction emulator. The definition of the field is still guarded by CONFIG_HVF for now, since it is the only user. No functional change. Signed-off-by

[RFC PATCH v1 17/19] target/i386: add a directory for x86 instruction emulator

2025-02-21 Thread Wei Liu
Signed-off-by: Wei Liu --- target/i386/meson.build | 1 + target/i386/x86-insn-emul/meson.build | 0 2 files changed, 1 insertion(+) create mode 100644 target/i386/x86-insn-emul/meson.build diff --git a/target/i386/meson.build b/target/i386/meson.build index 2e9c472f49

[RFC PATCH v1 10/19] taret/i386/hvf: provide and use write_mem in emul_ops

2025-02-21 Thread Wei Liu
Signed-off-by: Wei Liu --- target/i386/hvf/hvf.c | 6 ++ target/i386/hvf/x86_emu.c | 8 target/i386/hvf/x86_emu.h | 1 + 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c index 533b05577d..e108e2bbe6 100644 --- a/target

Re: [RFC v3 4/5] virtio: add in_xlat_addr & out_xlat_addr VirtQueueElement members

2025-01-22 Thread Si-Wei Liu
On 1/16/2025 11:02 AM, Eugenio Perez Martin wrote: On Fri, Jan 10, 2025 at 6:09 PM Jonah Palmer wrote: Adds the in_xlat_addr & out_xlat_addr hwaddr arrays to the VirtQueueElement struct and introduces an optional GPA output parameter to dma_memory_map(). These arrays will store a VirtQueue

Re: [RFC v2 1/2] vhost-vdpa: Implement IOVA->GPA tree

2024-10-10 Thread Si-Wei Liu
On 10/9/2024 2:29 AM, Eugenio Perez Martin wrote: On Tue, Oct 8, 2024 at 10:30 PM Si-Wei Liu wrote: On 10/8/2024 8:40 AM, Jonah Palmer wrote: On 10/8/24 2:51 AM, Eugenio Perez Martin wrote: On Tue, Oct 8, 2024 at 2:14 AM Si-Wei Liu wrote: On 10/7/2024 6:51 AM, Eugenio Perez Martin

Re: [RFC v2 1/2] vhost-vdpa: Implement IOVA->GPA tree

2024-10-08 Thread Si-Wei Liu
On 10/8/2024 8:40 AM, Jonah Palmer wrote: On 10/8/24 2:51 AM, Eugenio Perez Martin wrote: On Tue, Oct 8, 2024 at 2:14 AM Si-Wei Liu wrote: On 10/7/2024 6:51 AM, Eugenio Perez Martin wrote: On Fri, Oct 4, 2024 at 8:48 PM Jonah Palmer wrote: On 10/4/24 11:17 AM, Eugenio Perez

Re: [RFC v2 1/2] vhost-vdpa: Implement IOVA->GPA tree

2024-10-07 Thread Si-Wei Liu
On 10/7/2024 6:51 AM, Eugenio Perez Martin wrote: On Fri, Oct 4, 2024 at 8:48 PM Jonah Palmer wrote: On 10/4/24 11:17 AM, Eugenio Perez Martin wrote: On Fri, Oct 4, 2024 at 2:45 PM Jonah Palmer wrote: Implements the IOVA->GPA tree for handling mapping, unmapping, and translations for gu

Re: [RFC 1/2] vhost-vdpa: Decouple the IOVA allocator

2024-09-25 Thread Si-Wei Liu
On 9/11/2024 3:45 AM, Eugenio Perez Martin wrote: On Wed, Sep 11, 2024 at 11:06 AM Si-Wei Liu wrote: On 9/9/2024 11:22 PM, Eugenio Perez Martin wrote: On Tue, Sep 10, 2024 at 7:30 AM Si-Wei Liu wrote: Sorry for the delayed response, it seems I missed the email reply for some reason

Re: [PATCH v2 2/2] net: move backend cleanup to NIC cleanup

2024-09-17 Thread Si-Wei Liu
the first instance of QEMU is finished. Move the cleanup from qemu_cleanup to the NIC deletion and to net_cleanup. Fixes: a0d7215e33 ("vhost-vdpa: do not cleanup the vdpa/vhost-net structures if peer nic is present") Reported-by: Lei Yang Signed-off-by: Eugenio Pérez Reviewed-by: Si-Wei

Re: [RFC 1/2] vhost-vdpa: Decouple the IOVA allocator

2024-09-11 Thread Si-Wei Liu
On 9/9/2024 11:22 PM, Eugenio Perez Martin wrote: On Tue, Sep 10, 2024 at 7:30 AM Si-Wei Liu wrote: Sorry for the delayed response, it seems I missed the email reply for some reason during the long weekend. On 9/2/2024 4:09 AM, Eugenio Perez Martin wrote: On Fri, Aug 30, 2024 at 11:05 PM

Re: [RFC 1/2] vhost-vdpa: Decouple the IOVA allocator

2024-09-09 Thread Si-Wei Liu
Sorry for the delayed response, it seems I missed the email reply for some reason during the long weekend. On 9/2/2024 4:09 AM, Eugenio Perez Martin wrote: On Fri, Aug 30, 2024 at 11:05 PM Si-Wei Liu wrote: On 8/30/2024 1:05 AM, Eugenio Perez Martin wrote: On Fri, Aug 30, 2024 at 6:20 AM

Re: [PATCH 0/2] Move net backend cleanup to NIC cleanup

2024-09-09 Thread Si-Wei Liu
Hi Jason, It seems this series wasn't applied successfully, I still cannot see it from the latest tree. Any idea? In any case the fix LGTM. Reviewed-by: Si-Wei Liu Thanks, -Siwei On 1/31/2024 9:43 PM, Jason Wang wrote: On Mon, Jan 29, 2024 at 9:24 PM Eugenio Pérez wrote: C

Re: [RFC 1/2] vhost-vdpa: Decouple the IOVA allocator

2024-08-30 Thread Si-Wei Liu
On 8/30/2024 1:05 AM, Eugenio Perez Martin wrote: On Fri, Aug 30, 2024 at 6:20 AM Si-Wei Liu wrote: On 8/29/2024 9:53 AM, Eugenio Perez Martin wrote: On Wed, Aug 21, 2024 at 2:56 PM Jonah Palmer wrote: Decouples the IOVA allocator from the IOVA->HVA tree and instead adds the alloca

Re: [RFC 1/2] vhost-vdpa: Decouple the IOVA allocator

2024-08-29 Thread Si-Wei Liu
On 8/29/2024 9:53 AM, Eugenio Perez Martin wrote: On Wed, Aug 21, 2024 at 2:56 PM Jonah Palmer wrote: Decouples the IOVA allocator from the IOVA->HVA tree and instead adds the allocated IOVA range to an IOVA-only tree (iova_map). This IOVA tree will hold all IOVA ranges that have been alloca

Re: [RFC 0/2] Identify aliased maps in vdpa SVQ iova_tree

2024-08-01 Thread Si-Wei Liu
On 8/1/2024 1:22 AM, Eugenio Perez Martin wrote: On Thu, Aug 1, 2024 at 2:41 AM Si-Wei Liu wrote: Hi Jonah, On 7/31/2024 7:09 AM, Jonah Palmer wrote: Let me clarify, correct me if I was wrong: 1) IOVA allocator is still implemented via a tree, we just don't need to store how the IO

Re: [RFC 0/2] Identify aliased maps in vdpa SVQ iova_tree

2024-07-31 Thread Si-Wei Liu
Hi Jonah, On 7/31/2024 7:09 AM, Jonah Palmer wrote: Let me clarify, correct me if I was wrong: 1) IOVA allocator is still implemented via a tree, we just don't need to store how the IOVA is used 2) A dedicated GPA -> IOVA tree, updated via listeners and is used in the datapath SVQ transla

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-07 Thread Si-Wei Liu
On 5/1/2024 11:44 PM, Eugenio Perez Martin wrote: On Thu, May 2, 2024 at 1:16 AM Si-Wei Liu wrote: On 4/30/2024 10:19 AM, Eugenio Perez Martin wrote: On Tue, Apr 30, 2024 at 7:55 AM Si-Wei Liu wrote: On 4/29/2024 1:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-07 Thread Si-Wei Liu
On 5/1/2024 11:18 PM, Eugenio Perez Martin wrote: On Thu, May 2, 2024 at 12:09 AM Si-Wei Liu wrote: On 4/30/2024 11:11 AM, Eugenio Perez Martin wrote: On Mon, Apr 29, 2024 at 1:19 PM Jonah Palmer wrote: On 4/29/24 4:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-01 Thread Si-Wei Liu
On 4/30/2024 10:19 AM, Eugenio Perez Martin wrote: On Tue, Apr 30, 2024 at 7:55 AM Si-Wei Liu wrote: On 4/29/2024 1:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si-Wei Liu wrote: On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12:21 AM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-01 Thread Si-Wei Liu
On 4/30/2024 11:11 AM, Eugenio Perez Martin wrote: On Mon, Apr 29, 2024 at 1:19 PM Jonah Palmer wrote: On 4/29/24 4:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si-Wei Liu wrote: On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-29 Thread Si-Wei Liu
On 4/29/2024 1:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si-Wei Liu wrote: On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12:21 AM Si-Wei Liu wrote: On 4/22/2024 1:49 AM, Eugenio Perez Martin wrote: On Sat, Apr 20, 2024 at 1:50 AM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-25 Thread Si-Wei Liu
On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12:21 AM Si-Wei Liu wrote: On 4/22/2024 1:49 AM, Eugenio Perez Martin wrote: On Sat, Apr 20, 2024 at 1:50 AM Si-Wei Liu wrote: On 4/19/2024 1:29 AM, Eugenio Perez Martin wrote: On Thu, Apr 18, 2024 at 10:46 PM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-23 Thread Si-Wei Liu
On 4/22/2024 1:49 AM, Eugenio Perez Martin wrote: On Sat, Apr 20, 2024 at 1:50 AM Si-Wei Liu wrote: On 4/19/2024 1:29 AM, Eugenio Perez Martin wrote: On Thu, Apr 18, 2024 at 10:46 PM Si-Wei Liu wrote: On 4/10/2024 3:03 AM, Eugenio Pérez wrote: IOVA tree is also used to track the

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-19 Thread Si-Wei Liu
On 4/19/2024 1:29 AM, Eugenio Perez Martin wrote: On Thu, Apr 18, 2024 at 10:46 PM Si-Wei Liu wrote: On 4/10/2024 3:03 AM, Eugenio Pérez wrote: IOVA tree is also used to track the mappings of virtio-net shadow virtqueue. This mappings may not match with the GPA->HVA ones. This cause

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-18 Thread Si-Wei Liu
On 4/10/2024 3:03 AM, Eugenio Pérez wrote: IOVA tree is also used to track the mappings of virtio-net shadow virtqueue. This mappings may not match with the GPA->HVA ones. This causes a problem when overlapped regions (different GPA but same translated HVA) exists in the tree, as looking the

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-04-02 Thread Si-Wei Liu
On 4/2/2024 5:01 AM, Eugenio Perez Martin wrote: On Tue, Apr 2, 2024 at 8:19 AM Si-Wei Liu wrote: On 2/14/2024 11:11 AM, Eugenio Perez Martin wrote: On Wed, Feb 14, 2024 at 7:29 PM Si-Wei Liu wrote: Hi Michael, On 2/13/2024 2:22 AM, Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-04-01 Thread Si-Wei Liu
On 2/14/2024 11:11 AM, Eugenio Perez Martin wrote: On Wed, Feb 14, 2024 at 7:29 PM Si-Wei Liu wrote: Hi Michael, On 2/13/2024 2:22 AM, Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05:10:36PM -0800, Si-Wei Liu wrote: Hi Eugenio, I thought this new code looks good to me and the

Re: [External] : Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-25 Thread Si-Wei Liu
On 3/24/2024 11:13 PM, Jason Wang wrote: On Sat, Mar 23, 2024 at 5:14 AM Si-Wei Liu wrote: On 3/21/2024 10:08 PM, Jason Wang wrote: On Fri, Mar 22, 2024 at 5:43 AM Si-Wei Liu wrote: On 3/20/2024 8:56 PM, Jason Wang wrote: On Thu, Mar 21, 2024 at 5:03 AM Si-Wei Liu wrote: On 3/19

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-22 Thread Si-Wei Liu
On 3/21/2024 10:08 PM, Jason Wang wrote: On Fri, Mar 22, 2024 at 5:43 AM Si-Wei Liu wrote: On 3/20/2024 8:56 PM, Jason Wang wrote: On Thu, Mar 21, 2024 at 5:03 AM Si-Wei Liu wrote: On 3/19/2024 8:27 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:16 AM Si-Wei Liu wrote: On 3/17

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-21 Thread Si-Wei Liu
On 3/20/2024 8:56 PM, Jason Wang wrote: On Thu, Mar 21, 2024 at 5:03 AM Si-Wei Liu wrote: On 3/19/2024 8:27 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:16 AM Si-Wei Liu wrote: On 3/17/2024 8:22 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:45 AM Si-Wei Liu wrote: On 3/14/2024

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-20 Thread Si-Wei Liu
On 3/19/2024 8:27 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:16 AM Si-Wei Liu wrote: On 3/17/2024 8:22 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:45 AM Si-Wei Liu wrote: On 3/14/2024 9:03 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: On setups

Re: [PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-20 Thread Si-Wei Liu
On 3/19/2024 8:25 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:06 AM Si-Wei Liu wrote: On 3/17/2024 8:20 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:33 AM Si-Wei Liu wrote: On 3/14/2024 8:50 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: There could

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-18 Thread Si-Wei Liu
On 3/17/2024 8:22 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:45 AM Si-Wei Liu wrote: On 3/14/2024 9:03 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: On setups with one or more virtio-net devices with vhost on, dirty tracking iteration increases cost the

Re: [PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-18 Thread Si-Wei Liu
On 3/17/2024 8:20 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:33 AM Si-Wei Liu wrote: On 3/14/2024 8:50 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: There could be a mix of both vhost-user and vhost-kernel clients in the same QEMU process, where separate

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-15 Thread Si-Wei Liu
On 3/14/2024 9:03 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: On setups with one or more virtio-net devices with vhost on, dirty tracking iteration increases cost the bigger the number amount of queues are set up e.g. on idle guests migration the following is

Re: [PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-15 Thread Si-Wei Liu
On 3/14/2024 8:50 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: There could be a mix of both vhost-user and vhost-kernel clients in the same QEMU process, where separate vhost loggers for the specific vhost type have to be used. Make the vhost logger per backend

[PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-14 Thread Si-Wei Liu
eues -> 8.71%[.] vhost_dev_sync_region.isra.13 2 devices, 8 queues -> 7.97% [.] vhost_dev_sync_region.isra.14 Co-developed-by: Joao Martins Signed-off-by: Joao Martins Signed-off-by: Si-Wei Liu --- v3 -> v4: - add comment to clarify effect on cache locality and performance v2

[PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-14 Thread Si-Wei Liu
: Si-Wei Liu --- v3->v4: - remove checking NULL return value from vhost_log_get v2->v3: - remove non-effective assertion that never be reached - do not return NULL from vhost_log_get() - add neccessary assertions to vhost_log_get() --- hw/virtio/vhost.

Re: [PATCH v3 1/2] vhost: dirty log should be per backend type

2024-03-14 Thread Si-Wei Liu
On 3/14/2024 8:25 AM, Eugenio Perez Martin wrote: On Thu, Mar 14, 2024 at 9:38 AM Si-Wei Liu wrote: There could be a mix of both vhost-user and vhost-kernel clients in the same QEMU process, where separate vhost loggers for the specific vhost type have to be used. Make the vhost logger per

Re: [PATCH v3 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-14 Thread Si-Wei Liu
On 3/14/2024 8:34 AM, Eugenio Perez Martin wrote: On Thu, Mar 14, 2024 at 9:38 AM Si-Wei Liu wrote: On setups with one or more virtio-net devices with vhost on, dirty tracking iteration increases cost the bigger the number amount of queues are set up e.g. on idle guests migration the

[PATCH v3 1/2] vhost: dirty log should be per backend type

2024-03-14 Thread Si-Wei Liu
: Si-Wei Liu --- v2->v3: - remove non-effective assertion that never be reached - do not return NULL from vhost_log_get() - add neccessary assertions to vhost_log_get() --- hw/virtio/vhost.c | 50 ++ 1 file changed, 38 insertions(+),

[PATCH v3 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-14 Thread Si-Wei Liu
eues -> 8.71%[.] vhost_dev_sync_region.isra.13 2 devices, 8 queues -> 7.97% [.] vhost_dev_sync_region.isra.14 Co-developed-by: Joao Martins Signed-off-by: Joao Martins Signed-off-by: Si-Wei Liu --- v2 -> v3: - add after-fix benchmark to commit log - rename vhost_log_dev_enabled to

Re: [PATCH v2 1/2] vhost: dirty log should be per backend type

2024-03-13 Thread Si-Wei Liu
On 3/12/2024 8:07 AM, Michael S. Tsirkin wrote: On Wed, Feb 14, 2024 at 10:42:29AM -0800, Si-Wei Liu wrote: Hi Michael, I'm taking off for 2+ weeks, but please feel free to provide comment and feedback while I'm off. I'll be checking emails still, and am about to address an

Re: [PATCH 12/12] vdpa: fix network breakage after cancelling migration

2024-03-13 Thread Si-Wei Liu
On 3/13/2024 11:12 AM, Michael Tokarev wrote: 14.02.2024 14:28, Si-Wei Liu wrote: Fix an issue where cancellation of ongoing migration ends up with no network connectivity. When canceling migration, SVQ will be switched back to the passthrough mode, but the right call fd is not programed to

Re: [PATCH 04/12] vdpa: factor out vhost_vdpa_net_get_nc_vdpa

2024-02-14 Thread Si-Wei Liu
On 2/14/2024 10:54 AM, Eugenio Perez Martin wrote: On Wed, Feb 14, 2024 at 1:39 PM Si-Wei Liu wrote: Introduce new API. No functional change on existing API. Acked-by: Jason Wang Signed-off-by: Si-Wei Liu I'm ok with the new function, but doesn't the compiler complain becaus

Re: [PATCH v2 1/2] vhost: dirty log should be per backend type

2024-02-14 Thread Si-Wei Liu
Hi Michael, I'm taking off for 2+ weeks, but please feel free to provide comment and feedback while I'm off. I'll be checking emails still, and am about to address any opens as soon as I am back. Thanks, -Siwei On 2/14/2024 3:50 AM, Si-Wei Liu wrote: There could be a mix of

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-02-14 Thread Si-Wei Liu
Siwei On 2/13/2024 8:26 AM, Eugenio Perez Martin wrote: On Tue, Feb 13, 2024 at 11:22 AM Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05:10:36PM -0800, Si-Wei Liu wrote: Hi Eugenio, I thought this new code looks good to me and the original issue I saw with x-svq=on should be gone. However,

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-02-14 Thread Si-Wei Liu
Hi Michael, On 2/13/2024 2:22 AM, Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05:10:36PM -0800, Si-Wei Liu wrote: Hi Eugenio, I thought this new code looks good to me and the original issue I saw with x-svq=on should be gone. However, after rebase my tree on top of this, there's

[PATCH v2 2/2] vhost: Perform memory section dirty scans once per iteration

2024-02-14 Thread Si-Wei Liu
ogger device goes away for some reason, the logger will be re-selected from the rest of vhost devices. Co-developed-by: Joao Martins Signed-off-by: Joao Martins Signed-off-by: Si-Wei Liu --- hw/virtio/vhost.c | 75 +++ include/hw/virtio/vhost

[PATCH v2 1/2] vhost: dirty log should be per backend type

2024-02-14 Thread Si-Wei Liu
: Si-Wei Liu --- hw/virtio/vhost.c | 49 + 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 2c9ac79..ef6d9b5 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -43,8 +43,8 @@ do

[PATCH 07/12] vdpa: add vhost_vdpa_set_dev_vring_base trace for svq mode

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 5 - 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace

[PATCH 04/12] vdpa: factor out vhost_vdpa_net_get_nc_vdpa

2024-02-14 Thread Si-Wei Liu
Introduce new API. No functional change on existing API. Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 06c83b4..4168cad 100644 --- a/net/vhost

[PATCH 08/12] vdpa: add trace events for vhost_vdpa_net_load_cmd

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/trace-events | 2 ++ net/vhost-vdpa.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/net/trace-events b/net/trace-events index aab666a..88f56f2 100644 --- a/net/trace-events +++ b

[PATCH 10/12] vdpa: define SVQ transitioning state for mode switching

2024-02-14 Thread Si-Wei Liu
Will be used in following patches. DISABLING(-1) means SVQ is being switched off to passthrough mode. ENABLING(1) means passthrough VQs are being switched to SVQ. DONE(0) means SVQ switching is completed. Signed-off-by: Si-Wei Liu --- include/hw/virtio/vhost-vdpa.h | 9 + 1 file

[PATCH 01/12] vdpa: add back vhost_vdpa_net_first_nc_vdpa

2024-02-14 Thread Si-Wei Liu
Previous commits had it removed. Now adding it back because this function will be needed by future patches. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 15 +-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/net/vhost-vdpa.c b/net

[PATCH 09/12] vdpa: add trace event for vhost_vdpa_net_load_mq

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/trace-events | 1 + net/vhost-vdpa.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/net/trace-events b/net/trace-events index 88f56f2..cda960f 100644 --- a/net/trace-events +++ b

[PATCH 06/12] vdpa: add vhost_vdpa_get_vring_base trace for svq mode

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace

[PATCH 03/12] vdpa: factor out vhost_vdpa_last_dev

2024-02-14 Thread Si-Wei Liu
Generalize duplicated condition check for the last vq of vdpa device to a common function. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b

[PATCH 05/12] vdpa: add vhost_vdpa_set_address_space_id trace

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/trace-events | 3 +++ net/vhost-vdpa.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/net/trace-events b/net/trace-events index 823a071..aab666a 100644 --- a/net/trace-events +++ b

[PATCH 11/12] vdpa: indicate transitional state for SVQ switching

2024-02-14 Thread Si-Wei Liu
svq_switching indicates the transitional state whether or not SVQ mode switching is in progress, and towards which direction. Add the neccessary state around where the switching would take place. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 3 +++ 1 file changed, 3 insertions(+) diff --git

[PATCH 12/12] vdpa: fix network breakage after cancelling migration

2024-02-14 Thread Si-Wei Liu
oning period, the shadow_vqs_enabled hadn't been set back to false yet, causing the installation of call fd inadvertently bypassed. Fixes: a8ac88585da1 ("vhost: Add Shadow VirtQueue call forwarding capabilities") Cc: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu

[PATCH 02/12] vdpa: no repeat setting shadow_data

2024-02-14 Thread Si-Wei Liu
space. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 4479ffa..06c83b4 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -354,13

[PATCH 00/12] Preparatory patches for live migration downtime improvement

2024-02-14 Thread Si-Wei Liu
://lore.kernel.org/qemu-devel/1701970793-6865-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (12): vdpa: add back vhost_vdpa_net_first_nc_vdpa vdpa: no repeat setting shadow_data vdpa: factor out vhost_vdpa_last_dev vdpa: factor out vhost_vdpa_net_get_nc_vdpa vdpa: add

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-02-05 Thread Si-Wei Liu
o move all its dependencies to the initialization too. In particular devices with x-svq=on need a valid iova_tree from the beginning. Simplify the code also consolidating the two creation points: the first data vq in case of SVQ active and CVQ start in case only CVQ uses it. Suggested-by: Si-Wei L

Re: [PATCH 1/6] vdpa: check for iova tree initialized at net_client_start

2024-01-31 Thread Si-Wei Liu
Hi Eugenio, Maybe there's some patch missing, but I saw this core dump when x-svq=on is specified while waiting for the incoming migration on destination host: (gdb) bt #0  0x5643b24cc13c in vhost_iova_tree_map_alloc (tree=0x0, map=map@entry=0x7ffd58c54830) at ../hw/virtio/vhost-iova-tree

[PATCH 31/40] vdpa: batch map and unmap around cvq svq start/stop

2023-12-07 Thread Si-Wei Liu
Coalesce map or unmap operations to exact one DMA batch to reduce potential impact on performance. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index bc72345..1c1d61f 100644 --- a

[PATCH 28/40] vdpa: support iotlb_batch_asid

2023-12-07 Thread Si-Wei Liu
off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 25 +++-- include/hw/virtio/vhost-vdpa.h | 1 + net/vhost-vdpa.c | 1 + 3 files changed, 21 insertions(+), 6 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index d3f5721..b789

[PATCH 40/40] vdpa: add trace event for vhost_vdpa_net_load_mq

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- net/trace-events | 1 + net/vhost-vdpa.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/net/trace-events b/net/trace-events index be087e6..c128cc4 100644 --- a/net/trace-events +++ b/net/trace-events @@ -30,3

[PATCH 07/40] vdpa: move around vhost_vdpa_set_address_space_id

2023-12-07 Thread Si-Wei Liu
Move it a few lines ahead to make function call easier for those before it. No funtional change involved. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 36 ++-- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost

[PATCH 22/40] vdpa: factor out vhost_vdpa_map_batch_begin

2023-12-07 Thread Si-Wei Liu
Refactoring only. No functional change. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 25 - 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 9725d44..b0239b8

[PATCH 12/40] vdpa: check map_thread_enabled before join maps thread

2023-12-07 Thread Si-Wei Liu
The next patches will also register memory listener on demand, hence the need to differentiate the map_thread case from the rest. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost

[PATCH 06/40] vhost: make svq work with gpa without iova translation

2023-12-07 Thread Si-Wei Liu
with dedicated or isolated address space for SVQ descriptors, the IOVA is exactly same as the guest GPA space where translation would not be needed any more. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-shadow-virtqueue.c | 35 +++ 1 file changed, 23 insertions(+

[PATCH 04/40] vdpa: piggyback desc_group index when probing isolated cvq

2023-12-07 Thread Si-Wei Liu
Same as the previous commit, but do it for cvq instead of data vqs. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 21 + 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 0cf3147..cb5705d 100644 --- a/net/vhost-vdpa.c

[PATCH 25/40] vdpa: add asid to dma_batch_once API

2023-12-07 Thread Si-Wei Liu
So that DMA batching API can operate on other ASID than 0. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 4 ++-- hw/virtio/vhost-vdpa.c | 14 -- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 3411a07

[PATCH 35/40] vdpa: add vhost_vdpa_set_address_space_id trace

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- net/trace-events | 3 +++ net/vhost-vdpa.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/net/trace-events b/net/trace-events index 823a071..aab666a 100644 --- a/net/trace-events +++ b/net/trace-events @@ -23,3

[PATCH 21/40] vdpa: vhost_vdpa_dma_batch_end_once rename

2023-12-07 Thread Si-Wei Liu
No functional changes. Rename only. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 47c764b..013bfa2 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost

[PATCH 14/40] vdpa: convert iova_tree to ref count based

2023-12-07 Thread Si-Wei Liu
So that it can be freed from vhost_vdpa_cleanup on the last deref. The next few patches will try to make iova tree life cycle not depend on memory listener, and there's possiblity to keep iova tree around when memory mapping is not changed across device reset. Signed-off-by: Si-Wei Liu ---

[PATCH 11/40] vdpa: factor out vhost_vdpa_last_dev

2023-12-07 Thread Si-Wei Liu
Generalize duplicated condition check for the last vq of vdpa device to a common function. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 30dff95..2b1cc14

[PATCH 15/40] vdpa: add svq_switching and flush_map to header

2023-12-07 Thread Si-Wei Liu
Will be used in next patches. Signed-off-by: Si-Wei Liu --- include/hw/virtio/vhost-vdpa.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 7b8d3bf..0fe0f60 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include

[PATCH 34/40] vdpa: fix network breakage after cancelling migration

2023-12-07 Thread Si-Wei Liu
oning period, the shadow_vqs_enabled hadn't been set back to false yet, causing the installation of call fd inadvertently bypassed. Fixes: a8ac88585da1 ("vhost: Add Shadow VirtQueue call forwarding capabilities") Cc: Eugenio Pérez Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-v

[PATCH 09/40] vdpa: no repeat setting shadow_data

2023-12-07 Thread Si-Wei Liu
space. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index c9bfc6f..2555897 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -387,13 +387,12 @@ static int vhost_vdpa_net_data_start

  1   2   3   4   5   >