Re: [PATCH v6] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-12-02 Thread Chuang Xu
:56:38AM +0800, Chuang Xu wrote: Date: Wed, 9 Oct 2024 11:56:38 +0800 From: Chuang Xu Subject: [PATCH v6] i386/cpu: fixup number of addressable IDs for logical processors in the physical package X-Mailer: git-send-email 2.39.3 (Apple Git-146) When QEMU is started with: -cpu host,migratable=on

Re: [PATCH v6] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-10-12 Thread Chuang Xu
On 10/12/24 下午4:21, Xiaoyao Li wrote: On 10/9/2024 11:56 AM, Chuang Xu wrote: When QEMU is started with: -cpu host,migratable=on,host-cache-info=on,l3-cache=off -smp 180,sockets=2,dies=1,cores=45,threads=2 On Intel platform: CPUID.01H.EBX[23:16] is defined as "max number of addressabl

Re: [PATCH v6] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-10-12 Thread Chuang Xu
Hi, Xiaoyao On 10/12/24 下午3:13, Xiaoyao Li wrote: On 10/9/2024 11:56 AM, Chuang Xu wrote: When QEMU is started with: -cpu host,migratable=on,host-cache-info=on,l3-cache=off -smp 180,sockets=2,dies=1,cores=45,threads=2 On Intel platform: CPUID.01H.EBX[23:16] is defined as "max numb

[PATCH v6] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-10-08 Thread Chuang Xu
eets our expectation. So let us round up CPUID.01H.EBX[23:16] to the nearest power-of-2 integer only for Intel platform to solve the unexpected result. Reviewed-by: Zhao Liu Acked-by: Igor Mammedov Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i

[PATCH v5] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-10-08 Thread Chuang Xu
Acked-by: Igor Mammedov Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i386/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index ff227a8c5c..0749efc52c 100644 --- a/target/i386/cpu.c

Re: [PATCH v4] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-10-08 Thread Chuang Xu
Hi Zhao. On 10/8/24 上午10:55, Zhao Liu wrote: Hi Chuang. Look fine for me, and only some minor nits: On Mon, Oct 07, 2024 at 04:13:44PM +0800, Chuang Xu wrote: Date: Mon, 7 Oct 2024 16:13:44 +0800 From: Chuang Xu Subject: [PATCH v4] i386/cpu: fixup number of addressable IDs for logical

[PATCH v4] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-10-07 Thread Chuang Xu
ore should actually be 2. So let us round up CPUID.01H.EBX[23:16] to the nearest power-of-2 integer to solve the unexpected result. In addition, we introduce max_thread_number_in_package() instead of using pow2ceil() to be compatible with smp and hybrid. Signed-off-by: Guixiong Wei Signed

[PATCH v3] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-09-18 Thread Chuang Xu
ore should actually be 2. So let us round up CPUID.01H.EBX[23:16] to the nearest power-of-2 integer to solve the unexpected result. Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i386/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: [PATCH v2] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-09-18 Thread Chuang Xu
Hi, Igor: On 2024/9/18 下午8:02, Igor Mammedov wrote: On Sat, 14 Sep 2024 19:01:27 +0800 Chuang Xu wrote: When QEMU is started with: -cpu host,migratable=on,host-cache-info=on,l3-cache=off -smp 180,sockets=2,dies=1,cores=45,threads=2 Execute "cpuid -1 -l 1 -r" in guest, we&#

[PATCH v2] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-09-14 Thread Chuang Xu
hen guest will get "90/(1+63)=1" as the result, even though theads-per-core should actually be 2. So let us round up CPUID.01H.EBX[23:16] to the nearest power-of-2 integer to solve the unexpected result. Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu

[PATCH] i386/cpu: fixup number of addressable IDs for logical processors in the physical package

2024-09-11 Thread Chuang Xu
iong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i386/cpu.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 4c2e6f3a71..24d60ead9e 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -261,6 +261,1

[PATCH v3] i386/cpu: fixup number of addressable IDs for processor cores in the physical package

2024-06-10 Thread Chuang Xu
er package case. This is safe because cores_per_pkg will not be 0 and will be at least 1. Fixes: d7caf13b5fcf ("x86: cpu: fixup number of addressable IDs for logical processors sharing cache") Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i386

[PATCH v2] i386/cpu: fixup number of addressable IDs for processor cores in the physical package

2024-06-03 Thread Chuang Xu
the unnecessary condition. Fixes: d7caf13b5fcf742e5680c1d3448ba070fc811644 ("x86: cpu: fixup number of addressable IDs for logical processors sharing cache") Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i386/cpu.c | 6 ++ 1 file ch

Re: [PATCH] x86: cpu: fixup number of addressable IDs for processor cores in the physical package

2024-05-27 Thread Chuang Xu
Hi Zhao, On 2024/5/28 上午10:31, Zhao Liu wrote: Hi Chuang, On Mon, May 27, 2024 at 11:13:33AM +0800, Chuang Xu wrote: Date: Mon, 27 May 2024 11:13:33 +0800 From: Chuang Xu Subject: [PATCH] x86: cpu: fixup number of addressable IDs for processor cores in the physical package According to

[PATCH] x86: cpu: fixup number of addressable IDs for processor cores in the physical package

2024-05-26 Thread Chuang Xu
ommit d7caf13b5fcf742e5680c1d3448ba070fc811644. Fix it by changing the judgement condition to a >= 1. Signed-off-by: Chuang Xu Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin --- target/i386/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index cd

Re: [PATCH v2 04/11] multifd: Count the number of bytes sent correctly

2023-06-16 Thread Chuang Xu
Hi,Juan, On 2023/1/30 下午4:09, Juan Quintela wrote: > Current code asumes that all pages are whole. That is not true for > example for compression already. Fix it for creating a new field > ->sent_bytes that includes it. > > All ram_counters are used only from the migration thread, so we have > t

Re: [PATCH v8 0/6] migration: reduce time of loading non-iterable vmstate

2023-06-16 Thread Chuang Xu
Hi, Paolo, A few months ago, Juan told me that this series requires your or someone familiar with memory API's feedback. Could you please review it and provide some suggestions? On 2023/3/17 下午4:18, Chuang Xu wrote: In this version: - delete useless line change. - update comment

[PATCH v8 3/6] memory: Introduce memory_region_transaction_do_commit()

2023-03-17 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 inser

[PATCH v8 0/6] migration: reduce time of loading non-iterable vmstate

2023-03-17 Thread Chuang Xu
In this version: - delete useless line change. - update comments and commit messages. The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the tim

[PATCH v8 2/6] rcu: Introduce rcu_read_is_locked()

2023-03-17 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index 313fc414bc..7bf45602e1 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH v8 6/6] memory: Introduce address_space_to_flatview_rcu()

2023-03-17 Thread Chuang Xu
In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu --- include/exec/memory.h | 17 + softmmu/memory.c | 2 +- 2 files changed, 18 insertions(+), 1

[PATCH v8 4/6] memory: Add do_commit() and sanity check in address_space_to_flatview

2023-03-17 Thread Chuang Xu
address_space_to_flatview will return the newest flatview and it should logically only be triggered in a few spots during vm_load. Other than that, sanity check whether BQL or rcu is held before using any flatview. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu

[PATCH v8 1/6] memory: Reference as->current_map directly in memory commit

2023-03-17 Thread Chuang Xu
From: Peter Xu Calling RCU variants of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH v8 5/6] migration: Reduce time of loading non-iterable vmstate

2023-03-17 Thread Chuang Xu
number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 18

Re: [PATCH v7 6/6] memory: Introduce address_space_to_flatview_rcu()

2023-03-13 Thread Chuang Xu
Hi, Peter, On 2023/3/10 下午11:08, Peter Xu wrote: On Fri, Mar 10, 2023 at 10:24:25AM +0800, Chuang Xu wrote: In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu

Re: [PATCH v7 3/6] memory: Introduce memory_region_transaction_do_commit()

2023-03-12 Thread Chuang Xu
Hi, Peter, On 2023/3/10 下午10:51, Peter Xu wrote: On Fri, Mar 10, 2023 at 10:24:22AM +0800, Chuang Xu wrote: Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chua

[PATCH v7 6/6] memory: Introduce address_space_to_flatview_rcu()

2023-03-09 Thread Chuang Xu
In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu --- include/exec/memory-internal.h | 2 +- include/exec/memory.h | 20 softmmu/memory.c

[PATCH v7 4/6] memory: Add sanity check in address_space_to_flatview

2023-03-09 Thread Chuang Xu
Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu/memory.c

[PATCH v7 2/6] rcu: Introduce rcu_read_is_locked()

2023-03-09 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index 313fc414bc..7bf45602e1 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH v7 3/6] memory: Introduce memory_region_transaction_do_commit()

2023-03-09 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 inser

[PATCH v7 1/6] memory: Reference as->current_map directly in memory commit

2023-03-09 Thread Chuang Xu
From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH v7 5/6] migration: Reduce time of loading non-iterable vmstate

2023-03-09 Thread Chuang Xu
number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 19

[PATCH v7 0/6] migration: reduce time of loading non-iterable vmstate

2023-03-09 Thread Chuang Xu
In this version: - introduce address_space_to_flatview_rcu() - squash peter's fix into patch 1 - rebase to latest upstream - update test results The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and endin

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-09 Thread Chuang Xu
Hi, Peter, On 2023/3/8 下午11:46, Peter Xu wrote: 1. squash fix into patch1 of yours. 2. introduce address_space_to_flatview_rcu() 3. add specific comment to define when to use which as_to_flat() This can be together with 2). We should suggest using address_space_to_flatview() by default in the

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-08 Thread Chuang Xu
Hi, Peter, On 2023/3/8 下午10:58, Peter Xu wrote: On Wed, Mar 08, 2023 at 06:03:45AM -0800, Chuang Xu wrote: IIUC, Do you mean that different ways to get flatview are tricky? Yes, and properly define when to use which. As you said, it's slightly beyond what this series does. Maybe it wou

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-08 Thread Chuang Xu
Hi, Peter, On 2023/3/8 上午1:04, Peter Xu wrote: > On Tue, Mar 07, 2023 at 09:24:31PM +0800, Chuang Xu wrote: >>> Why do we need address_space_get_flatview_rcu()? I'm not sure whether you >> address_space_cahce_init() uses address_space_get_flatview() to acquire >> a r

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-07 Thread Chuang Xu
Hi, Peter, On 2023/3/7 上午4:48, Peter Xu wrote: On Mon, Mar 06, 2023 at 08:48:05PM +0800, Chuang Xu wrote: Hi, Peter, On 2023/3/6 上午6:05, Peter Xu wrote: 1.virtio_load->virtio_init_region_cache 2.virtio_load->virtio_set_features_nocheck What is this one specifically? I failed to see q

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-06 Thread Chuang Xu
Hi, Peter, On 2023/3/6 上午6:05, Peter Xu wrote: Hi, Chuang, On Fri, Mar 03, 2023 at 06:56:50PM +0800, Chuang Xu wrote: Sorry to forget to update the test results in the last patch of v6. In this version: - add peter's patch. - split mr_do_commit() from mr_commit(). - adjust the sanity

[PATCH RESEND v6 5/5] migration: Reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index b5e6962bb6..3dd9daabd8 100644

[PATCH RESEND v6 3/5] memory: Introduce memory_region_transaction_do_commit()

2023-03-03 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 inser

[PATCH RESEND v6 1/5] memory: Reference as->current_map directly in memory commit

2023-03-03 Thread Chuang Xu
From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
Sorry to forget to update the test results in the last patch of v6. In this version: - add peter's patch. - split mr_do_commit() from mr_commit(). - adjust the sanity check in address_space_to_flatview(). - rebase to latest upstream. - replace 8260 with 8362 as testing host. - update the lates

[PATCH RESEND v6 2/5] rcu: Introduce rcu_read_is_locked()

2023-03-03 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH RESEND v6 4/5] memory: Add sanity check in address_space_to_flatview

2023-03-03 Thread Chuang Xu
Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu/memory.c

[PATCH v6 1/5] memory: Reference as->current_map directly in memory commit

2023-03-03 Thread Chuang Xu
From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH v6 2/5] rcu: Introduce rcu_read_is_locked()

2023-03-03 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
In this version: - add peter's patch. - split mr_do_commit() from mr_commit(). - adjust the sanity check in address_space_to_flatview(). - rebase to latest upstream. - replace 8260 with 8362 as testing host. - update the latest test results. Here I list some cases which will trigger do_commit(

[PATCH v6 5/5] migration: Reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index b5e6962bb6..3dd9daabd8 100644 --- a

[PATCH v6 4/5] memory: Add sanity check in address_space_to_flatview

2023-03-03 Thread Chuang Xu
Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu/memory.c

[PATCH v6 3/5] memory: Introduce memory_region_transaction_do_commit()

2023-03-03 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 inser

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-27 Thread Chuang Xu
Hi, Peter On 2023/2/25 下午11:32, Peter Xu wrote: On Thu, Feb 23, 2023 at 11:28:46AM +0800, Chuang Xu wrote: Hi, Peter Hi, Chuang, On 2023/2/22 下午11:57, Peter Xu wrote: I don't see why it's wrong, and that's exactly what I wanted to guarantee, that if memory_region_update_pen

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-22 Thread Chuang Xu
Hi, Peter On 2023/2/22 下午11:57, Peter Xu wrote: On Wed, Feb 22, 2023 at 02:27:55PM +0800, Chuang Xu wrote: Hi, Peter Hi, Chuang, Note that as I mentioned in the comment, we temporarily replace this value to prevent commit() and address_space_to_flatview() call each other recursively, and

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-21 Thread Chuang Xu
Hi, Peter On 2023/2/22 上午4:36, Peter Xu wrote: On 2023/2/21 上午11:38, Chuang Xu wrote: I think we need a memory_region_transaction_commit_force() to force commit some transactions when load vmstate. This function is designed like this: /*  * memory_region_transaction_commit_force() is

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-21 Thread Chuang Xu
Hi, Peter This email is a supplement to the previous one. On 2023/2/21 上午11:38, Chuang Xu wrote: I think we need a memory_region_transaction_commit_force() to force commit some transactions when load vmstate. This function is designed like this: /*  * memory_region_transaction_commit_force

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-20 Thread Chuang Xu
Hi, Peter It seems that there is a problem with the code format in my last email. I adjusted the format and resend this to you. Hope the format of this email won't be wrong again..  :) On 2023/2/17 下午11:52, Peter Xu wrote: Hello, Chuang, On Fri, Feb 17, 2023 at 04:11:19PM +0800, Chua

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-20 Thread Chuang Xu
Hi, Peter On 2023/2/17 下午11:52, Peter Xu wrote: Hello, Chuang, On Fri, Feb 17, 2023 at 04:11:19PM +0800, Chuang Xu wrote: Error 1 was triggered by our sanity check. I try to add RCU_READ_LOCK_GUARD() in address_space_init() and it works. But I'm not sure if this code change is appropriat

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-20 Thread Chuang Xu
Hi, Juan On 2023/2/16 上午3:10, Juan Quintela wrote: Chuang Xu wrote: In this version: Hi I had to drop this. It breaks migration of dbus-vmstate. .[K144/179 qemu:qtest+qtest-x86_64 / qtest-x86_64/virtio-net-failover ERROR 5.66s killed by signal 6 SIGABRT

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-17 Thread Chuang Xu
Hi, Peter! In my last email to Juan, I mentioned two errors. Now I want to discuss them with you. On 2023/2/16 下午11:41, Chuang Xu wrote: I ran qtest with reference to your environment, and finally reported two errors. Error 1(the same as yours):  QTEST_QEMU_BINARY=./qemu-system-x86_64

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-16 Thread Chuang Xu
Hi, Juan Thanks for your test results! On 2023/2/16 上午3:10, Juan Quintela wrote: > Chuang Xu wrote: >> In this version: > Hi > > I had to drop this. It breaks migration of dbus-vmstate. Previously, I only focused on the precopy migration test in the normal scenario, but did n

Re: [RFC v5 1/3] rcu: introduce rcu_read_is_locked()

2023-02-14 Thread Chuang Xu
Hi, Paolo! On 2023/2/2 下午6:59, Juan Quintela wrote: Chuang Xu wrote: add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu Reviewed-by: Juan Quintela Althought I think that petting a review from Paolo or anyone that knows more than RCU could be a good idea

[RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-01-17 Thread Chuang Xu
In this version: - rename rcu_read_locked() to rcu_read_is_locked(). - adjust the sanity check in address_space_to_flatview(). - improve some comments. The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and

[RFC v5 3/3] migration: reduce time of loading non-iterable vmstate

2023-01-17 Thread Chuang Xu
commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 18 ++ 1 file changed, 18 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..8ca6d396f4 100644 --- a

[RFC v5 1/3] rcu: introduce rcu_read_is_locked()

2023-01-17 Thread Chuang Xu
add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[RFC v5 2/3] memory: add depth assert in address_space_to_flatview

2023-01-17 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 15 +++ softmmu/memory.c | 5 + 2 files changed, 20 insertions(+) diff --git a/include/exec/memor

Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-13 Thread Chuang Xu
Hi, Peter, On 2023/1/12 下午11:13, Peter Xu wrote: We wanted to capture outliers when you apply the follow up patch to vm load procedure. That will make depth>0 for the whole process of vm load during migration, and we wanted to make sure it's safe, hence this patch, right? In my perspective, b

Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-12 Thread Chuang Xu
Hi, Peter, Paolo, On 2023/1/10 下午10:45, Peter Xu wrote: On Tue, Jan 10, 2023 at 12:09:41AM -0800, Chuang Xu wrote: Hi, Peter and Paolo, Hi, Chuang, Paolo, I'm sorry I didn't reply to your email in time. I was infected with COVID-19 two weeks ago, so I couldn't think abo

Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-10 Thread Chuang Xu
Hi, Peter and Paolo, I'm sorry I didn't reply to your email in time. I was infected with COVID-19 two weeks ago, so I couldn't think about the problems discussed in your email for a long time.😷 On 2023/1/4 上午1:43, Peter Xu wrote: > Hi, Paolo, > > On Wed, Dec 28, 2022 at 09:27:50AM +0100, Paolo Bon

Re: [RFC v4 1/3] rcu: introduce rcu_read_locked()

2023-01-05 Thread Chuang Xu
On 2023/1/4 下午10:20, Alex Bennée wrote: > Chuang Xu writes: > >> add rcu_read_locked() to detect holding of rcu lock. >> >> Signed-off-by: Chuang Xu >> --- >> include/qemu/rcu.h | 7 +++ >> 1 file changed, 7 insertions(+) >> >> diff --

Re: [External] Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-03 Thread Chuang Xu
On 2022/12/28 下午6:50, Philippe Mathieu-Daudé wrote: On 23/12/22 15:23, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/mem

Re: [RFC v4 3/3] migration: reduce time of loading non-iterable vmstate

2023-01-03 Thread Chuang Xu
On 2022/12/24 上午12:06, David Hildenbrand wrote: On 23.12.22 15:23, Chuang Xu wrote: The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time

Re: [RFC v4 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-23 Thread Chuang Xu
On 2022/12/23 下午11:50, Peter Xu wrote: Chuang, On Fri, Dec 23, 2022 at 10:23:04PM +0800, Chuang Xu wrote: In this version: - attach more information in the cover letter. - remove changes on virtio_load(). - add rcu_read_locked() to detect holding of rcu lock. The duration of loading non

[RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2022-12-23 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 5 + 2 files changed, 14 insertions(+) diff --git a/include/exec/memory.h b/include

[RFC v4 1/3] rcu: introduce rcu_read_locked()

2022-12-23 Thread Chuang Xu
add rcu_read_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..42cbd0080f 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -119,6

[RFC v4 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-23 Thread Chuang Xu
In this version: - attach more information in the cover letter. - remove changes on virtio_load(). - add rcu_read_locked() to detect holding of rcu lock. The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop an

[RFC v4 3/3] migration: reduce time of loading non-iterable vmstate

2022-12-23 Thread Chuang Xu
commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..19785e5a54 100644 --- a/migration

Re: [RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-20 Thread Chuang Xu
On 2022/12/16 上午12:04, Peter Xu wrote: On Wed, Dec 14, 2022 at 04:38:52PM -0500, Peter Xu wrote: On Wed, Dec 14, 2022 at 08:03:38AM -0800, Chuang Xu wrote: On 2022/12/13 下午9:35, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the ma

Re: [RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-20 Thread Chuang Xu
On 2022/12/16 上午12:51, Peter Maydell wrote: On Tue, 13 Dec 2022 at 13:36, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/mem

Re: [RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-14 Thread Chuang Xu
On 2022/12/13 下午9:35, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 del

Re: [RFC v3 2/3] virtio: support delay of checks in virtio_load()

2022-12-14 Thread Chuang Xu
On 2022/12/14 上午12:31, Peter Xu wrote: On Tue, Dec 13, 2022 at 09:35:09PM +0800, Chuang Xu wrote: Delay checks in virtio_load() to avoid possible address_space_to_flatview() call during memory region's begin/commit. I didn't notice virtio has the vm change handler already, looks goo

[RFC v3 3/3] migration: reduce time of loading non-iterable vmstate

2022-12-13 Thread Chuang Xu
device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/savevm.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..19785e5a54 100644

[RFC v3 2/3] virtio: support delay of checks in virtio_load()

2022-12-13 Thread Chuang Xu
Delay checks in virtio_load() to avoid possible address_space_to_flatview() call during memory region's begin/commit. Signed-off-by: Chuang Xu --- hw/virtio/virtio.c | 37 +++-- include/hw/virtio/virtio.h | 2 ++ 2 files changed, 29 insertions(+

[RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-13 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/exec/memor

[RFC v3 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-13 Thread Chuang Xu
Hi! In this version: - move virtio_load_check_delay() from virtio_memory_listener_commit() to virtio_vmstate_change(). - add delay_check flag to VirtIODevice to make sure virtio_load_check_delay() will be called when delay_check is true. Please review, Chuang. [v2] - rebase to latest ups

Re: [RFC v2 2/3] virtio: support delay of checks in virtio_load()

2022-12-13 Thread Chuang Xu
On 2022/12/13 上午4:18, Peter Xu wrote: On Tue, Dec 13, 2022 at 12:49:41AM +0800, Chuang Xu wrote: +bool migration_enable_load_check_delay; I'm just afraid this is still too hacky. One thing is because this variable itself to be only set at specific phase during migration to cover that c

Re: [RFC v2 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-13 Thread Chuang Xu
On 2022/12/13 上午4:23, Peter Xu wrote: On Tue, Dec 13, 2022 at 12:49:39AM +0800, Chuang Xu wrote: Hi! Chuang, In this version: - rebase to latest upstream. - add sanity check to address_space_to_flatview(). - postpone the init of the vring cache until migration's loading completes.

[RFC v2 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-12 Thread Chuang Xu
Hi! In this version: - rebase to latest upstream. - add sanity check to address_space_to_flatview(). - postpone the init of the vring cache until migration's loading completes. Please review, Chuang. [v1] The duration of loading non-iterable vmstate accounts for a significant portion of dow

[RFC v2 3/3] migration: reduce time of loading non-iterable vmstate

2022-12-12 Thread Chuang Xu
device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/savevm.c | 13 + 1 file changed, 13 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..68a7a99b79

[RFC v2 1/3] memory: add depth assert in address_space_to_flatview

2022-12-12 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/exec/memor

[RFC v2 2/3] virtio: support delay of checks in virtio_load()

2022-12-12 Thread Chuang Xu
Delay checks in virtio_load() to avoid possible address_space_to_flatview() call during memory region's begin/commit. Signed-off-by: Chuang Xu --- hw/virtio/virtio.c | 33 ++--- include/sysemu/sysemu.h | 1 + softmmu/globals.c | 3 +++ 3 files ch

Re: [External] Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-09 Thread Chuang Xu
Hi, Peter! This email is a supplement to my previous email 7 hours ago. On 2022/12/9 上午12:00, Peter Xu wrote: If the assert will work that'll be even better. I'm actually worried this can trigger like what you mentioned in the virtio path, I didn't expect it comes that soon. So if there's a mi

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-08 Thread Chuang Xu
On 2022/12/9 上午12:00, Peter Xu wrote: On Thu, Dec 08, 2022 at 10:39:11PM +0800, Chuang Xu wrote: On 2022/12/8 上午6:08, Peter Xu wrote: On Thu, Dec 08, 2022 at 12:07:03AM +0800, Chuang Xu wrote: On 2022/12/6 上午12:28, Peter Xu wrote: Chuang, No worry on the delay; you're faster than when I

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-08 Thread Chuang Xu
On 2022/12/8 上午6:08, Peter Xu wrote: On Thu, Dec 08, 2022 at 12:07:03AM +0800, Chuang Xu wrote: On 2022/12/6 上午12:28, Peter Xu wrote: Chuang, No worry on the delay; you're faster than when I read yours. :) On Mon, Dec 05, 2022 at 02:56:15PM +0800, Chuang Xu wrote: As a start, maybe yo

Re: [External] Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-07 Thread Chuang Xu
On 2022/12/6 上午12:28, Peter Xu wrote: Chuang, No worry on the delay; you're faster than when I read yours. :) On Mon, Dec 05, 2022 at 02:56:15PM +0800, Chuang Xu wrote: As a start, maybe you can try with poison address_space_to_flatview() (by e.g. checking the start_pack_mr_change fla

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-04 Thread Chuang Xu
Peter, I'm sorry I didn't reply to your email in time, because I was busy with other work last week. Here is my latest progress. On 2022/11/29 上午1:41, Peter Xu wrote: On Mon, Nov 28, 2022 at 05:42:43PM +0800, Chuang Xu wrote: On 2022/11/25 上午12:40, Peter Xu wrote: On Fri, Nov 18,

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-11-28 Thread Chuang Xu
On 2022/11/25 上午12:40, Peter Xu wrote: On Fri, Nov 18, 2022 at 04:36:48PM +0800, Chuang Xu wrote: The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start

[RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-11-18 Thread Chuang Xu
device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/migration.c | 1 + migration/migration.h | 2 ++ migration/savevm.c| 8 3 files changed, 11 insertions(+) diff --git a/migration

Re: [PATCH v7 10/12] multifd: Support for zero pages transmission

2022-10-25 Thread chuang xu
On 2022/8/2 下午2:39, Juan Quintela wrote: This patch adds counters and similar. Logic will be added on the following patch. Signed-off-by: Juan Quintela --- Added counters for duplicated/non duplicated pages. Removed reviewed by from David. Add total_zero_pages --- migration/multifd.h

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-15 Thread chuang xu
endmsg again when the errno is ENOBUFS and set optmem to the initial 20KB(echo 20480 > /proc/sys/net/core/optmem_max), now the multifd zero_copy migration goes well. Here are the changes I made to the code: Signed-off-by: chuang xu ---  io/channel-socket.c | 4 +---  1 file changed, 1 inserti

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-14 Thread chuang xu
re/optmem_max) as the RFC says.Then I tested the multifd zero_copy migration repeatedly,and the error disappeared. So when sendmsg returns -1 with errno ENOBUFS, should we distinguish between error ''socket exceeds optmem limit" and error "user exceeds ulimit on locked pages"? Or is there any better way to avoid this problem? Best Regards, chuang xu

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-07 Thread chuang xu
On 2022/6/8 下午1:24, Leonardo Bras Soares Passos wrote: I will send a fix shortly. Is that ok if I include a "Reported-by: 徐闯 " in the patch? okay. Best Regards, chuang xu

  1   2   >