[PATCH] migration: Introduce dirtylimit capability

2022-03-03 Thread huangy81
From: Hyman Huang(黄勇) Signed-off-by: Hyman Huang(黄勇) --- migration/migration.c | 19 +++ migration/migration.h | 1 + migration/ram.c | 24 qapi/migration.json | 36 +++- 4 files changed, 75 insertions(+), 5 delet

[PATCH v1] introduce dirty ring size for guestperf tool

2022-03-03 Thread huangy81
From: Hyman Huang(黄勇) v1: -original version sent by accident, send the right patch Dirtylimit implementation has been reviewed in the past few months, if things go well, it will be merged in the near future, which is the first step to implement a new live migration feature. For more details refe

[PATCH v1] tests/migration: Introduce dirty-ring-size option into guestperf

2022-03-03 Thread huangy81
From: Hyman Huang(黄勇) Guestperf tool does not enable diry ring feature when test migration by default. To support dirty ring migration performance test, introduce dirty-ring-size option into guestperf tools, which ranges in [1024, 65536]. To set dirty ring size with 4096 during migration test:

[PATCH v12 0/5] support dirty restraint on vCPU

2022-01-24 Thread huangy81
From: Hyman Huang(黄勇) v12 - rebase on master - add a new commmit to refactor per-vcpu dirty ring reaping, which can resolve the "vcpu miss the chances to sleep" problem - remove the dirtylimit_thread and implemtment throttle in bottom half instead. - let the dirty ring reaper thread keep sleep

[PATCH v12 4/5] softmmu/dirtylimit: implement virtual CPU throttle

2022-01-24 Thread huangy81
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is enabled. Start a thread to track current dirty pa

[PATCH v12 3/5] softmmu/dirtylimit: implement vCPU dirtyrate calculation periodically

2022-01-24 Thread huangy81
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PATCH v12 2/5] migration/dirtyrate: refactor dirty page rate calculation

2022-01-24 Thread huangy81
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PATCH v12 1/5] accel/kvm/kvm-all: refactor per-vcpu dirty ring reaping

2022-01-24 Thread huangy81
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty ring reaping scenario. Only just reaping the target vcpu dirty ring instead of all when vcpu thread return to user space caused by KVM_EXIT_DIRTY_RING_FULL. Signed-off-by:

[PATCH v12 5/5] softmmu/dirtylimit: implement dirty page rate limit

2022-01-24 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PATCH] docs: fix qemu incorrect tag

2021-11-05 Thread huangy81
From: Hyman Huang(黄勇) The last modification of the patchset may be 2 month before merging dirtyrate implementation, it used the wrong tag, so fix with 6.2. Signed-off-by: Hyman Huang(黄勇) --- qapi/migration.json | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/qapi

[PATCH] docs: fix 'sample-pages' option tag

2021-11-07 Thread huangy81
From: Hyman Huang(黄勇) commit f78d4ed701 has fixed qemu tag, making 'sample-pages' option tag involved by accident, which introduced since 6.1 in commit 7afa08cd8fd. revert this line. Signed-off-by: Hyman Huang(黄勇) --- qapi/migration.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) d

[RFC 4/6] migration: Introduce dirtylimit capability

2022-05-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce migration dirtylimit capability, which can be turned on before live migration and limit dirty page rate durty live migration. Dirtylimit dirtylimit capability is kind of like auto-converge but using dirtylimit instead of traditional cpu-throttle to throttle guest

[RFC 3/6] migration: Implement dirtylimit convergence algo

2022-05-16 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtylimit convergence algo for live migration, which is kind of like auto-converge algo but using dirtylimit instead of cpu throttle to make migration convergent. Signed-off-by: Hyman Huang(黄勇) --- migration/ram.c| 53 +++

[RFC 5/6] migration: Add dirtylimit data into migration info

2022-05-16 Thread huangy81
From: Hyman Huang(黄勇) Add dirtylimit throttle data into migration info, through which we can observe the process of dirtylimit during live migration. Signed-off-by: Hyman Huang(黄勇) --- include/sysemu/dirtylimit.h | 2 ++ migration/migration.c | 10 ++ qapi/migration.json

[RFC 1/6] qapi/migration: Introduce vcpu-dirtylimit-period parameters

2022-05-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce "vcpu-dirtylimit-period" migration parameters, which is used to makes dirtyrate calculation period configurable. To implement that, refactor vcpu_dirty_rate_stat_collect so that period can be configured instead of hardcode. Meanwhile, introduce migrate_dirtylimit

[RFC 2/6] qapi/migration: Introduce vcpu-dirtylimit parameters

2022-05-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce "vcpu-dirtylimit" migration parameter used to limit dirty page rate during live migration. "vcpu-dirtylimit" and "vcpu-dirtylimit-period" are two dirtylimit-related migration parameters, which can be set before and during live migration by qmp migrate-set-paramete

[RFC 0/6] migration: introduce dirtylimit capability

2022-05-16 Thread huangy81
From: Hyman Huang(黄勇) Abstract This series added a new migration capability called "dirtylimit". It can be enabled when dirty ring is enabled, and it'll improve the vCPU performance during the process of migration. It is based on the previous patchset: https://lore.kernel.org/qemu-deve

[RFC 6/6] tests: Add migration dirtylimit capability test

2022-05-16 Thread huangy81
From: Hyman Huang(黄勇) Add migration dirtylimit capability test if kernel support dirty ring. Migration dirtylimit capability introduce dirtylimit capability , two parameters: vcpu-dirtylimit-period and vcpu-dirtylimit to implement the live migration with dirtylimit. The test case enable the cap

[PATCH v23 0/8] support dirty restraint on vCPU

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) This is v23 of dirtylimit series. Since v22 posted abount 1 month ago, i did some modifications to make sure it's ready to be queued: - rebased the master and changed the qapi version tag from 7.0 to 7.1 - do not set error if when query_vcpu_dirty_limit find dirtylimit not

[PATCH v23 4/8] softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PATCH v23 6/8] softmmu/dirtylimit: Implement virtual CPU throttle

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is in service. Signed-off-by: Hyman Huang(黄勇) Revie

[PATCH v23 1/8] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) Reviewed-by: Peter Xu --- accel/kvm/kvm-all.c | 23 +-- 1 file changed, 13 insertions(+), 10 del

[PATCH v23 8/8] tests: Add dirty page rate limit test

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Add dirty page rate limit test if kernel support dirty ring, create a standalone file to implement the test case. The following qmp commands are covered by this test case: "calc-dirty-rate", "query-dirty-rate", "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit" and "query-vc

[PATCH v23 5/8] accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Introduce kvm_dirty_ring_size util function to help calculate dirty ring ful time. Signed-off-by: Hyman Huang(黄勇) Acked-by: Peter Xu --- accel/kvm/kvm-all.c| 5 + accel/stubs/kvm-stub.c | 6 ++ include/sysemu/kvm.h | 2 ++ 3 files changed, 13 insertions(+)

[PATCH v23 2/8] cpus: Introduce cpu_list_generation_id

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. cpu_list_generation_id could be used to detect changes of cpu list, which is prepared for dirty page rate measurement. Signed-off

[PATCH v23 3/8] migration/dirtyrate: Refactor dirty page rate calculation

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PATCH v23 7/8] softmmu/dirtylimit: Implement dirty page rate limit

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PATCH v24 0/8] support dirty restraint on vCPU

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) v24: - add "Acked-by: Peter Xu " tag in (PATCH [8/8]) v23: This is v23 of dirtylimit series. Since v22 posted abount 1 month ago, i did some modifications to make sure it's ready to be queued: - rebased the master and changed the qapi version tag from 7.0 to 7.1 - do not

[PATCH v24 1/8] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) Reviewed-by: Peter Xu --- accel/kvm/kvm-all.c | 23 +-- 1 file changed, 13 insertions(+), 10 del

[PATCH v24 2/8] cpus: Introduce cpu_list_generation_id

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. cpu_list_generation_id could be used to detect changes of cpu list, which is prepared for dirty page rate measurement. Signed-off

[PATCH v24 3/8] migration/dirtyrate: Refactor dirty page rate calculation

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PATCH v24 4/8] softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PATCH v24 5/8] accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Introduce kvm_dirty_ring_size util function to help calculate dirty ring ful time. Signed-off-by: Hyman Huang(黄勇) Acked-by: Peter Xu --- accel/kvm/kvm-all.c| 5 + accel/stubs/kvm-stub.c | 6 ++ include/sysemu/kvm.h | 2 ++ 3 files changed, 13 insertions(+)

[PATCH v24 6/8] softmmu/dirtylimit: Implement virtual CPU throttle

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is in service. Signed-off-by: Hyman Huang(黄勇) Revie

[PATCH v24 7/8] softmmu/dirtylimit: Implement dirty page rate limit

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PATCH v24 8/8] tests: Add dirty page rate limit test

2022-05-18 Thread huangy81
From: Hyman Huang(黄勇) Add dirty page rate limit test if kernel support dirty ring. The following qmp commands are covered by this test case: "calc-dirty-rate", "query-dirty-rate", "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit" and "query-vcpu-dirty-limit". Signed-off-by: Hyman Huang(黄勇) Ack

[PATCH v11 2/4] softmmu/dirtylimit: implement vCPU dirtyrate calculation periodically

2022-01-04 Thread huangy81
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PATCH v11 0/4] support dirty restraint on vCPU

2022-01-04 Thread huangy81
From: Hyman Huang(黄勇) v11 - rebase on master - add a commit " refactor dirty page rate calculation" so that dirty page rate limit can reuse the calculation logic. - handle the cpu hotplug/unplug case in the dirty page rate calculation logic. - modify the qmp commands according to Markus's ad

[PATCH v11 1/4] migration/dirtyrate: refactor dirty page rate calculation

2022-01-04 Thread huangy81
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PATCH v11 3/4] softmmu/dirtylimit: implement virtual CPU throttle

2022-01-04 Thread huangy81
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is enabled. Start a thread to track current dirty pa

[PATCH v11 4/4] softmmu/dirtylimit: implement dirty page rate limit

2022-01-04 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PATCH v14 0/7] support dirty restraint on vCPU

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) v14 - v13 sent by accident, resend patchset. v13 - rebase on master - passing NULL to kvm_dirty_ring_reap in commit "refactor per-vcpu dirty ring reaping" to keep the logic unchanged. In other word, we still try the best to reap as much PFNs as possible if dirtylimit

[PATCH v14 4/7] softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PATCH V13 0/7] support dirty restraint on vCPU

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) v13 - rebase on master - passing NULL to kvm_dirty_ring_reap in commit "refactor per-vcpu dirty ring reaping" to keep the logic unchanged. In other word, we still try the best to reap as much PFNs as possible if dirtylimit not in service. - move the cpu list gen id cha

[PATCH V13 2/7] cpus: Introduce cpu_list_generation_id

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. Signed-off-by: Hyman Huang(黄勇) --- cpus-common.c | 8 include/exec/cpu-common.h | 1 + 2 files changed, 9 i

[PATCH v14 6/7] softmmu/dirtylimit: Implement virtual CPU throttle

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is in service. Signed-off-by: Hyman Huang(黄勇) ---

[PATCH v14 3/7] migration/dirtyrate: Refactor dirty page rate calculation

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PATCH v14 5/7] accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Introduce kvm_dirty_ring_size util function to help calculate dirty ring ful time. Signed-off-by: Hyman Huang(黄勇) --- accel/kvm/kvm-all.c| 5 + accel/stubs/kvm-stub.c | 5 + include/sysemu/kvm.h | 2 ++ 3 files changed, 12 insertions(+) diff --git a/accel/k

[PATCH V13 1/7] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) --- accel/kvm/kvm-all.c | 23 +-- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a

[PATCH v14 1/7] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) --- accel/kvm/kvm-all.c | 23 +-- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a

[PATCH v14 2/7] cpus: Introduce cpu_list_generation_id

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. Signed-off-by: Hyman Huang(黄勇) --- cpus-common.c | 8 include/exec/cpu-common.h | 1 + 2 files changed, 9 i

[PATCH v14 7/7] softmmu/dirtylimit: Implement dirty page rate limit

2022-02-10 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PATCH v15 3/7] migration/dirtyrate: Refactor dirty page rate calculation

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PATCH v15 1/7] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) --- accel/kvm/kvm-all.c | 23 +-- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a

[PATCH v15 2/7] cpus: Introduce cpu_list_generation_id

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. cpu_list_generation_id could be used to detect changes of cpu list, which is prepared for dirty page rate measurement. Signed-off

[PATCH v15 5/7] accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) Introduce kvm_dirty_ring_size util function to help calculate dirty ring ful time. Signed-off-by: Hyman Huang(黄勇) --- accel/kvm/kvm-all.c| 5 + accel/stubs/kvm-stub.c | 5 + include/sysemu/kvm.h | 2 ++ 3 files changed, 12 insertions(+) diff --git a/accel/k

[PATCH v15 0/7] support dirty restraint on vCPU

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) v15 - rebase on master - drop the 'init_time_ms' parameter in function vcpu_calculate_dirtyrate - drop the 'setup' field in dirtylimit_state and call dirtylimit_process directly, which makes code cleaner. - code clean in dirtylimit_adjust_throttle - fix miss dirtylimit_st

[PATCH v15 4/7] softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PATCH v15 6/7] softmmu/dirtylimit: Implement virtual CPU throttle

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is in service. Signed-off-by: Hyman Huang(黄勇) ---

[PATCH v15 7/7] softmmu/dirtylimit: Implement dirty page rate limit

2022-02-15 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PATCH v16 0/7] support dirty restraint on vCPU

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) v16 - rebase on master - drop the unused typedef syntax in [PATCH v15 6/7] - add the Reviewed-by and Acked-by tags by the way v15 - rebase on master - drop the 'init_time_ms' parameter in function vcpu_calculate_dirtyrate - drop the 'setup' field in dirtylimit_state and

[PATCH v16 1/7] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) Reviewed-by: Peter Xu --- accel/kvm/kvm-all.c | 23 +-- 1 file changed, 13 insertions(+), 10 del

[PATCH v16 2/7] cpus: Introduce cpu_list_generation_id

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. cpu_list_generation_id could be used to detect changes of cpu list, which is prepared for dirty page rate measurement. Signed-off

[PATCH v16 6/7] softmmu/dirtylimit: Implement virtual CPU throttle

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is in service. Signed-off-by: Hyman Huang(黄勇) Revie

[PATCH v16 5/7] accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce kvm_dirty_ring_size util function to help calculate dirty ring ful time. Signed-off-by: Hyman Huang(黄勇) Acked-by: Peter Xu --- accel/kvm/kvm-all.c| 5 + accel/stubs/kvm-stub.c | 5 + include/sysemu/kvm.h | 2 ++ 3 files changed, 12 insertions(+)

[PATCH v16 3/7] migration/dirtyrate: Refactor dirty page rate calculation

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PATCH v16 4/7] softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PATCH v16 7/7] softmmu/dirtylimit: Implement dirty page rate limit

2022-02-16 Thread huangy81
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PATCH v1 1/8] qapi/migration: Introduce x-vcpu-dirty-limit-period parameter

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Introduce "x-vcpu-dirty-limit-period" migration experimental parameter, which is used to make dirtyrate calculation period configurable. Signed-off-by: Hyman Huang(黄勇) --- migration/migration.c | 16 monitor/hmp-cmds.c| 8 qapi/migration.jso

[PATCH v1 6/8] tests: Add migration dirty-limit capability test

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Add migration dirty-limit capability test if kernel support dirty ring. Migration dirty-limit capability introduce dirty limit capability, two parameters: x-vcpu-dirty-limit-period and x-vcpu-dirty-limit are introduced to implement the live migration with dirty limit. The

[PATCH v1 2/8] qapi/migration: Introduce x-vcpu-dirty-limit parameters

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Introduce "x-vcpu-dirty-limit" migration parameter used to limit dirty page rate during live migration. "x-vcpu-dirty-limit" and "x-vcpu-dirty-limit-period" are two dirty-limit-related migration parameters, which can be set before and during live migration by qmp migrate-se

[PATCH v1 7/8] tests/migration: Introduce dirty-ring-size option into guestperf

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Guestperf tool does not enable diry ring feature when test migration by default. To support dirty ring migration performance test, introduce dirty-ring-size option into guestperf tools, which ranges in [1024, 65536]. To set dirty ring size with 4096 during migration test:

[PATCH v1 5/8] migration: Export dirty-limit time info

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Export dirty limit throttle time and estimated ring full time, through which we can observe the process of dirty limit during live migration. Signed-off-by: Hyman Huang(黄勇) --- include/sysemu/dirtylimit.h | 2 ++ migration/migration.c | 10 ++ monitor/hmp-c

[PATCH v1 4/8] migration: Implement dirty-limit convergence algo

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Implement dirty-limit convergence algo for live migration, which is kind of like auto-converge algo but using dirty-limit instead of cpu throttle to make migration convergent. Signed-off-by: Hyman Huang(黄勇) --- migration/migration.c | 1 + migration/ram.c| 53 ++

[PATCH v1 8/8] tests/migration: Introduce dirty-limit into guestperf

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Guestperf tool does not cover the dirty-limit migration currently, support this feature. To enable dirty-limit, setting x-vcpu-dirty-limit-period as 500ms and x-vcpu-dirty-limit as 10MB/s: $ ./tests/migration/guestperf.py \ --dirty-limit --x-vcpu-dirty-limit-period 500

[PATCH v1 3/8] migration: Introduce dirty-limit capability

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) Introduce migration dirty-limit capability, which can be turned on before live migration and limit dirty page rate durty live migration. Introduce migrate_dirty_limit function to help check if dirty-limit capability enabled during live migration. Meanwhile, refactor vcpu_d

[PATCH v1 0/8] migration: introduce dirtylimit capability

2022-09-01 Thread huangy81
From: Hyman Huang(黄勇) v1: - make parameter vcpu-dirty-limit experimental - switch dirty limit off when cancel migrate - add cancel logic in migration test Please review, thanks, Yong Abstract This series added a new migration capability called "dirtylimit". It can be enabled when

[PATCH v4 02/10] softmmu/dirtylimit: Add parameter check for hmp "set_vcpu_dirty_limit"

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) dirty_rate paraemter of hmp command "set_vcpu_dirty_limit" is invalid if less than 0, so add parameter check for it. Note that this patch also delete the unsolicited help message and clean up the code. Signed-off-by: Hyman Huang(黄勇) Signed-off-by: Markus Armbruster Revie

[PATCH v4 01/10] dirtylimit: Fix overflow when computing MB

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Coverity points out a overflow problem when computing MB, dirty_ring_size and TARGET_PAGE_SIZE are both 32 bits, multiplication will be done as a 32-bit operation, which could overflow. Simplify the formula. Meanwhile, fix spelling mistake of variable name. Reported-by: Pe

[PATCH v4 07/10] migration: Refactor auto-converge capability logic

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Check if block migration is running before throttling guest down in auto-converge way. Note that this modification is kind of like code clean, because block migration does not depend on auto-converge capability, so the order of checks can be adjusted. Signed-off-by: Hyman

[PATCH v4 04/10] qapi/migration: Introduce x-vcpu-dirty-limit-period parameter

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce "x-vcpu-dirty-limit-period" migration experimental parameter, which is in the range of 1 to 1000ms and used to make dirtyrate calculation period configurable. Currently with the "x-vcpu-dirty-limit-period" varies, the total time of live migration changes, test res

[PATCH v4 00/10] migration: introduce dirtylimit capability

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) v4: 1. Polish the docs and update the release version suggested by Markus 2. Rename the migrate exported info "dirty-limit-throttle-time-per-round" to "dirty-limit-throttle-time-per-full". The following 5 commits hasn't been acked or reviewed yet: kvm: dirty-ring: Fix

[PATCH v4 03/10] kvm: dirty-ring: Fix race with vcpu creation

2023-02-16 Thread huangy81
From: Peter Xu It's possible that we want to reap a dirty ring on a vcpu that is during creation, because the vcpu is put onto list (CPU_FOREACH visible) before initialization of the structures. In this case: qemu_init_vcpu x86_cpu_realizefn cpu_exec_realizefn cpu_list_a

[PATCH v4 09/10] migration: Extend query-migrate to provide dirty page limit info

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Extend query-migrate to provide throttle time and estimated ring full time with dirty-limit capability enabled, through which we can observe if dirty limit take effect during live migration. Signed-off-by: Hyman Huang(黄勇) Signed-off-by: Markus Armbruster --- include/syse

[PATCH v4 10/10] tests: Add migration dirty-limit capability test

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Add migration dirty-limit capability test if kernel support dirty ring. Migration dirty-limit capability introduce dirty limit capability, two parameters: x-vcpu-dirty-limit-period and vcpu-dirty-limit are introduced to implement the live migration with dirty limit. The te

[PATCH v4 05/10] qapi/migration: Introduce vcpu-dirty-limit parameters

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce "vcpu-dirty-limit" migration parameter used to limit dirty page rate during live migration. "vcpu-dirty-limit" and "x-vcpu-dirty-limit-period" are two dirty-limit-related migration parameters, which can be set before and during live migration by qmp migrate-set-pa

[PATCH v4 08/10] migration: Implement dirty-limit convergence algo

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Implement dirty-limit convergence algo for live migration, which is kind of like auto-converge algo but using dirty-limit instead of cpu throttle to make migration convergent. Enable dirty page limit if dirty_rate_high_cnt greater than 2 when dirty-limit capability enabled,

[PATCH v4 06/10] migration: Introduce dirty-limit capability

2023-02-16 Thread huangy81
From: Hyman Huang(黄勇) Introduce migration dirty-limit capability, which can be turned on before live migration and limit dirty page rate durty live migration. Introduce migrate_dirty_limit function to help check if dirty-limit capability enabled during live migration. Meanwhile, refactor vcpu_d

[PATCH RFC 0/4] Export netdev capabilities and information

2022-10-31 Thread huangy81
From: Hyman Huang(黄勇) This series is enlightened by Michael when we fixed a virtio features negotiation flaw, see the details here: https://lore.kernel.org/qemu-devel/cover.1667136717.git.huang...@chinatelecom.cn/ And a test is suggested to be added to test behavior of virtio-net features negoti

[PATCH RFC 4/4] vhost-user-test: Add negotiated features check

2022-10-31 Thread huangy81
From: Hyman Huang(黄勇) For vhost-user network device, Qemu backup the final features as acked_features internally after guest acknowledging features during virtio-net driver initialization, so the acked_features could be used as input of VHOST_USER_SET_FEATURES command when slave device restore fr

[PATCH RFC 3/4] hmp: Add netdev information into output of hmp cmd "info network"

2022-10-31 Thread huangy81
From: Hyman Huang(黄勇) Add netdev information into output of hmp command hmp_info_network so developers can analyze interface capability more easily. Signed-off-by: Hyman Huang(黄勇) --- net/net.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/net/net.c b/net/net.c index c27

[PATCH RFC 1/4] net: Introduce qmp cmd "query-netdev"

2022-10-31 Thread huangy81
From: Hyman Huang(黄勇) For netdev device that can offload virtio-net dataplane to slave, such as vhost-net, vhost-user and vhost-vdpa, exporting it's capability information and acked features would be more friendly for developers. These infomation can be analyzed and compare to slave capability pr

[PATCH RFC 2/4] hmp: Add "info netdev" cmd

2022-10-31 Thread huangy81
From: Hyman Huang(黄勇) Introduce "info netdev" command so developers can play with it easier. Signed-off-by: Hyman Huang(黄勇) --- hmp-commands-info.hx | 14 ++ include/monitor/hmp.h | 1 + net/net.c | 31 +++ 3 files changed, 46 insertions(+)

[PATCH 2/2] vhost-net: Fix the virito features negotiation flaw

2022-09-25 Thread huangy81
From: Hyman Huang(黄勇) Save the acked_features once it be configured by guest virtio driver so it can't miss any features. Signed-off-by: Hyman Huang(黄勇) Signed-off-by: Guoyi Tu --- hw/net/vhost_net.c | 9 + hw/net/virtio-net.c | 5 + include/net/vhost_net.h | 2 ++ 3 file

[PATCH 1/2] vhost-user: Refactor vhost acked features saving

2022-09-25 Thread huangy81
From: Hyman Huang(黄勇) Abstract vhost acked features saving into vhost_user_save_acked_features, export it as util function. Signed-off-by: Hyman Huang(黄勇) Signed-off-by: Guoyi Tu --- include/net/vhost-user.h | 2 ++ net/vhost-user.c | 35 +++ 2 files c

[PATCH 0/2] Fix the virito features negotiation flaw

2022-09-25 Thread huangy81
From: "Hyman Huang(黄勇)" This patchset aim to fix the unexpected negotiation features for vhost-user netdev interface. Steps to reproduce the issue: Prepare a vm (CentOS 8 in my work scenario) with vhost-user backend interface and configure qemu as server mode. So dpdk would connect qemu's unix

[PATCH v1 2/2] vhost-net: Fix the virito features negotiation flaw

2022-10-28 Thread huangy81
From: Hyman Huang(黄勇) Save the acked_features once it be configured by guest virtio driver so it can't miss any features. Signed-off-by: Hyman Huang(黄勇) Signed-off-by: Guoyi Tu --- hw/net/vhost_net.c | 9 + hw/net/virtio-net.c | 5 + include/net/vhost_net.h | 2 ++ 3 file

[PATCH v1 1/2] vhost-user: Refactor vhost acked features saving

2022-10-28 Thread huangy81
From: Hyman Huang(黄勇) Abstract vhost acked features saving into vhost_user_save_acked_features, export it as util function. Signed-off-by: Hyman Huang(黄勇) Signed-off-by: Guoyi Tu --- include/net/vhost-user.h | 2 ++ net/vhost-user.c | 35 +++ 2 files c

[PATCH v1 0/2] Fix the virtio features negotiation flaw

2022-10-28 Thread huangy81
From: Hyman Huang(黄勇) v1: This is the version 1 of the series and it is exactly the same as RFC version, but fixing a typo in subject, which is reported by Michael. As for test for the behavior suggested by Michael, IMHO, it could be post in another series, since i found that testing the negoti

[PATCH v2 0/2] Fix the virtio features negotiation flaw

2022-10-28 Thread huangy81
From: Hyman Huang(黄勇) v2: Fix the typo in subject of [PATCH v2 2/2] v1: This is the version 1 of the series and it is exactly the same as RFC version, but fixing a typo in subject, which is reported by Michael. As for test for the behavior suggested by Michael, IMHO, it could be post in anoth

<    1   2   3   4   5   >