[Qemu-devel] [PATCH V4 05/16] COLO: Add block replication into colo process

2018-01-19 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- migration/colo.c

[Qemu-devel] [PATCH V4 04/16] COLO: integrate colo compare with colo frame

2018-01-19 Thread Zhang Chen
-by: zhanghailiang Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 -- migration/migration.c | 4 ++-- 2 files changed, 42 insertions(+), 4 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index

[Qemu-devel] [PATCH V4 08/16] ram/COLO: Record the dirty pages that SVM received

2018-01-19 Thread Zhang Chen
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the

[Qemu-devel] [PATCH V4 14/16] filter-rewriter: handle checkpoint and failover event

2018-01-19 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen --- migration/colo.c | 13 + net/filter-rewriter.c | 40 2 files changed, 53 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index a931ff2..9eab4a3 100644 --- a/migration/colo.c +++ b

[Qemu-devel] [PATCH V4 07/16] COLO: Load dirty pages into SVM's RAM cache firstly

2018-01-19 Thread Zhang Chen
eive all PVM's state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- include/exec/ram_addr.h | 1 + migration/migration.c | 2 + migration/ram.c | 97 +++-- migration/ram.h | 4 ++ mig

[Qemu-devel] [PATCH V4 09/16] COLO: Flush memory data from ram cache

2018-01-19 Thread Zhang Chen
From: zhanghailiang During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all

[Qemu-devel] [PATCH V4 12/16] COLO: flush host dirty ram from cache

2018-01-19 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- migration/ram.c | 12 1 file changed, 12 insertions(+) diff --git a

[Qemu-devel] [PATCH V4 10/16] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-01-19 Thread Zhang Chen
x27;s failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Eric Blake --- migration/colo.c| 19 +++ qapi/migration.json | 35

[Qemu-devel] [PATCH V4 13/16] filter: Add handle_event method for NetFilterClass

2018-01-19 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 28 3 files changed, 50 insertions(+)

[Qemu-devel] [PATCH V4 11/16] savevm: split the process of different stages for loadvm/savevm

2018-01-19 Thread Zhang Chen
_main() and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PATCH V4 15/16] COLO: notify net filters about checkpoint/failover event

2018-01-19 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang --- migration/colo.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 9eab4a3..10bc80c 100644 --- a/migration/colo.c +++ b

[Qemu-devel] [PATCH V4 16/16] COLO: quick failover process by kick COLO thread

2018-01-19 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang --- migration/colo.c | 8 1 file changed, 8 insertions(+) diff --git a/migration/colo.c b/migra

Re: [Qemu-devel] Subject: [PATCH]COLO: Fix spell error in Colo doc

2017-03-20 Thread Zhang Chen
iki.qemu-project.org/Features/COLO. So I think this patch is not necessary, I will update new command in docs/COLO-FT.txt later. Please next time use scripts/get_maintainer.pl to cc related maintainers. Thanks Zhang Chen On 03/21/2017 11:01 AM, Eric Blake wrote: On 03/20/2017 09:49 PM, wanggua

Re: [Qemu-devel] [PATCH v4]COLO:Fix Colo doc, secondeary should be secondary

2017-03-20 Thread Zhang Chen
send-email' working properly. I agree. -- Thanks Zhang Chen

Re: [Qemu-devel] [PATCH 0/3] Add COLO-proxy virtio-net support

2017-03-20 Thread Zhang Chen
On 03/21/2017 11:39 AM, Jason Wang wrote: On 2017年03月16日 17:52, Zhang Chen wrote: If user use -device virtio-net-pci, virtio-net driver will add a header to raw net packet that colo-proxy can't handle it. COLO-proxy just focus on the packet payload, so we skip the virtio-net head

Re: [Qemu-devel] [PATCH 0/3] Add COLO-proxy virtio-net support

2017-03-20 Thread Zhang Chen
On 03/21/2017 02:10 PM, Jason Wang wrote: On 2017年03月21日 13:47, Zhang Chen wrote: On 03/21/2017 11:39 AM, Jason Wang wrote: On 2017年03月16日 17:52, Zhang Chen wrote: If user use -device virtio-net-pci, virtio-net driver will add a header to raw net packet that colo-proxy can't h

Re: [Qemu-devel] [PATCH 0/3] Add COLO-proxy virtio-net support

2017-03-21 Thread Zhang Chen
On 03/21/2017 02:30 PM, Jason Wang wrote: On 2017年03月21日 14:16, Zhang Chen wrote: On 03/21/2017 02:10 PM, Jason Wang wrote: On 2017年03月21日 13:47, Zhang Chen wrote: On 03/21/2017 11:39 AM, Jason Wang wrote: On 2017年03月16日 17:52, Zhang Chen wrote: If user use -device virtio-net-pci

Re: [Qemu-devel] 答复: Re: [BUG]COLO failover hang

2017-03-21 Thread Zhang Chen
Hi,Wang. You can test this branch: https://github.com/coloft/qemu/tree/colo-v5.1-developing-COLO-frame-v21-with-shared-disk and please follow wiki ensure your own configuration correctly. http://wiki.qemu-project.org/Features/COLO Thanks Zhang Chen On 03/21/2017 03:27 PM, wang.guan

Re: [Qemu-devel] [PATCH 0/3] Add COLO-proxy virtio-net support

2017-03-21 Thread Zhang Chen
On 03/21/2017 05:15 PM, Jason Wang wrote: On 2017年03月21日 15:08, Zhang Chen wrote: On 03/21/2017 02:30 PM, Jason Wang wrote: On 2017年03月21日 14:16, Zhang Chen wrote: On 03/21/2017 02:10 PM, Jason Wang wrote: On 2017年03月21日 13:47, Zhang Chen wrote: On 03/21/2017 11:39 AM, Jason

Re: [Qemu-devel] [PATCH 0/3] Add COLO-proxy virtio-net support

2017-03-21 Thread Zhang Chen
On 03/22/2017 11:13 AM, Jason Wang wrote: On 2017年03月22日 09:50, Zhang Chen wrote: On 03/21/2017 05:15 PM, Jason Wang wrote: On 2017年03月21日 15:08, Zhang Chen wrote: On 03/21/2017 02:30 PM, Jason Wang wrote: On 2017年03月21日 14:16, Zhang Chen wrote: On 03/21/2017 02:10 PM, Jason

Re: [Qemu-devel] [PATCH for-4.0 v4 5/7] colo: check chardev can switch context

2018-12-05 Thread Zhang Chen
On Thu, Dec 6, 2018 at 4:38 AM Marc-André Lureau < marcandre.lur...@redhat.com> wrote: > COLO uses a worker context (iothread) to drive the chardev. All > backends are not able to switch the context, let's report an error in > this case. > > Signed-off-by: Marc-André Lur

Re: [Qemu-devel] [PATCH V2] migration/colo.c: Fix compilation issue when disable replication

2018-11-05 Thread Zhang Chen
On Mon, Nov 5, 2018 at 6:02 PM Thomas Huth wrote: > On 2018-11-01 03:12, Zhang Chen wrote: > > This compilation issue will occur when user use --disable-replication > > to config Qemu. > > > > Reported-by: Thomas Huth > > Signed-off-by: Zhang Chen &

[Qemu-devel] [PATCH] migration/migration.c: Add COLO dependency checks

2018-11-14 Thread Zhang Chen
From: Zhang Chen Current COLO mode(independent disk mode) need replication module work together. Suggested by Dr. David Alan Gilbert . Signed-off-by: Zhang Chen --- migration/migration.c | 9 + 1 file changed, 9 insertions(+) diff --git a/migration/migration.c b/migration/migration.c

Re: [Qemu-devel] [PATCH V2] migration/colo.c: Fix compilation issue when disable replication

2018-11-14 Thread Zhang Chen
On Wed, Nov 14, 2018 at 7:17 PM Peter Maydell wrote: > On 14 November 2018 at 11:06, Thomas Huth wrote: > > On 2018-11-14 11:47, Peter Xu wrote: > >> On Thu, Nov 01, 2018 at 10:12:26AM +0800, Zhang Chen wrote: > >>> This compilation issue will occur when user use

Re: [Qemu-devel] [PATCH V2] migration/colo.c: Fix compilation issue when disable replication

2018-11-17 Thread Zhang Chen
On Sat, Nov 17, 2018, 2:29 AM Dr. David Alan Gilbert * Zhang Chen (zhangc...@gmail.com) wrote: > > On Wed, Nov 14, 2018 at 7:17 PM Peter Maydell > > wrote: > > > > > On 14 November 2018 at 11:06, Thomas Huth wrote: > > > > On 2018-11-14 11:47, Peter Xu wro

[Qemu-devel] [PATCH] qapi: Fix COLOStatus and query-colo-status since version

2018-10-22 Thread Zhang Chen
This structure and command have missed qemu version 3.0, so fix it to since version 3.1. Signed-off-by: Zhang Chen --- qapi/migration.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 0928f4b727..38d4c41d88 100644 --- a

Re: [Qemu-devel] [PATCH V12 05/19] COLO: Add block replication into colo process

2018-10-24 Thread Zhang Chen
On Wed, Oct 24, 2018 at 3:21 PM Thomas Huth wrote: > On 2018-09-03 05:38, Zhang Chen wrote: > > Make sure master start block replication after slave's block > > replication started. > > > > Besides, we need to activate VM's blocks before goes into

[Qemu-devel] [PATCH] migration/colo.c: Fix compilation issue when disable replication

2018-10-24 Thread Zhang Chen
This compilation issue will occur when user use --disable-replication to config Qemu. Reported-by: Thomas Huth Signed-off-by: Zhang Chen --- migration/colo.c | 18 +- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c index

Re: [Qemu-devel] [PULL V2 01/26] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-10-29 Thread Zhang Chen
On Tue, Oct 30, 2018 at 10:02 AM Jason Wang wrote: > > On 2018/10/29 下午7:01, Peter Maydell wrote: > > On 19 October 2018 at 04:22, Jason Wang wrote: > >> From: Zhang Chen > >> > >> We add almost full TCP state machine in filter-rewriter, except > >

[Qemu-devel] [PATCH] net/filter-rewirter.c: Fix coverity static analysis issue

2018-10-29 Thread Zhang Chen
The original code just follow the TCP state diagram, but in this case, we can skip the TCPS_TIME_WAIT state to simplify the implementation. Signed-off-by: Zhang Chen --- net/filter-rewriter.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/filter-rewriter.c b/net

Re: [Qemu-devel] [PATCH] net/filter-rewirter.c: Fix coverity static analysis issue

2018-10-30 Thread Zhang Chen
Oh, Sorry. I will send the V2 patch. Thank Zhang Chen On Wed, Oct 31, 2018 at 12:18 AM Peter Maydell wrote: > On 30 October 2018 at 03:50, Zhang Chen wrote: > > The original code just follow the TCP state diagram, > > but in this case, we can skip the TCPS_TIME_WAIT state to s

[Qemu-devel] [PATCH V2] net/filter-rewriter.c: Fix coverity static analysis issue

2018-10-30 Thread Zhang Chen
The original code just follow the TCP state diagram, but in this case, we can skip the TCPS_TIME_WAIT state to simplify the implementation. Signed-off-by: Zhang Chen --- net/filter-rewriter.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/filter-rewriter.c b/net

Re: [Qemu-devel] [PATCH] migration/colo.c: Fix compilation issue when disable replication

2018-10-31 Thread Zhang Chen
On Thu, Nov 1, 2018 at 12:47 AM Dr. David Alan Gilbert wrote: > * Dr. David Alan Gilbert (dgilb...@redhat.com) wrote: > > * Zhang Chen (zhangc...@gmail.com) wrote: > > > This compilation issue will occur when user use --disable-replication > > > to config Qemu. >

Re: [Qemu-devel] [PATCH] migration/colo.c: Fix compilation issue when disable replication

2018-10-31 Thread Zhang Chen
On Wed, Oct 31, 2018 at 3:54 AM Dr. David Alan Gilbert wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > This compilation issue will occur when user use --disable-replication > > to config Qemu. > > > > Reported-by: Thomas Huth > > Signed-off-by: Zhang

[Qemu-devel] [PATCH V2] migration/colo.c: Fix compilation issue when disable replication

2018-10-31 Thread Zhang Chen
This compilation issue will occur when user use --disable-replication to config Qemu. Reported-by: Thomas Huth Signed-off-by: Zhang Chen --- migration/colo.c | 28 +++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c

Re: [Qemu-devel] [PATCH] docs/block-replication.txt: Add more detail about replication_do_checkpoint_all

2018-11-02 Thread Zhang Chen
Hi All, Maybe we have forgot this patch? Ping again. Related discussion: https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg03352.html Thanks Zhang Chen On Fri, Sep 14, 2018 at 2:32 PM Zhang Chen wrote: > Hi All, > > No news update? > Ping... > > Thanks > Zhang C

Re: [Qemu-devel] [PATCH] migration/migration.c: Add COLO dependency checks

2018-11-21 Thread Zhang Chen
On Thu, Nov 15, 2018 at 11:55 AM Peter Xu wrote: > On Thu, Nov 15, 2018 at 03:09:12AM +0800, Zhang Chen wrote: > > From: Zhang Chen > > > > Current COLO mode(independent disk mode) need replication module work > > together. Suggested by Dr. David Alan Gilbert . > &

Re: [Qemu-devel] [PATCH v4 2/2] colo: compare the packet based on the tcp sequence number

2019-01-14 Thread Zhang Chen
Signed-off-by: Mao Zhongyi > > Signed-off-by: Li Zhijian > > Signed-off-by: Zhang Chen > > Reviewed-by: Zhang Chen > > Tested-by: Zhang Chen > > --- > > net/colo-compare.c | 343 > +++-- > > net/colo.c

[Qemu-devel] [PATCH] net/colo-compare.c: Remove duplicated code

2019-01-14 Thread Zhang Chen
From: Zhang Chen Fix duplicated code: https://bugs.launchpad.net/qemu/+bug/1811499 Signed-off-by: Zhang Chen --- net/colo-compare.c | 8 1 file changed, 8 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 9156ab3349..fa3fd0632d 100644 --- a/net/colo-compare.c

[Qemu-devel] [Bug 1811499] Re: qemu/net/colo-compare.c:288: possible pointless code duplication ?

2019-01-14 Thread Zhang Chen
Have fixed in this patch. https://lists.nongnu.org/archive/html/qemu-devel/2019-01/msg02859.html Thanks Zhang Chen -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1811499 Title: qemu/net/colo

Re: [Qemu-devel] test-filter-mirror hangs

2019-01-14 Thread Zhang Chen
also hangs > > occasionally with TCG due to some bug I'm sure we've investigated > > in the past) ? > > I don't think it relies on the CPU at all. > > Dave > > I have no idea about this currently, but Jason and I designed the test case. Add Jason: Have any c

[Qemu-devel] [PATCH V7 16/17] COLO: notify net filters about checkpoint/failover event

2018-05-13 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang --- migration/colo.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 5e517dc..d67bdc2 100644 --- a/migration/colo.c +++ b

[Qemu-devel] [PATCH V7 04/17] COLO: integrate colo compare with colo frame

2018-05-13 Thread Zhang Chen
-by: zhanghailiang Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 -- migration/migration.c | 4 ++-- 2 files changed, 42 insertions(+), 4 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index

[Qemu-devel] [PATCH V7 12/17] savevm: split the process of different stages for loadvm/savevm

2018-05-13 Thread Zhang Chen
_main() and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PATCH V7 15/17] filter-rewriter: handle checkpoint and failover event

2018-05-13 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen --- migration/colo.c | 13 + net/filter-rewriter.c | 40 2 files changed, 53 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 7869d55..5e517dc 100644 --- a/migration/colo.c +++ b

[Qemu-devel] [PATCH V7 14/17] filter: Add handle_event method for NetFilterClass

2018-05-13 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 28 3 files changed, 50 insertions(+)

[Qemu-devel] [PATCH V7 11/17] qapi: Add new command to query colo status

2018-05-13 Thread Zhang Chen
Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen --- migration/colo.c| 34 ++ qapi

[Qemu-devel] [PATCH V7 08/17] ram/COLO: Record the dirty pages that SVM received

2018-05-13 Thread Zhang Chen
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the

[Qemu-devel] [PATCH V7 10/17] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-05-13 Thread Zhang Chen
x27;s failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Eric Blake --- migration/colo.c| 20 qapi/migration.json | 37 +++

[Qemu-devel] [PATCH V7 17/17] COLO: quick failover process by kick COLO thread

2018-05-13 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang --- migration/colo.c | 8 1 file changed, 8 insertions(+) diff --git a/migration/colo.c b/migra

[Qemu-devel] [PATCH V7 RESEND 01/17] filter-rewriter: fix memory leak for connection in connection_track_table

2018-05-14 Thread Zhang Chen
After a net connection is closed, we didn't clear its releated resources in connection_track_table, which will lead to memory leak. Let't track the state of net connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by:

[Qemu-devel] [PATCH V7 RESEND 07/17] COLO: Load dirty pages into SVM's RAM cache firstly

2018-05-14 Thread Zhang Chen
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- include/exec/ram_addr.h | 1 + migration/migration.c | 2 + migration/ram.c | 99 +++-- migration/ram.h | 4 ++ migration/savevm.c | 2

[Qemu-devel] [PATCH V7 RESEND 02/17] colo-compare: implement the process of checkpoint

2018-05-14 Thread Zhang Chen
While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- include

[Qemu-devel] [PATCH V7 RESEND 10/17] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-05-14 Thread Zhang Chen
x27;s failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Eric Blake --- migration/colo.c| 20 qapi/migration.json | 37 +++

[Qemu-devel] [PATCH V7 RESEND 03/17] colo-compare: use notifier to notify packets comparing result

2018-05-14 Thread Zhang Chen
It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 32 +--- net/colo-compare.h | 2 ++ 2 files changed, 27 insertions(+), 7 deletions(-)

[Qemu-devel] [PATCH V7 RESEND 00/17] COLO: integrate colo frame with block replication and COLO proxy

2018-05-14 Thread Zhang Chen
ic Blake's comments, use the enum to feedback in patch 11/17. - Fixed QAPI command separator problem in patch 11/17. Zhang Chen (12): filter-rewriter: fix memory leak for connection in connection_track_table colo-compare: implement the process of checkpoint colo-compare: use notifier to

[Qemu-devel] [PATCH V7 RESEND 09/17] COLO: Flush memory data from ram cache

2018-05-14 Thread Zhang Chen
During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all content of PVM's RAM

[Qemu-devel] [PATCH V7 RESEND 04/17] COLO: integrate colo compare with colo frame

2018-05-14 Thread Zhang Chen
-by: zhanghailiang Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 -- migration/migration.c | 4 ++-- 2 files changed, 42 insertions(+), 4 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index

[Qemu-devel] [PATCH V7 RESEND 12/17] savevm: split the process of different stages for loadvm/savevm

2018-05-14 Thread Zhang Chen
_main() and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PATCH V7 RESEND 05/17] COLO: Add block replication into colo process

2018-05-14 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- migration/colo.c

[Qemu-devel] [PATCH V7 RESEND 14/17] filter: Add handle_event method for NetFilterClass

2018-05-14 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 28 3 files changed, 50 insertions(+)

[Qemu-devel] [PATCH V7 RESEND 13/17] COLO: flush host dirty ram from cache

2018-05-14 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- migration/ram.c | 12 1 file changed, 12 insertions(+) diff --git a

[Qemu-devel] [PATCH V7 RESEND 06/17] COLO: Remove colo_state migration struct

2018-05-14 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen --- include/migration/colo.h | 5 +-- migration/Makefile.objs | 2 +- migration/colo-comm.c| 76 migration/colo.c | 13 ++- migration/migration.c| 23 +++- migration/savevm.c | 20

[Qemu-devel] [PATCH V7 RESEND 17/17] COLO: quick failover process by kick COLO thread

2018-05-14 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang --- migration/colo.c | 8 1 file changed, 8 insertions(+) diff --git a/migration/colo.c b/migra

[Qemu-devel] [PATCH V7 RESEND 15/17] filter-rewriter: handle checkpoint and failover event

2018-05-14 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen --- migration/colo.c | 13 + net/filter-rewriter.c | 40 2 files changed, 53 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 5b055f79f1..3dfd84d897 100644 --- a/migration/colo.c

[Qemu-devel] [PATCH V7 RESEND 16/17] COLO: notify net filters about checkpoint/failover event

2018-05-14 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang --- migration/colo.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 3dfd84d897..15463e2823 100644 --- a/migration/colo.c

[Qemu-devel] [PATCH V7 RESEND 08/17] ram/COLO: Record the dirty pages that SVM received

2018-05-14 Thread Zhang Chen
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the

[Qemu-devel] [PATCH V7 RESEND 11/17] qapi: Add new command to query colo status

2018-05-14 Thread Zhang Chen
Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen --- migration/colo.c| 34 ++ qapi

Re: [Qemu-devel] [PATCH V7 11/17] qapi: Add new command to query colo status

2018-05-16 Thread Zhang Chen
On Tue, May 15, 2018 at 10:26 PM, Markus Armbruster wrote: > Zhang Chen writes: > > > Libvirt or other high level software can use this command query colo > status. > > You can test this command like that: > > {'execute':'query-colo-

Re: [Qemu-devel] [PATCH V7 10/17] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-05-16 Thread Zhang Chen
On Tue, May 15, 2018 at 10:29 PM, Markus Armbruster wrote: > Zhang Chen writes: > > > From: zhanghailiang > > > > If some errors happen during VM's COLO FT stage, it's important to > > notify the users of this event. Together with 'x-colo-lost-

Re: [Qemu-devel] [PATCH V7 RESEND 04/17] COLO: integrate colo compare with colo frame

2018-05-16 Thread Zhang Chen
On Wed, May 16, 2018 at 7:12 PM, Dr. David Alan Gilbert wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > For COLO FT, both the PVM and SVM run at the same time, > > only sync the state while it needs. > > > > So here, let SVM runs while n

Re: [Qemu-devel] [PATCH V7 RESEND 06/17] COLO: Remove colo_state migration struct

2018-05-16 Thread Zhang Chen
On Wed, May 16, 2018 at 12:02 AM, Dr. David Alan Gilbert < dgilb...@redhat.com> wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > We need to know if migration is going into COLO state for > > incoming side before start normal migration. > > > > Instead by usi

Re: [Qemu-devel] [PATCH V7 10/17] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-05-20 Thread Zhang Chen
On Thu, May 17, 2018 at 4:19 PM, Markus Armbruster wrote: > Zhang Chen writes: > > > On Tue, May 15, 2018 at 10:29 PM, Markus Armbruster > > wrote: > > > >> Zhang Chen writes: > >> > >> > From: zhanghailiang > >> > > &g

Re: [Qemu-devel] [PATCH V7 RESEND 09/17] COLO: Flush memory data from ram cache

2018-05-20 Thread Zhang Chen
On Tue, May 15, 2018 at 10:44 PM, Dr. David Alan Gilbert < dgilb...@redhat.com> wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > During the time of VM's running, PVM may dirty some pages, we will > transfer > > PVM's dirty pages to SVM and store

Re: [Qemu-devel] [PATCH V7 RESEND 07/17] COLO: Load dirty pages into SVM's RAM cache firstly

2018-05-20 Thread Zhang Chen
On Wed, May 16, 2018 at 12:55 AM, Dr. David Alan Gilbert < dgilb...@redhat.com> wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > We should not load PVM's state directly into SVM, because there maybe > some > > errors happen when SVM is receving data, which will

Re: [Qemu-devel] [PATCH V11 16/20] filter: Add handle_event method for NetFilterClass

2018-08-23 Thread Zhang Chen
On Wed, Aug 22, 2018 at 4:22 PM Jason Wang wrote: > > > On 2018年08月21日 17:25, Zhang Chen wrote: > > On Tue, Aug 21, 2018 at 11:30 AM Jason Wang wrote: > > > >> > >> On 2018年08月12日 04:59, Zhang Chen wrote: > >>> Filter needs to process the event

Re: [Qemu-devel] [PATCH V11 16/20] filter: Add handle_event method for NetFilterClass

2018-08-29 Thread Zhang Chen
On Tue, Aug 28, 2018 at 3:19 PM Jason Wang wrote: > > > On 2018年08月24日 13:57, Zhang Chen wrote: > > On Wed, Aug 22, 2018 at 4:22 PM Jason Wang wrote: > > > >> On 2018年08月21日 17:25, Zhang Chen wrote: > >>> On Tue, Aug 21, 2018 at 11:30 AM Jason Wang

[Qemu-devel] [PATCH V12 01/19] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-09-02 Thread Zhang Chen
connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo.c| 2 +- net/colo.h| 9 ++-- net/filter-rewriter.c | 105 ++---

[Qemu-devel] [PATCH V12 00/19] COLO: integrate colo frame with block replication and COLO proxy

2018-09-02 Thread Zhang Chen
h segmentation fault. V7: - Addressed Markus's comments in 11/17. - Rebased on upstream. V6: - Addressed Eric Blake's comments, use the enum to feedback in patch 11/17. - Fixed QAPI command separator problem in patch 11/17. Zhang Chen (15): filter-rewriter: Add TCP state machine a

[Qemu-devel] [PATCH V12 02/19] colo-compare: implement the process of checkpoint

2018-09-02 Thread Zhang Chen
While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by

[Qemu-devel] [PATCH V12 03/19] colo-compare: use notifier to notify packets comparing result

2018-09-02 Thread Zhang Chen
It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 37 ++--- net/colo-compare.h | 2 ++ 2 files change

[Qemu-devel] [PATCH V12 07/19] COLO: Load dirty pages into SVM's RAM cache firstly

2018-09-02 Thread Zhang Chen
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/exec/ram_addr.h | 1 + migration/migration.c | 7 migration/ram.c | 83 - mig

[Qemu-devel] [PATCH V12 04/19] COLO: integrate colo compare with colo frame

2018-09-02 Thread Zhang Chen
-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 -- migration/migration.c | 6 ++ 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/migration

[Qemu-devel] [PATCH V12 08/19] ram/COLO: Record the dirty pages that SVM received

2018-09-02 Thread Zhang Chen
the dirty pages caused by PVM and SVM, we only flush those dirty pages from RAM cache while do checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 43 --- 1

[Qemu-devel] [PATCH V12 09/19] COLO: Flush memory data from ram cache

2018-09-02 Thread Zhang Chen
During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all content of PVM's RAM

[Qemu-devel] [PATCH V12 05/19] COLO: Add block replication into colo process

2018-09-02 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- migration/colo.c

[Qemu-devel] [PATCH V12 06/19] COLO: Remove colo_state migration struct

2018-09-02 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/migration/colo.h | 5 +-- migration/Makefile.objs | 2 +- migration/colo-comm.c| 76 migration/colo.c | 13 ++- migration

[Qemu-devel] [PATCH V12 10/19] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-09-02 Thread Zhang Chen
x27;s failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo.c| 31 +

[Qemu-devel] [PATCH V12 19/19] docs: Add COLO status diagram to COLO-FT.txt

2018-09-02 Thread Zhang Chen
From: Zhang Chen This diagram make user better understand COLO. Suggested by Markus Armbruster. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- docs/COLO-FT.txt | 34 ++ 1 file changed, 34 insertions(+) diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt

[Qemu-devel] [PATCH V12 11/19] qapi/migration.json: Rename COLO unknown mode to none mode.

2018-09-02 Thread Zhang Chen
From: Zhang Chen Suggested by Markus Armbruster rename COLO unknown mode to none mode. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo-failover.c | 2 +- migration/colo.c | 2 +- qapi/migration.json

[Qemu-devel] [PATCH V12 12/19] qapi: Add new command to query colo status

2018-09-02 Thread Zhang Chen
Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- migration/colo.c| 21 + qapi

[Qemu-devel] [PATCH V12 14/19] COLO: flush host dirty ram from cache

2018-09-02 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration

[Qemu-devel] [PATCH V12 13/19] savevm: split the process of different stages for loadvm/savevm

2018-09-02 Thread Zhang Chen
) and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PATCH V12 15/19] filter: Add handle_event method for NetFilterClass

2018-09-02 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 19

[Qemu-devel] [PATCH V12 18/19] COLO: quick failover process by kick COLO thread

2018-09-02 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 8 1 file changed, 8 insertions(+)

[Qemu-devel] [PATCH V12 17/19] COLO: notify net filters about checkpoint/failover event

2018-09-02 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 25d279decf..

[Qemu-devel] [PATCH V12 16/19] filter-rewriter: handle checkpoint and failover event

2018-09-02 Thread Zhang Chen
ff-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo-compare.c| 12 - net/colo.c| 8 ++ net/colo.h| 2 ++ net/filter-rewriter.c | 57 +++ 4 files changed, 73 insertions(+), 6 dele

Re: [Qemu-devel] [PATCH V12 05/19] COLO: Add block replication into colo process

2018-09-03 Thread Zhang Chen
Sorry, forgot to add the reviewed by: Reviewed-by: Dr. David Alan Gilbert Thanks Zhang Chen On Mon, Sep 3, 2018 at 12:40 PM Zhang Chen wrote: > Make sure master start block replication after slave's block > replication started. > > Besides, we need to activate VM's b

[Qemu-devel] [PATCH] docs/block-replication.txt: Add more detail about replication_do_checkpoint_all

2018-09-05 Thread Zhang Chen
Add more detail description for COLO checkpoint use case. Suggested by Dr. David Alan Gilbert Signed-off-by: Zhang Chen --- docs/block-replication.txt | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/block-replication.txt b/docs/block-replication.txt index

Re: [Qemu-devel] [PATCH V12 00/19] COLO: integrate colo frame with block replication and COLO proxy

2018-09-10 Thread Zhang Chen
Hi All. Have any comments? Ping... Thanks Zhang Chen On Mon, Sep 3, 2018 at 12:39 PM Zhang Chen wrote: > Hi~ All~ > > COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror, > filter-redirector,filter-rewriter) have been exist in qemu > for long time, it'

<    5   6   7   8   9   10   11   12   13   14   >