Re: [Xen-devel] [PATCH 1/3] Don't create default ioreq server

2016-12-06 Thread Wen Congyang
At 2016/11/30 20:25, Andrew Cooper wrote: On 30/11/16 09:47, Zhang Chen wrote: The ioreq server make colo run failed. Signed-off-by: Zhang Chen Signed-off-by: Wen Congyang Nack. You can simply "fix" a COLO issue by breaking a much more common usecase. Yes, this patch is wr

Re: [Xen-devel] Error migrating VM to secondary host using COLO replication

2016-11-09 Thread Wen Congyang
for the --set-mark argument to have value equal '1' , not a hex? This log is very useful, we will investigate it. Thanks Wen Congyang > > The i got running the command is: > root@colop:~# arptables -I INPUT -i eth1 -j MARK --set-mark 1 > Bad argument `1

Re: [Xen-devel] Error migrating VM to secondary host using COLO replication

2016-11-02 Thread Wen Congyang
>>libxl: error: libxl_qmp.c:702:libxl__qmp_initialize: Connection error: No >>such file or directory >>libxl: error: libxl_colo_restore.c:817:colo_restore_setup_cds_done: COLO: >>failed to setup device >for guest with domid 1 According to the log, we cannot connect to

Re: [Xen-devel] [PATCH for-4.7] tools/libxl: Fix legacy migration following COLO backchannel breakage

2016-04-14 Thread Wen Congyang
encounter an error as the > legacy stream appears to skip ahead. > > Undo the adjustments to libxl__stream_read_start(), and introduce a > better description of what is going on. Introduce some extra assertions > to try and catch similar breakage in the future. > > Reported-by: Olaf Heri

Re: [Xen-devel] [PATCH 0/3] COLO: only build on Linux

2016-04-06 Thread Wen Congyang
On 04/06/2016 06:17 PM, Wei Liu wrote: > On Wed, Apr 06, 2016 at 09:24:26AM +0800, Wen Congyang wrote: >> On 04/06/2016 04:05 AM, Wei Liu wrote: >>> COLO depends on netlink which is only available on Linux. This series >>> cleans up >>> COLO code and make it

Re: [Xen-devel] [PATCH 0/3] COLO: only build on Linux

2016-04-05 Thread Wen Congyang
I also > have a question why COLO doesn't use libnl? It is using Linux header directly. It is implemented by Yang one year ago. I don't know the reason. We will investigate how to use libnl. Thanks Wen Congyang > > Roger, can you run functional tests on FreeBSD? I don't have

Re: [Xen-devel] [PATCH v13 00/26] COarse-grain LOck-stepping Virtual Machines for Non-stop Service

2016-03-30 Thread Wen Congyang
Yes, it is required for COLO. > > > What is the status of the default ioreq server patch ? Why is it in > your git branch ? I have reported this bug last year: http://lists.xenproject.org/archives/html/xen-devel/2015-12/msg02850.html This patch is just a temporary patch. Thanks We

Re: [Xen-devel] [PATCH v11 20/27] Support colo mode for qemu disk

2016-03-19 Thread Wen Congyang
On 03/18/2016 01:18 AM, Ian Jackson wrote: > Wen Congyang writes ("Re: [PATCH v11 20/27] Support colo mode for qemu disk"): >> How does block replication work: > > Thanks for this explanation, which is reallt helpful. > > I would like to repeat back to you

Re: [Xen-devel] [PATCH v4 3/3] libxl: rename checkpointed_stream to stream_type

2016-03-15 Thread Wen Congyang
On 03/15/2016 08:54 PM, Wei Liu wrote: > On Tue, Mar 15, 2016 at 09:58:37AM +0800, Wen Congyang wrote: >> Signed-off-by: Wen Congyang >> --- >> v3->v4: Remove the new macro, and updte the macro >> LIBXL_HAVE_CHECKPOINTED_STREAM >> tools/libxl/libxl.c

[Xen-devel] [PATCH v4 2/3] tools: change checkpointed_stream's type from int to xc_migration_stream_t

2016-03-14 Thread Wen Congyang
checkpointed_stream is also renamed to stream_type Signed-off-by: Wen Congyang Acked-by: Wei Liu --- v2->v3: Rename checkpointed_stream to stream_type tools/libxc/include/xenguest.h | 8 tools/libxc/xc_nomigrate.c | 4 ++-- tools/libxc/xc_sr_restore.c |

[Xen-devel] [PATCH v4 3/3] libxl: rename checkpointed_stream to stream_type

2016-03-14 Thread Wen Congyang
Signed-off-by: Wen Congyang --- v3->v4: Remove the new macro, and updte the macro LIBXL_HAVE_CHECKPOINTED_STREAM tools/libxl/libxl.c | 4 ++-- tools/libxl/libxl.h | 4 +++- tools/libxl/libxl_create.c | 4 ++-- tools/libxl/libxl_dom_save.c | 6 +++--- to

[Xen-devel] [PATCH v4 1/3] libxc: move migration_stream's definition to xenguest.h

2016-03-14 Thread Wen Congyang
xc_domain_save() and xc_domain_restore's parameter will use this type, so it should be public. Signed-off-by: Wen Congyang Acked-by: Wei Liu --- v2->v3: Rename MIG_STREAM_* to XC_MIG_STREAM_* tools/libxc/include/xenguest.h | 7 ++- tools/libxc/xc_sr_common.h | 10 -

Re: [Xen-devel] [PATCH v10 26/31] COLO proxy: implement setup/teardown of COLO proxy module

2016-03-14 Thread Wen Congyang
v)); >> +} >> +return ret; >> +} >> + >> +/* = colo-proxy: setup and teardown == */ >> + >> +int colo_proxy_setup(libxl__colo_proxy_state *cps) >> +{ >> +int skfd = 0; >> +struct sockaddr_nl sa; >>

Re: [Xen-devel] [PATCH v11 10/27] tools/libxl: add back channel support to write stream

2016-03-11 Thread Wen Congyang
On 03/05/2016 01:00 AM, Ian Jackson wrote: > Changlong Xie writes ("[PATCH v11 10/27] tools/libxl: add back channel > support to write stream"): >> From: Wen Congyang >> >> Add back channel support to write stream. If the write stream is >> a back channe

Re: [Xen-devel] [PATCH v11 07/27] docs/libxl: Introduce CHECKPOINT_CONTEXT to support migration v2 colo streams

2016-03-10 Thread Wen Congyang
On 03/05/2016 12:51 AM, Ian Jackson wrote: > Changlong Xie writes ("[PATCH v11 07/27] docs/libxl: Introduce > CHECKPOINT_CONTEXT to support migration v2 colo streams"): >> From: Wen Congyang > > I think we will want to see an ack from Andy Cooper on this, in d

[Xen-devel] [PATCH v3 1/3] libxc: move migration_stream's definition to xenguest.h

2016-03-10 Thread Wen Congyang
xc_domain_save() and xc_domain_restore's parameter will use this type, so it should be public. Signed-off-by: Wen Congyang --- v2->v3: Rename MIG_STREAM_* to XC_MIG_STREAM_* tools/libxc/include/xenguest.h | 7 ++- tools/libxc/xc_sr_common.h | 10 -- tools/libxc/xc_s

[Xen-devel] [PATCH v3 3/3] libxl: rename checkpointed_stream to stream_type

2016-03-10 Thread Wen Congyang
Signed-off-by: Wen Congyang --- tools/libxl/libxl.c | 4 ++-- tools/libxl/libxl.h | 8 tools/libxl/libxl_create.c | 4 ++-- tools/libxl/libxl_dom_save.c | 6 +++--- tools/libxl/libxl_internal.h | 2 +- tools/libxl/libxl_save_callout.c | 4

[Xen-devel] [PATCH v3 2/3] tools: change checkpointed_stream's type from int to xc_migration_stream_t

2016-03-10 Thread Wen Congyang
checkpointed_stream is also renamed to stream_type Signed-off-by: Wen Congyang --- v2->v3: Rename checkpointed_stream to stream_type tools/libxc/include/xenguest.h | 8 tools/libxc/xc_nomigrate.c | 4 ++-- tools/libxc/xc_sr_restore.c | 10 +- tools/li

[Xen-devel] [PATCH v2 2/2] tools: change checkpointed_stream's type from int to xc_migration_stream_t

2016-03-09 Thread Wen Congyang
Signed-off-by: Wen Congyang --- tools/libxc/include/xenguest.h | 4 ++-- tools/libxc/xc_nomigrate.c | 4 ++-- tools/libxc/xc_sr_restore.c | 2 +- tools/libxc/xc_sr_save.c| 2 +- tools/libxl/libxl_save_helper.c | 38 +++--- 5 files changed, 25

[Xen-devel] [PATCH v2 1/2] libxc: move migration_stream's definition to xenguest.h

2016-03-09 Thread Wen Congyang
xc_domain_save() and xc_domain_restore's parameter will use this type, so it should be public. Signed-off-by: Wen Congyang --- tools/libxc/include/xenguest.h | 5 + tools/libxc/xc_sr_common.h | 10 -- 2 files changed, 5 insertions(+), 10 deletions(-) diff --git a/tools/

Re: [Xen-devel] [PATCH] libxc: move migration_stream's definition to xenguest.h

2016-03-08 Thread Wen Congyang
On 03/08/2016 06:38 PM, Andrew Cooper wrote: > On 08/03/16 05:32, Wen Congyang wrote: >> xc_save_domain()'s parameter use this type, so it should >> be public. > > xc_domain_save() currently uses an int, which also needs fixing. OK. Will fix it in the next version. &

[Xen-devel] [PATCH] libxc: move migration_stream's definition to xenguest.h

2016-03-07 Thread Wen Congyang
xc_save_domain()'s parameter use this type, so it should be public. Signed-off-by: Wen Congyang --- tools/libxc/include/xenguest.h | 10 ++ tools/libxc/xc_sr_common.h | 10 -- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/tools/libxc/include/xengues

Re: [Xen-devel] [PATCH v11 18/27] implement the cmdline for COLO

2016-03-06 Thread Wen Congyang
TREAM_COLO; >> +else >> +dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_REMUS; > > If you prefer, a ? : expression would do here as well. > > +dss->checkpointed_stream = >libxl_defbool_val() ? LIBXL_CHECKPOINTED_STREAM_COLO :.

Re: [Xen-devel] [PATCH v11 17/27] libxc/save: support COLO save

2016-03-06 Thread Wen Congyang
On 03/05/2016 01:18 AM, Ian Jackson wrote: > Changlong Xie writes ("[PATCH v11 17/27] libxc/save: support COLO save"): >> From: Wen Congyang >> >> After suspend primary vm, get dirty bitmap on secondary vm, >> and send pages both dirty on primary/secondar

Re: [Xen-devel] [PATCH v11 15/27] primary vm suspend/resume/checkpoint code

2016-03-06 Thread Wen Congyang
On 03/05/2016 01:14 AM, Ian Jackson wrote: > Changlong Xie writes ("[PATCH v11 15/27] primary vm suspend/resume/checkpoint > code"): >> From: Wen Congyang > > I would look at this on the same basis as the previous patch. > >> diff --git a/tools

Re: [Xen-devel] [PATCH v11 14/27] secondary vm suspend/resume/checkpoint code

2016-03-06 Thread Wen Congyang
On 03/05/2016 01:11 AM, Ian Jackson wrote: > Changlong Xie writes ("[PATCH v11 14/27] secondary vm > suspend/resume/checkpoint code"): >> From: Wen Congyang >> >> Secondary vm is running in colo mode. So we will do >> the following things again and again

Re: [Xen-devel] [PATCH v11 12/27] tools/libx{l, c}: introduce wait_checkpoint callback

2016-03-06 Thread Wen Congyang
On 03/05/2016 04:23 AM, Konrad Rzeszutek Wilk wrote: > On Fri, Mar 04, 2016 at 05:03:16PM +, Ian Jackson wrote: >> Changlong Xie writes ("[PATCH v11 12/27] tools/libx{l,c}: introduce >> wait_checkpoint callback"): >>> From: Wen Congyang >>> >&g

Re: [Xen-devel] [PATCH v11 10/27] tools/libxl: add back channel support to write stream

2016-03-06 Thread Wen Congyang
On 03/05/2016 01:00 AM, Ian Jackson wrote: > Changlong Xie writes ("[PATCH v11 10/27] tools/libxl: add back channel > support to write stream"): >> From: Wen Congyang >> >> Add back channel support to write stream. If the write stream is >> a back channe

Re: [Xen-devel] [PATCH v11 20/27] Support colo mode for qemu disk

2016-03-06 Thread Wen Congyang
backend? I don't know how to work with PV block backend. It is one reason that why we only support pure HVM now. For PV block backend, there is also other problem. For exampe resuming it in the secondary side is very slow, because we need to disconnect and reconnect. Thanks Wen Congyang &g

Re: [Xen-devel] [PATCH v11 20/27] Support colo mode for qemu disk

2016-03-06 Thread Wen Congyang
On 03/05/2016 01:44 AM, Ian Jackson wrote: > Changlong Xie writes ("[PATCH v11 20/27] Support colo mode for qemu disk"): >> From: Wen Congyang >> >> Usage: disk = >> ['...,colo,colo-host=xxx,colo-port=xxx,colo-export=xxx,active-disk=xxx,hidden-disk=xxx.

Re: [Xen-devel] Patching error while setting up COLO

2016-03-06 Thread Wen Congyang
>configure was not able to find it. >Install xen devel > > What do you think what I am missing? Thank you! Do you build xen before? Thanks Wen Congyang > > Victor > > > > On Thu, Mar 3, 2016 at 6:15 PM, Wen Congyang <mailto:we...@cn.fujitsu.com

Re: [Xen-devel] Patching error while setting up COLO

2016-03-03 Thread Wen Congyang
enable_trace_backend='' ; \ fi ; \ $$source/configure --enable-xen --target-list=i386-softmmu \ $(QEMU_XEN_ENABLE

Re: [Xen-devel] Patching error while setting up COLO

2016-03-03 Thread Wen Congyang
On 03/03/2016 05:39 PM, Yu-An(Victor) Chen wrote: > Hi Changlong, > > Thanks for the reply, > > Again when I am trying to do the following: > > 5. build qemu-colo > 1) cd ~/qemu-colo/; *git checkout colo-xen-v2* > * > * > I got this error message *"error: pathspec 'colo-xen-v2' did not match any

Re: [Xen-devel] [PATCH v10 31/31] cmdline switches and config vars to control colo-proxy

2016-03-02 Thread Wen Congyang
On 03/02/2016 11:05 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:35AM +0800, Wen Congyang wrote: >> Add cmdline switches to 'xl migrate-receive' command to specify >> a domain-specific hotplug script to setup COLO proxy. >> >> Add a new config var '

Re: [Xen-devel] [PATCH v10 24/31] Support colo mode for qemu disk

2016-03-02 Thread Wen Congyang
On 03/02/2016 11:04 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:28AM +0800, Wen Congyang wrote: >> Usage: disk = >> ['...,colo,colo-host=xxx,colo-port=xxx,colo-export=xxx,active-disk=xxx,hidden-disk=xxx...'] >> For QEMU block replication details: &

Re: [Xen-devel] [PATCH v10 22/31] implement the cmdline for COLO

2016-03-02 Thread Wen Congyang
On 03/02/2016 11:03 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:26AM +0800, Wen Congyang wrote: > [...] >> +if (libxl_defbool_val(info->colo)) { >> +if (libxl_defbool_val(info->compression)) { > > This can be simplified as > &

Re: [Xen-devel] [PATCH v10 10/31] tools/libxl: add back channel support to write stream

2016-03-02 Thread Wen Congyang
On 03/02/2016 11:02 PM, Wei Liu wrote: > On Fri, Feb 26, 2016 at 10:11:27AM +0800, Wen Congyang wrote: >> On 02/25/2016 11:54 PM, Wei Liu wrote: >>> On Mon, Feb 22, 2016 at 10:52:14AM +0800, Wen Congyang wrote: >>>> Add back channel support to write stream. If t

Re: [Xen-devel] [PATCH v10 16/31] secondary vm suspend/resume/checkpoint code

2016-03-01 Thread Wen Congyang
On 02/25/2016 11:56 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:20AM +0800, Wen Congyang wrote: >> Secondary vm is running in colo mode. So we will do >> the following things again and again: >> 1. Resume secondary vm >>a. Send CHECKPOINT_SVM_READY to master

Re: [Xen-devel] [PATCH v10 21/31] libxc/save: support COLO save

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:58 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:25AM +0800, Wen Congyang wrote: > [...] >> /* >> * Suspend the domain and send dirty memory. >> * This is the last iteration of the live migration and the >> @@ -558,6 +610,16 @@ static int

Re: [Xen-devel] [PATCH v10 18/31] libxc/restore: support COLO restore

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:57 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:22AM +0800, Wen Congyang wrote: > [...] >> - * With Remus, we buffer the records sent by the primary at checkpoint, >> + * With Remus/COLO, we buffer the records sent by the primary at checkpoint, >>

Re: [Xen-devel] [PATCH v10 17/31] primary vm suspend/resume/checkpoint code

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:57 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:21AM +0800, Wen Congyang wrote: > [...] >> +#include "libxl_osdeps.h" /* must come before any other headers */ >> + >> +#include "libxl_internal.h" >&

Re: [Xen-devel] [PATCH v10 16/31] secondary vm suspend/resume/checkpoint code

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:56 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:20AM +0800, Wen Congyang wrote: >> Secondary vm is running in colo mode. So we will do >> the following things again and again: >> 1. Resume secondary vm >>a. Send CHECKPOINT_SVM_READY to master

Re: [Xen-devel] [PATCH v10 12/31] tools/libxl: add back channel support to read stream

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:54 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:16AM +0800, Wen Congyang wrote: >> This is used by primay to read records sent by secondary. >> >> Signed-off-by: Yang Hongyang >> Signed-off-by: Wen Congyang >> --- >> tools/libxl/lib

Re: [Xen-devel] [PATCH v10 10/31] tools/libxl: add back channel support to write stream

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:54 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:14AM +0800, Wen Congyang wrote: >> Add back channel support to write stream. If the write stream is >> a back channel stream, this means the write stream is used by >> Secondary to send some records back

Re: [Xen-devel] [PATCH v10 07/31] docs/libxl: Introduce CHECKPOINT_CONTEXT to support migration v2 colo streams

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:54 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:11AM +0800, Wen Congyang wrote: >> It is the negotiation record for COLO. >> Primary->Secondary: >> control_id 0x: Secondary VM is out of sync, start a new >> ch

Re: [Xen-devel] [PATCH v10 01/31] tools/libxl: introduce libxl__domain_restore_device_model to load qemu state

2016-02-25 Thread Wen Congyang
On 02/25/2016 11:53 PM, Wei Liu wrote: > On Mon, Feb 22, 2016 at 10:52:05AM +0800, Wen Congyang wrote: >> In normal migration, the qemu state is passed to qemu as a parameter. >> With COLO, secondary vm is running. So we will do the following steps >> at every checkpoin

[Xen-devel] [PATCH v10 03/31] tools/libxl: Add back channel to allow migration target send data back

2016-02-21 Thread Wen Congyang
only can be written in primary, and only can be read in secondary. Save recv_fd in domain_suspend_state, and send_fd in domain_create_state. Extend libxl_domain_create_restore API, add a send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate the API change. Signed-off-by: Wen

[Xen-devel] [PATCH v10 02/31] tools/libxl: introduce libxl__domain_common_switch_qemu_logdirty()

2016-02-21 Thread Wen Congyang
s lds->callback before exits. This new API will be used by the patch: secondary vm suspend/resume/checkpoint code Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Andrew Cooper Acked-by: Ian Campbell Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/libxl_dom_sav

[Xen-devel] [PATCH v10 15/31] tools/libx{l, c}: add postcopy/suspend callback to restore side

2016-02-21 Thread Wen Congyang
Secondary(restore side) is running under COLO, we also need postcopy/suspend callbacks. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxc/include/xenguest.h | 10 ++ tools/libxl/libxl_save_msgs_gen.pl | 4 ++-- 2 files changed, 12 insertions(+), 2 deletions

[Xen-devel] [PATCH v10 07/31] docs/libxl: Introduce CHECKPOINT_CONTEXT to support migration v2 colo streams

2016-02-21 Thread Wen Congyang
03: Secondary VM is resumed Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang --- docs/specs/libxl-migration-stream.pandoc | 25 +++-- tools/libxl/libxl_sr_stream_format.h | 11 +++ tools/python/xen/migration/libxl.py | 9 + 3 files chang

[Xen-devel] [PATCH v10 06/31] docs: add colo readme

2016-02-21 Thread Wen Congyang
add colo readme, refer to http://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping Signed-off-by: Yang Hongyang Acked-by: Ian Campbell Signed-off-by: Wen Congyang --- docs/README.colo | 9 + 1 file changed, 9 insertions(+) create mode 100644 docs/README.colo diff --git a/docs

[Xen-devel] [PATCH v10 11/31] tools/libxl: write checkpoint_state records into the stream

2016-02-21 Thread Wen Congyang
write checkpoint_state records into the stream, used by both primary and secondary to send colo context. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl/libxl_internal.h | 5 tools/libxl/libxl_stream_write.c | 60 +++- 2

[Xen-devel] [PATCH v10 01/31] tools/libxl: introduce libxl__domain_restore_device_model to load qemu state

2016-02-21 Thread Wen Congyang
e resuming secondary vm. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Cc: Anthony Perard Reviewed-by: Konrad Rzeszutek Wilk --- tools/libxl/libxl_dom_save.c | 20 tools/libxl/libxl_internal.h | 4 tools/libxl/libxl_qmp.c | 10 ++ 3 files c

[Xen-devel] [PATCH v10 20/31] send store gfn and console gfn to xl before resuming secondary vm

2016-02-21 Thread Wen Congyang
. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang CC: Andrew Cooper --- tools/libxc/include/xenguest.h | 8 tools/libxc/xc_sr_restore.c| 7 +-- tools/libxl/libxl_colo_restore.c | 5 - tools/libxl/libxl_create.c | 3 +++ tools/libxl

[Xen-devel] [PATCH v10 08/31] libxc/migration: Specification update for DIRTY_PFN_LIST records

2016-02-21 Thread Wen Congyang
Used by secondary to send it's dirty bitmap to primary under COLO. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- docs/specs/libxc-migration-stream.pandoc | 24 ++- tools/libxc/xc_sr_common.c | 31 --- tools/libxc/xc_sr_stream_for

[Xen-devel] [PATCH v10 19/31] libxc/restore: send dirty pfn list to primary when checkpoint under colo

2016-02-21 Thread Wen Congyang
Send dirty pfn list to primary when checkpoint under colo. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxc/xc_sr_common.h | 4 ++ tools/libxc/xc_sr_restore.c | 120 +++- 2 files changed, 123 insertions(+), 1 deletion(-) diff

[Xen-devel] [PATCH v10 00/31] COarse-grain LOck-stepping Virtual Machines for Non-stop Service

2016-02-21 Thread Wen Congyang
newest xen 2. bug fix Changlog from v2 to v3: 1. rebase to newest remus 2. add nic replication support Changlog from v1 to v2: 1. rebase to newest remus 2. add disk replication support Wen Congyang (31): tools/libxl: introduce libxl__domain_restore_device_model to load qemu state tools

[Xen-devel] [PATCH v10 17/31] primary vm suspend/resume/checkpoint code

2016-02-21 Thread Wen Congyang
CHECKPOINT_SVM_RESUMED from slave 3. Wait a new checkpoint a. Wait a new checkpoint(not implemented) b. Send CHECKPOINT_NEW to slave Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang --- tools/libxl/Makefile | 2 +- tools/libxl/libxl.c | 6 +- tools/libxl/libxl_colo.h

[Xen-devel] [PATCH v10 22/31] implement the cmdline for COLO

2016-02-21 Thread Wen Congyang
Add a new option -c to the command 'xl remus'. If you want to use COLO HA instead of Remus HA, please use -c option. Update man pages to reflect the addition of a new option to 'xl remus' command. Also add a new option -c to the internal command 'xl migrate-receive'

[Xen-devel] [PATCH v10 31/31] cmdline switches and config vars to control colo-proxy

2016-02-21 Thread Wen Congyang
ff-by: Yang Hongyang Signed-off-by: Wen Congyang --- docs/man/xl.conf.pod.5 | 6 ++ docs/man/xl.pod.1 | 1 - tools/libxl/libxl.c | 6 ++ tools/libxl/libxl_create.c | 14 -- tools/libxl/libxl_types.idl | 1 + tools/libxl/xl.c| 3 +

[Xen-devel] [PATCH v10 16/31] secondary vm suspend/resume/checkpoint code

2016-02-21 Thread Wen Congyang
aster. 2. Wait a new checkpoint a. Call libxl__checkpoint_devices_commit(). b. Read CHECKPOINT_NEW from master. 3. Suspend secondary vm a. Suspend secondary vm. b. Call libxl__checkpoint_devices_postsuspend(). c. Send CHECKPOINT_SVM_SUSPENDED to master. Signed-off-by: Wen Congyang S

[Xen-devel] [PATCH v10 27/31] COLO proxy: preresume, postresume and checkpoint

2016-02-21 Thread Wen Congyang
preresume, postresume and checkpoint Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl/libxl_colo.h | 4 +++ tools/libxl/libxl_colo_proxy.c | 62 ++ 2 files changed, 66 insertions(+) diff --git a/tools/libxl/libxl_colo.h b

[Xen-devel] [PATCH v10 26/31] COLO proxy: implement setup/teardown of COLO proxy module

2016-02-21 Thread Wen Congyang
setup/teardown of COLO proxy module. we use netlink to communicate with proxy module. About colo-proxy module: https://lkml.org/lkml/2015/6/18/32 How to use: http://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl

[Xen-devel] [PATCH v10 10/31] tools/libxl: add back channel support to write stream

2016-02-21 Thread Wen Congyang
Add back channel support to write stream. If the write stream is a back channel stream, this means the write stream is used by Secondary to send some records back. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl/libxl_dom_save.c | 1 + tools/libxl/libxl_internal.h

[Xen-devel] [PATCH v10 12/31] tools/libxl: add back channel support to read stream

2016-02-21 Thread Wen Congyang
This is used by primay to read records sent by secondary. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl/libxl_create.c | 1 + tools/libxl/libxl_internal.h| 1 + tools/libxl/libxl_stream_read.c | 27 +++ 3 files changed, 25

[Xen-devel] [PATCH v10 25/31] COLO: use qemu block replication

2016-02-21 Thread Wen Congyang
Use qemu block replication as our block replication solution. Note that guest must be paused before starting COLO, otherwise, the disk won't be consistent between primary and secondary. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang --- tools/libxl/Makefile | 1 +

[Xen-devel] [PATCH v10 28/31] COLO nic: implement COLO nic subkind

2016-02-21 Thread Wen Congyang
implement COLO nic subkind. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/hotplug/Linux/Makefile | 1 + tools/hotplug/Linux/colo-proxy-setup | 135 +++ tools/libxl/Makefile | 1 + tools/libxl/libxl_colo_nic.c | 321

[Xen-devel] [PATCH v10 09/31] libxc/migration: export read_record for common use

2016-02-21 Thread Wen Congyang
per Signed-off-by: Wen Congyang --- tools/libxc/xc_sr_common.c | 49 +++ tools/libxc/xc_sr_common.h | 14 ++ tools/libxc/xc_sr_restore.c | 63 + 3 files changed, 64 insertions(+), 62 deletions(-) diff --git a/to

[Xen-devel] [PATCH v10 24/31] Support colo mode for qemu disk

2016-02-21 Thread Wen Congyang
Usage: disk = ['...,colo,colo-host=xxx,colo-port=xxx,colo-export=xxx,active-disk=xxx,hidden-disk=xxx...'] For QEMU block replication details: http://wiki.qemu.org/Features/BlockReplication Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang --- docs/man/xl.pod.1

[Xen-devel] [PATCH v10 18/31] libxc/restore: support COLO restore

2016-02-21 Thread Wen Congyang
call the callbacks resume/checkpoint/suspend while secondary vm status is consistent with primary. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Andrew Cooper --- tools/libxc/xc_sr_common.h | 4 +-- tools/libxc/xc_sr_restore.c | 60

[Xen-devel] [PATCH v10 23/31] COLO: introduce new API to prepare/start/do/get_error/stop replication

2016-02-21 Thread Wen Congyang
We will use qemu block replication, and qemu provides some qmp commands to prepare replication, start replication, get replication error, and stop replication. Introduce new API to execute these qmp commands. Signed-off-by: Wen Congyang --- tools/libxl/libxl_internal.h | 20 + tools

[Xen-devel] [PATCH v10 13/31] tools/libxl: handle checkpoint_state records in a libxl migration v2 read stream

2016-02-21 Thread Wen Congyang
Read a checkpoint_state and call stream->checkpoint_callback to handle it. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl/libxl_internal.h| 3 +++ tools/libxl/libxl_stream_read.c | 57 - 2 files changed, 59 insertions(+)

[Xen-devel] [PATCH v10 30/31] setup and control colo proxy on secondary side

2016-02-21 Thread Wen Congyang
setup and control colo proxy on secondary side Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl/libxl_colo_restore.c | 28 +--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/tools/libxl/libxl_colo_restore.c b/tools/libxl

[Xen-devel] [PATCH v10 05/31] tools/libx{l, c}: add back channel to libxc

2016-02-21 Thread Wen Congyang
In COLO mode, both VMs are running, and are considered in sync if the visible network traffic is identical. After some time, they fall out of sync. At this point, the two VMs have definitely diverged. Lets call the primary dirty bitmap set A, while the secondary dirty bitmap set B. Sets A and B

[Xen-devel] [PATCH v10 04/31] tools/libxl: Introduce new helper function dup_fd_helper()

2016-02-21 Thread Wen Congyang
It is pure refactoring and no functional changes. Signed-off-by: Wen Congyang --- tools/libxl/libxl_save_callout.c | 24 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/tools/libxl/libxl_save_callout.c b/tools/libxl/libxl_save_callout.c index 7f1f5d4

[Xen-devel] [PATCH v10 14/31] tools/libx{l, c}: introduce wait_checkpoint callback

2016-02-21 Thread Wen Congyang
Under COLO, we are doing checkpoint on demand, if this callback returns 1, we will take another checkpoint. 0 indicates unexpected error. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxc/include/xenguest.h | 18 ++ tools/libxl/libxl_save_msgs_gen.pl

[Xen-devel] [PATCH v10 21/31] libxc/save: support COLO save

2016-02-21 Thread Wen Congyang
After suspend primary vm, get dirty bitmap on secondary vm, and send pages both dirty on primary/secondary to secondary. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Andrew Cooper --- tools/libxc/xc_sr_common.h | 2 + tools/libxc/xc_sr_save.c | 104

[Xen-devel] [PATCH v10 29/31] setup and control colo proxy on primary side

2016-02-21 Thread Wen Congyang
setup and control colo proxy on primary side Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang --- tools/libxl/libxl_colo_save.c | 124 +++--- tools/libxl/libxl_internal.h | 1 + 2 files changed, 117 insertions(+), 8 deletions(-) diff --git a

[Xen-devel] [PATCH v8 06/13] tools/libxl: introduce enum type libxl_checkpointed_stream

2016-02-17 Thread Wen Congyang
ed_stream is still an int. Because we will pass the value from libxl to libxc. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/libxl.h | 7 +++ tools/libxl/libxl_create.c | 8 ++-- tools/libxl

[Xen-devel] [PATCH v8 04/13] libxl/save: Refactor libxl__domain_suspend_state

2016-02-17 Thread Wen Congyang
stands for libxl__domain_save_state, dsps stands for libxl__domain_suspend_state. Also introduce libxl__domain_suspend_init to initialise the libxl__domain_suspend_state. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Ian Campbell CC: Ian Jackson CC: Wei Liu CC: Andrew Cooper

[Xen-devel] [PATCH v8 12/13] tools/libxl: move remus state into a seperate structure

2016-02-17 Thread Wen Congyang
after libxl__remus_setup(). Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/libxl.c | 2 +- tools/libxl/libxl_dom_save.c| 3 +-- tools/libxl/libxl_internal.h

[Xen-devel] [PATCH v8 08/13] tools/libxl: export logdirty_init

2016-02-17 Thread Wen Congyang
We need to enable logdirty on secondary, so we export logdirty_init for internal use. Rename it to libxl__logdirty_init. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Acked-by: Ian Campbell Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/libxl_dom_save.c | 4

[Xen-devel] [PATCH v8 03/13] tools/libxl: move save/restore code into libxl_dom_save.c

2016-02-17 Thread Wen Congyang
This is purely code motion. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Ian Jackson Acked-by: Ian Campbell Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/Makefile | 2 +- tools/libxl/libxl_dom.c | 509

[Xen-devel] [PATCH v8 02/13] tools/libxl: move remus code into libxl_remus.c

2016-02-17 Thread Wen Congyang
: Wen Congyang CC: Ian Campbell CC: Ian Jackson Acked-by:Ian Campbell Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/Makefile | 2 +- tools/libxl/libxl.c | 75 - tools/libxl/libxl_create.c | 32 tools/libxl/libxl_dom.c | 223

[Xen-devel] [PATCH v8 10/13] tools/libxl: adjust the indentation

2016-02-17 Thread Wen Congyang
This is just tidying up after the "tools/libxl: rename remus device to checkpoint device" patch automatic renaming. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Acked-by: Ian Campbell Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/libxl_checkpoin

[Xen-devel] [PATCH v8 13/13] tools/libxl: seperate device init/cleanup from checkpoint device layer

2016-02-17 Thread Wen Congyang
. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Acked-by: Ian Campbell Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/libxl_checkpoint_device.c | 42 ++- tools/libxl/libxl_remus.c | 42

[Xen-devel] [PATCH v8 00/13] Prerequisite patches for COLO

2016-02-17 Thread Wen Congyang
eckpoint callback - Introduce DIRTY_BITMAP record on libxc side - Introduce COLO_CONTEXT record on libxl side - Ported to Libxl migration v2 v1->v2: - Rebased to [PATCH v2 0/6] Misc cleanups for libxl - Add a bugfix for the error handling of process_record Wen Congyang (13): libxl/remus:

[Xen-devel] [PATCH v8 01/13] libxl/remus: init checkpoint callback in Remus setup callback

2016-02-17 Thread Wen Congyang
Init stream {read/write} state checkpoint_callback, suspend/resume/checkpoint callback in Remus setup callback. There's no functional change, it's just refactoring so that we can move all remus code into one file. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Andrew

[Xen-devel] [PATCH v8 07/13] migration/save: pass checkpointed_stream from libxl to libxc

2016-02-17 Thread Wen Congyang
Pass checkpointed_stream from libxl to libxc. It won't affact legacy migration because legacy migration won't use this param. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Ian Campbell CC: Ian Jackson CC: Wei Liu CC: Andrew Cooper Reviewed-by: Konrad Rzeszutek Wilk

[Xen-devel] [PATCH v8 09/13] tools/libxl: rename remus device to checkpoint device

2016-02-17 Thread Wen Congyang
device_(/g' tools/libxl/libxl_internal.h The patch also fixes the following backword compatibility: The error code ERROR_REMUS_XXX was introduced in Xen 4.5, and changed to ERROR_CHECKPOINT_XXX after previous renaming. Signed-off-by: Wen Congyang Signed-o

[Xen-devel] [PATCH v8 05/13] tools/libxc: support to resume uncooperative HVM guests

2016-02-17 Thread Wen Congyang
low path to resume the guest. While resuming HVM using slow path is not supported currently, this patch is to make the resume call to not fail. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Reviewed-by: Konrad Rzeszutek Wilk --- tools/libxc/xc_resume.c | 25 +

[Xen-devel] [PATCH v8 11/13] tools/libxl: store remus_ops in checkpoint device state

2016-02-17 Thread Wen Congyang
checkpoint device layer will seperate remus from checkpoint device layer. We use remus ops directly in checkpoint device. Store it in checkpoint device state so that we do not aware of remus_ops in the checkpoint device layer. It is pure refactoring and no functional changes. Signed-off-by: Wen Congyang

Re: [Xen-devel] [PATCH v7 05/18] tools/libxc: support to resume uncooperative HVM guests

2016-02-03 Thread Wen Congyang
On 02/04/2016 03:40 AM, Wei Liu wrote: > On Fri, Jan 29, 2016 at 01:27:21PM +0800, Wen Congyang wrote: >> Before this patch: >> 1. suspend >> a. PVHVM and PV: we use the same way to suspend the guest (send the suspend >>request to the guest). If the guest doesn

Re: [Xen-devel] [PATCH v7 12/18] tools/libx{l, c}: add back channel to libxc

2016-02-03 Thread Wen Congyang
On 02/04/2016 03:40 AM, Wei Liu wrote: > On Fri, Jan 29, 2016 at 01:27:28PM +0800, Wen Congyang wrote: >> In COLO mode, both VMs are running, and are considered in sync if the >> visible network traffic is identical. After some time, they fall out of >> sync. >> >>

Re: [Xen-devel] [PATCH v7 08/18] tools/libxl: introduce libxl__domain_restore_device_model to load qemu state

2016-02-03 Thread Wen Congyang
On 02/04/2016 03:40 AM, Wei Liu wrote: > On Fri, Jan 29, 2016 at 01:27:24PM +0800, Wen Congyang wrote: >> In normal migration, the qemu state is passed to qemu as a parameter. >> With COLO, secondary vm is running. So we will do the following steps >> at every checkpoin

Re: [Xen-devel] [PATCH v7 07/18] migration/save: pass checkpointed_stream from libxl to libxc

2016-02-03 Thread Wen Congyang
On 02/04/2016 03:40 AM, Wei Liu wrote: > On Fri, Jan 29, 2016 at 01:27:23PM +0800, Wen Congyang wrote: >> Pass checkpointed_stream from libxl to libxc. >> It won't affact legacy migration because legacy migration >> won't use this param. >> >> Signed-o

Re: [Xen-devel] [PATCH v7 01/18] libxl/remus: init checkpoint_callback in Remus setup callback

2016-02-03 Thread Wen Congyang
On 02/04/2016 03:39 AM, Wei Liu wrote: > On Fri, Jan 29, 2016 at 01:27:17PM +0800, Wen Congyang wrote: >> init stream {read/write} state checkpoint_callback in Remus setup callback. >> There's no functional change, it's just refactoring so that we can move >&g

Re: [Xen-devel] [PATCH v7 12/18] tools/libx{l, c}: add back channel to libxc

2016-01-31 Thread Wen Congyang
On 01/30/2016 12:38 AM, Konrad Rzeszutek Wilk wrote: > On Fri, Jan 29, 2016 at 01:27:28PM +0800, Wen Congyang wrote: >> In COLO mode, both VMs are running, and are considered in sync if the >> visible network traffic is identical. After some time, they fall out of >> sync

[Xen-devel] [PATCH v7 09/18] tools/libxl: introduce libxl__domain_common_switch_qemu_logdirty()

2016-01-28 Thread Wen Congyang
s lds->callback before exits. This new API will be used by the patch: secondary vm suspend/resume/checkpoint code Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Andrew Cooper Acked-by: Ian Campbell --- tools/libxl/libxl_dom_save.c | 95 ---

[Xen-devel] [PATCH v7 16/18] tools/libxl: store remus_ops in checkpoint device state

2016-01-28 Thread Wen Congyang
checkpoint device layer will seperate remus from checkpoint device layer. We use remus ops directly in checkpoint device. Store it in checkpoint device state so that we do not aware of remus_ops in the checkpoint device layer. It is pure refactoring and no functional changes. Signed-off-by: Wen Congyang

  1   2   3   4   >