[Qemu-devel] [PATCH V6 14/17] filter: Add handle_event method for NetFilterClass

2018-03-10 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 28 3 files changed, 50 insertions(+)

[Qemu-devel] [PATCH V6 15/17] filter-rewriter: handle checkpoint and failover event

2018-03-10 Thread Zhang Chen
After one round of checkpoint, the states between PVM and SVM become consistent, so it is unnecessary to adjust the sequence of net packets for old connections, besides, while failover happens, filter-rewriter needs to check if it still needs to adjust sequence of net packets. Signed-off-by: zhang

[Qemu-devel] [PATCH V6 12/17] savevm: split the process of different stages for loadvm/savevm

2018-03-10 Thread Zhang Chen
From: zhanghailiang There are several stages during loadvm/savevm process. In different stage, migration incoming processes different types of sections. We want to control these stages more accuracy, it will benefit COLO performance, we don't have to save type of QEMU_VM_SECTION_START sections ev

[Qemu-devel] [PATCH V6 13/17] COLO: flush host dirty ram from cache

2018-03-10 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- migration/ram.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/r

[Qemu-devel] [PATCH V6 11/17] qapi: Add new command to query colo status

2018-03-10 Thread Zhang Chen
Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen --- migration/colo.c| 31 +++ qapi/migration.json | 33 + 2

[Qemu-devel] [PATCH V6 04/17] COLO: integrate colo compare with colo frame

2018-03-10 Thread Zhang Chen
For COLO FT, both the PVM and SVM run at the same time, only sync the state while it needs. So here, let SVM runs while not doing checkpoint, change DEFAULT_MIGRATE_X_CHECKPOINT_DELAY to 200*100. Besides, we forgot to release colo_checkpoint_semd and colo_delay_timer, fix them here. Signed-off-b

[Qemu-devel] [PATCH V6 10/17] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-03-10 Thread Zhang Chen
From: zhanghailiang If some errors happen during VM's COLO FT stage, it's important to notify the users of this event. Together with 'x-colo-lost-heartbeat', Users can intervene in COLO's failover work immediately. If users don't want to get involved in COLO's failover verdict, it is still necess

[Qemu-devel] [PATCH V6 09/17] COLO: Flush memory data from ram cache

2018-03-10 Thread Zhang Chen
From: zhanghailiang During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all

[Qemu-devel] [PATCH V6 05/17] COLO: Add block replication into colo process

2018-03-10 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- migration/colo.c | 43 +

[Qemu-devel] [PATCH V6 07/17] COLO: Load dirty pages into SVM's RAM cache firstly

2018-03-10 Thread Zhang Chen
From: zhanghailiang We should not load PVM's state directly into SVM, because there maybe some errors happen when SVM is receving data, which will break SVM. We need to ensure receving all data before load the state into SVM. We use an extra memory to cache these data (PVM's ram). The ram cache

[Qemu-devel] [PATCH V6 02/17] colo-compare: implement the process of checkpoint

2018-03-10 Thread Zhang Chen
While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- include/migra

[Qemu-devel] [PATCH V6 08/17] ram/COLO: Record the dirty pages that SVM received

2018-03-10 Thread Zhang Chen
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the

[Qemu-devel] [PATCH V6 01/17] filter-rewriter: fix memory leak for connection in connection_track_table

2018-03-10 Thread Zhang Chen
After a net connection is closed, we didn't clear its releated resources in connection_track_table, which will lead to memory leak. Let't track the state of net connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- net

[Qemu-devel] [PATCH V6 06/17] COLO: Remove colo_state migration struct

2018-03-10 Thread Zhang Chen
We need to know if migration is going into COLO state for incoming side before start normal migration. Instead by using the VMStateDescription to send colo_state from source side to destination side, we use MIG_CMD_ENABLE_COLO to indicate whether COLO is enabled or not. Signed-off-by: zhanghailia

[Qemu-devel] [PATCH V6 00/17] COLO: integrate colo frame with block replication and COLO proxy

2018-03-10 Thread Zhang Chen
Hi~ All~ COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror, filter-redirector,filter-rewriter) have been exist in qemu for long time, it's time to integrate these three parts to make COLO really works. In this series, we have some optimizations for COLO frame, including sep

[Qemu-devel] [PATCH V6 03/17] colo-compare: use notifier to notify packets comparing result

2018-03-10 Thread Zhang Chen
It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 32 +--- net/colo-compare.h | 2 ++ 2 files changed, 27 insertions(+), 7 deletions(-) diff --git

[Qemu-devel] [PATCH V6 00/17] COLO: integrate colo frame with block replication and COLO proxy

2018-03-10 Thread Zhang Chen
Hi~ COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror, filter-redirector,filter-rewriter) have been exist in qemu for long time, it's time to integrate these three parts to make COLO really works. In this series, we have some optimizations for COLO frame, including separati

[Qemu-devel] qemu 2.10.2 : util/qemu-thread-posix.c: 64: qemu_mutex_lock: Assertion 'mutex->initialized' failed

2018-03-10 Thread Eva Chen
Hello, I am working on compiling QEMU 2.10.2 to a static library. Everything works fine in compiling QEMU to an execution file(qemu-system-aarch64), but after I link all .o and .a file to create a QEMU library and write a c file to call qemu_main() in qemu/util/vl.c(I have changed the name "main(

Re: [Qemu-devel] [PATCH v6 0/2] block latency histogram

2018-03-10 Thread Eric Blake
On 03/09/2018 10:52 AM, Vladimir Sementsov-Ogievskiy wrote: v6: Use correct header qapi/qapi-builtin-types.h, to fix build again. Sorry for spam =( Vladimir Sementsov-Ogievskiy (2): block/accounting: introduce latency histogram qapi: add block latency histogram interface This touche

Re: [Qemu-devel] [PATCH v6 2/2] qapi: add block latency histogram interface

2018-03-10 Thread Eric Blake
On 03/09/2018 10:52 AM, Vladimir Sementsov-Ogievskiy wrote: Set (and clear) histogram through new command block-latency-histogram-set and show new statistics in query-blockstats results. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 111 +++

Re: [Qemu-devel] [PATCH v6 1/2] block/accounting: introduce latency histogram

2018-03-10 Thread Eric Blake
On 03/09/2018 10:52 AM, Vladimir Sementsov-Ogievskiy wrote: Introduce latency histogram statics for block devices. For each accounted operation type latency region [0, +inf) is Hard to read; I suggest: s/type latency/type, the latency/ divided into subregions by several points. Then, calculat

Re: [Qemu-devel] [PATCH v4] qapi: Add device ID and head parameters to screendump

2018-03-10 Thread Eric Blake
On 03/05/2018 10:37 AM, Thomas Huth wrote: QEMU's screendump command can only take dumps from the primary display. When using multiple VGA cards, there is no way to get a dump from a secondary card or other display heads yet. So let's add an 'device' and a 'head' parameter to the HMP and QMP comm

[Qemu-devel] [PATCH v6 5/5] test/acpi-test-data: add ACPI tables for dimmpxm test

2018-03-10 Thread Haozhong Zhang
Reviewers can use ACPI tables in this patch to run test_acpi_{piix4,q35}_tcg_dimm_pxm cases. Signed-off-by: Haozhong Zhang --- tests/acpi-test-data/pc/APIC.dimmpxm | Bin 0 -> 144 bytes tests/acpi-test-data/pc/DSDT.dimmpxm | Bin 0 -> 6803 bytes tests/acpi-test-data/pc/NFIT.dimmpxm | Bin 0 ->

[Qemu-devel] [PATCH v6 0/5] hw/acpi-build: build SRAT memory affinity structures for DIMM devices

2018-03-10 Thread Haozhong Zhang
(Patch 5 is only for reviewers to run test cases in patch 4) ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity domain of a NVDIMM SPA range must match with corresponding entry in SRAT table. The address ranges of vNVDIMM in QEMU are allocated from the hot-pluggable address space,

[Qemu-devel] [PATCH v6 4/5] tests/bios-tables-test: add test cases for DIMM proximity

2018-03-10 Thread Haozhong Zhang
QEMU now builds one SRAT memory affinity structure for each PC-DIMM and NVDIMM device presented at boot time with the proximity domain specified in the device option 'node', rather than only one SRAT memory affinity structure covering the entire hotpluggable address space with the proximity domain

[Qemu-devel] [PATCH v6 3/5] hw/acpi-build: build SRAT memory affinity structures for DIMM devices

2018-03-10 Thread Haozhong Zhang
ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity domain of a NVDIMM SPA range must match with corresponding entry in SRAT table. The address ranges of vNVDIMM in QEMU are allocated from the hot-pluggable address space, which is entirely covered by one SRAT memory affinity structu

[Qemu-devel] [PATCH v6 1/5] pc-dimm: make qmp_pc_dimm_device_list() sort devices by address

2018-03-10 Thread Haozhong Zhang
Make qmp_pc_dimm_device_list() return sorted by start address list of devices so that it could be reused in places that would need sorted list*. Reuse existing pc_dimm_built_list() to get sorted list. While at it hide recursive callbacks from callers, so that: qmp_pc_dimm_device_list(qdev_get_m

Re: [Qemu-devel] [PATCH v5 2/5] qmp: distinguish PC-DIMM and NVDIMM in MemoryDeviceInfoList

2018-03-10 Thread Haozhong Zhang
On 03/10/18 20:31 -0600, Eric Blake wrote: > On 03/10/2018 07:34 PM, Haozhong Zhang wrote: > > It may need to treat PC-DIMM and NVDIMM differently, e.g., when > > deciding the necessity of non-volatile flag bit in SRAT memory > > affinity structures. > > > > NVDIMMDeviceInfo, which inherits from P

[Qemu-devel] [PATCH v6 2/5] qmp: distinguish PC-DIMM and NVDIMM in MemoryDeviceInfoList

2018-03-10 Thread Haozhong Zhang
It may need to treat PC-DIMM and NVDIMM differently, e.g., when deciding the necessity of non-volatile flag bit in SRAT memory affinity structures. A new field 'nvdimm' is added to the union type MemoryDeviceInfo for such purpose. Its type is currently PCDIMMDeviceInfo and will be updated when nec

Re: [Qemu-devel] [PATCH v8 00/23] QMP: out-of-band (OOB) execution support

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: Based-on: <20180306053320.15401-1-pet...@redhat.com> The series is based on the following series: [PATCH v2 0/9] chardev: qio related non-default context support (actually patch 9 is based on version 2.1, Dan's version) This version only contains s

Re: [Qemu-devel] [PATCH v8 23/23] tests: qmp-test: add oob test

2018-03-10 Thread Eric Blake
On 03/09/2018 03:00 AM, Peter Xu wrote: Test the new OOB capability. Here we used the new "x-oob-test" command. Firstly, we send a lock=true and oob=false command to hang the main s/Firstly/First/ thread. Then send another lock=false and oob=true command (which will be run inside parser thi

Re: [Qemu-devel] [PATCH v8 22/23] tests: qmp-test: verify command batching

2018-03-10 Thread Eric Blake
On 03/09/2018 03:00 AM, Peter Xu wrote: OOB introduced DROP event for flow control. This should not affect old QMP clients. Add a command batching check to make sure of it. Reviewed-by: Stefan Hajnoczi Signed-off-by: Peter Xu --- tests/qmp-test.c | 22 ++ 1 file change

Re: [Qemu-devel] [PATCH v8 21/23] qmp: add command "x-oob-test"

2018-03-10 Thread Eric Blake
On 03/09/2018 03:00 AM, Peter Xu wrote: This command is only used to test OOB functionality. It should not be used for any other purposes. Reviewed-by: Stefan Hajnoczi Reviewed-by: Fam Zheng Signed-off-by: Peter Xu --- qapi/misc.json | 18 ++ qmp.c | 16 ++

Re: [Qemu-devel] [PATCH v8 18/23] qmp: support out-of-band (oob) execution

2018-03-10 Thread Eric Blake
On 03/09/2018 03:00 AM, Peter Xu wrote: Having "allow-oob" to true for a command does not mean that this command s/to/:/ will always be run in out-of-band mode. The out-of-band quick path will only be executed if we specify the extra "run-oob" flag when sending the QMP request: { "exec

Re: [Qemu-devel] [PATCH v5 2/5] qmp: distinguish PC-DIMM and NVDIMM in MemoryDeviceInfoList

2018-03-10 Thread Eric Blake
On 03/10/2018 07:34 PM, Haozhong Zhang wrote: It may need to treat PC-DIMM and NVDIMM differently, e.g., when deciding the necessity of non-volatile flag bit in SRAT memory affinity structures. NVDIMMDeviceInfo, which inherits from PCDIMMDeviceInfo, is added to union type MemoryDeviceInfo to rec

Re: [Qemu-devel] [PATCH v8 17/23] qapi: introduce new cmd option "allow-oob"

2018-03-10 Thread Eric Blake
On 03/09/2018 03:00 AM, Peter Xu wrote: Here "oob" stands for "Out-Of-Band". When "allow-oob" is set, it means the command allows out-of-band execution. The "oob" idea is proposed by Markus Armbruster in following thread: https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg02057.html

Re: [Qemu-devel] [PATCH v8 16/23] monitor: send event when command queue full

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: Set maximum QMP command queue length to 8. If queue full, instead of If the queue is full queue the command, we directly return a "command-dropped" event, telling s/queue/queuing/ client that specific command is dropped. s/client that/the client

Re: [Qemu-devel] [PATCH v8 15/23] qmp: add new event "command-dropped"

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: This event will be emitted if one QMP command is dropped. Along, s/Along,/Also,/ declare an enum for the reasons. Reviewed-by: Fam Zheng Reviewed-by: Stefan Hajnoczi Signed-off-by: Peter Xu --- qapi/misc.json | 37

Re: [Qemu-devel] [PATCH v8 14/23] monitor: separate QMP parser and dispatcher

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: Originally QMP goes through these steps: JSON Parser --> QMP Dispatcher --> Respond /|\(2)(3) | (1) | \|/ (4) +- main thread + This patch does this: JSON Par

Re: [Qemu-devel] [PATCH v8 13/23] monitor: let suspend/resume work even with QMPs

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: This patches allows QMP monitors to be suspended/resumed. One thing to mention is that for QMPs that are using IOThreads, we need an explicit kick for the IOThread in case it is sleeping. Meanwhile, we need to take special care on non-interactive HMPs. Cu

[Qemu-devel] [PATCH v5 5/5][DO NOT APPLY] test/acpi-test-data: add ACPI tables for dimmpxm test

2018-03-10 Thread Haozhong Zhang
Reviewers can use ACPI tables in this patch to run test_acpi_{piix4,q35}_tcg_dimm_pxm cases. Signed-off-by: Haozhong Zhang --- tests/acpi-test-data/pc/APIC.dimmpxm | Bin 0 -> 144 bytes tests/acpi-test-data/pc/DSDT.dimmpxm | Bin 0 -> 6803 bytes tests/acpi-test-data/pc/NFIT.dimmpxm | Bin 0 ->

[Qemu-devel] [PATCH v5 4/5] tests/bios-tables-test: add test cases for DIMM proximity

2018-03-10 Thread Haozhong Zhang
QEMU now builds one SRAT memory affinity structure for each PC-DIMM and NVDIMM device presented at boot time with the proximity domain specified in the device option 'node', rather than only one SRAT memory affinity structure covering the entire hotpluggable address space with the proximity domain

Re: [Qemu-devel] [PATCH v8 11/23] monitor: introduce monitor_qmp_respond()

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: A tiny refactoring, preparing to split the QMP dispatcher away. Reviewed-by: Fam Zheng Reviewed-by: Stefan Hajnoczi Signed-off-by: Peter Xu --- monitor.c | 50 +- 1 file changed, 33 insertions(+), 17 de

[Qemu-devel] [PATCH v5 3/5] hw/acpi-build: build SRAT memory affinity structures for DIMM devices

2018-03-10 Thread Haozhong Zhang
ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity domain of a NVDIMM SPA range must match with corresponding entry in SRAT table. The address ranges of vNVDIMM in QEMU are allocated from the hot-pluggable address space, which is entirely covered by one SRAT memory affinity structu

[Qemu-devel] [PATCH v5 1/5] pc-dimm: make qmp_pc_dimm_device_list() sort devices by address

2018-03-10 Thread Haozhong Zhang
Make qmp_pc_dimm_device_list() return sorted by start address list of devices so that it could be reused in places that would need sorted list*. Reuse existing pc_dimm_built_list() to get sorted list. While at it hide recursive callbacks from callers, so that: qmp_pc_dimm_device_list(qdev_get_m

[Qemu-devel] [PATCH v5 0/5] hw/acpi-build: build SRAT memory affinity structures for DIMM devices

2018-03-10 Thread Haozhong Zhang
(Patch 5 is only for reviewers to run test cases in patch 4) ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity domain of a NVDIMM SPA range must match with corresponding entry in SRAT table. The address ranges of vNVDIMM in QEMU are allocated from the hot-pluggable address space,

[Qemu-devel] [PATCH v5 2/5] qmp: distinguish PC-DIMM and NVDIMM in MemoryDeviceInfoList

2018-03-10 Thread Haozhong Zhang
It may need to treat PC-DIMM and NVDIMM differently, e.g., when deciding the necessity of non-volatile flag bit in SRAT memory affinity structures. NVDIMMDeviceInfo, which inherits from PCDIMMDeviceInfo, is added to union type MemoryDeviceInfo to record information of NVDIMM devices. The NVDIMM-sp

Re: [Qemu-devel] [PATCH v8 10/23] qmp: introduce QMPCapability

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: There were no QMP capabilities defined. Define the first "oob" as capability to allow out-of-band messages. Reads better as "Define the first capability, "oob", to allow..." After this patch, we will allow QMP clients to enable QMP capabilities when s

Re: [Qemu-devel] [PATCH v8 09/23] monitor: allow using IO thread for parsing

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: For each Monitor, add one field "use_io_thr" to show whether it will be using the dedicated monitor IO thread to handle input/output. When set, monitor IO parsing work will be offloaded to the dedicated monitor IO thread, rather than the original main loop

Re: [Qemu-devel] [PATCH v8 06/23] monitor: move the cur_mon hack deeper for QMP

2018-03-10 Thread Eric Blake
On 03/09/2018 02:59 AM, Peter Xu wrote: In monitor_qmp_read(), we have the hack to temporarily replace the cur_mon pointer. Now we move this hack deeper inside the QMP dispatcher routine since the Monitor pointer can be actually obtained using container_of() upon the parser object, just like mos

[Qemu-devel] [PATCH v3] fsl-imx6: Swap Ethernet interrupt defines

2018-03-10 Thread Guenter Roeck
The sabrelite machine model used by qemu-system-arm is based on the Freescale/NXP i.MX6Q processor. This SoC has an on-board ethernet controller which is supported in QEMU using the imx_fec.c module (actually called imx.enet for this model.) The include/hw/arm/fsm-imx6.h file defines the interrupt

Re: [Qemu-devel] [PATCH v3 0/7] block: Handle null backing link

2018-03-10 Thread Eric Blake
On 02/24/2018 09:40 AM, Max Reitz wrote: Currently, we try to rewrite every occurrence of "backing": null into "backing": "" in qmp_blockdev_add(). However, that breaks using the same "backing": null construction in json:{} file names (which do not go through qmp_blockdev_add()). Currently, the

Re: [Qemu-devel] [PATCH v3 3/7] qapi: Replace qobject_to_X(o) by qobject_to(o, X)

2018-03-10 Thread Eric Blake
On 02/28/2018 12:08 PM, Max Reitz wrote: +    options = qobject_to(options_obj, QDict); Bikeshedding - would it read any easier as: options = qobject_to(QDict, options_obj); So at this point, I'm 70:30 in favor of doing the rename to have qobject_to(type, obj) for consistency with majorit

[Qemu-devel] [PATCH] iotests: 163 is not quick

2018-03-10 Thread Eric Blake
Testing on ext4, most 'quick' qcow2 tests took less than 5 seconds, but 163 took more than 20. Let's remove it from the quick set. Signed-off-by: Eric Blake --- tests/qemu-iotests/group | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/group b/tests/qemu-iot

Re: [Qemu-devel] [PATCH v7 00/23] Migration: postcopy failure recovery

2018-03-10 Thread no-reply
Hi, This series failed build test on s390x host. Please find the details below. Type: series Message-id: 20180309091535.13315-1-pet...@redhat.com Subject: [Qemu-devel] [PATCH v7 00/23] Migration: postcopy failure recovery === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked und

Re: [Qemu-devel] [PATCH v2] RISC-V: Fix riscv_isa_string, use popcount to count bits

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 10:01 PM, Michael Clark wrote: > Logic bug caused the string size calculation for the RISC-V > format ISA string to be small. This fix allows slack for rv128. > > Cc: Palmer Dabbelt > Cc: Peter Maydell > Cc: Eric Blake > Signed-off-by: Michael Clark > --- > target/riscv/cpu.c |

Re: [Qemu-devel] [PATCH V5 11/17] qapi: Add new command to query colo status

2018-03-10 Thread Eric Blake
On 03/10/2018 03:22 PM, Zhang Chen wrote: Libvirt or other high level sofware can use this command query colo status. s/sofware/software/ You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen --- migration/colo.c| 35 +++

[Qemu-devel] [PATCH V5 13/17] COLO: flush host dirty ram from cache

2018-03-10 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- migration/ram.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/r

[Qemu-devel] [PATCH V5 16/17] COLO: notify net filters about checkpoint/failover event

2018-03-10 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang --- migration/colo.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 6834f3d..5fb4d8c 100644 --- a/migration/colo.c +++ b

[Qemu-devel] [PATCH V5 17/17] COLO: quick failover process by kick COLO thread

2018-03-10 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang --- migration/colo.c | 8 1 file changed, 8 insertions(+) diff --git a/migration/colo.c b/migra

[Qemu-devel] [PATCH V5 11/17] qapi: Add new command to query colo status

2018-03-10 Thread Zhang Chen
Libvirt or other high level sofware can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen --- migration/colo.c| 35 +++ qapi/migration.json | 35 ++

[Qemu-devel] [PATCH V5 09/17] COLO: Flush memory data from ram cache

2018-03-10 Thread Zhang Chen
From: zhanghailiang During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all

[Qemu-devel] [PATCH V5 10/17] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-03-10 Thread Zhang Chen
From: zhanghailiang If some errors happen during VM's COLO FT stage, it's important to notify the users of this event. Together with 'x-colo-lost-heartbeat', Users can intervene in COLO's failover work immediately. If users don't want to get involved in COLO's failover verdict, it is still necess

[Qemu-devel] [PATCH V5 08/17] ram/COLO: Record the dirty pages that SVM received

2018-03-10 Thread Zhang Chen
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the

[Qemu-devel] [PATCH V5 15/17] filter-rewriter: handle checkpoint and failover event

2018-03-10 Thread Zhang Chen
After one round of checkpoint, the states between PVM and SVM become consistent, so it is unnecessary to adjust the sequence of net packets for old connections, besides, while failover happens, filter-rewriter needs to check if it still needs to adjust sequence of net packets. Signed-off-by: zhang

[Qemu-devel] [PATCH V5 14/17] filter: Add handle_event method for NetFilterClass

2018-03-10 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 28 3 files changed, 50 insertions(+)

[Qemu-devel] [PATCH V5 07/17] COLO: Load dirty pages into SVM's RAM cache firstly

2018-03-10 Thread Zhang Chen
From: zhanghailiang We should not load PVM's state directly into SVM, because there maybe some errors happen when SVM is receving data, which will break SVM. We need to ensure receving all data before load the state into SVM. We use an extra memory to cache these data (PVM's ram). The ram cache

[Qemu-devel] [PATCH V5 12/17] savevm: split the process of different stages for loadvm/savevm

2018-03-10 Thread Zhang Chen
From: zhanghailiang There are several stages during loadvm/savevm process. In different stage, migration incoming processes different types of sections. We want to control these stages more accuracy, it will benefit COLO performance, we don't have to save type of QEMU_VM_SECTION_START sections ev

[Qemu-devel] [PATCH V5 05/17] COLO: Add block replication into colo process

2018-03-10 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- migration/colo.c | 43 +

[Qemu-devel] [PATCH V5 04/17] COLO: integrate colo compare with colo frame

2018-03-10 Thread Zhang Chen
For COLO FT, both the PVM and SVM run at the same time, only sync the state while it needs. So here, let SVM runs while not doing checkpoint, change DEFAULT_MIGRATE_X_CHECKPOINT_DELAY to 200*100. Besides, we forgot to release colo_checkpoint_semd and colo_delay_timer, fix them here. Signed-off-b

[Qemu-devel] [PATCH V5 02/17] colo-compare: implement the process of checkpoint

2018-03-10 Thread Zhang Chen
While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- include/migra

[Qemu-devel] [PATCH V5 01/17] filter-rewriter: fix memory leak for connection in connection_track_table

2018-03-10 Thread Zhang Chen
After a net connection is closed, we didn't clear its releated resources in connection_track_table, which will lead to memory leak. Let't track the state of net connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- net

[Qemu-devel] [PATCH V5 03/17] colo-compare: use notifier to notify packets comparing result

2018-03-10 Thread Zhang Chen
It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 32 +--- net/colo-compare.h | 2 ++ 2 files changed, 27 insertions(+), 7 deletions(-) diff --git

[Qemu-devel] [PATCH V5 00/17] COLO: integrate colo frame with block replication and COLO proxy

2018-03-10 Thread Zhang Chen
Hi~ COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror, filter-redirector,filter-rewriter) have been exist in qemu for long time, it's time to integrate these three parts to make COLO really works. In this series, we have some optimizations for COLO frame, including separati

[Qemu-devel] [PATCH V5 06/17] COLO: Remove colo_state migration struct

2018-03-10 Thread Zhang Chen
We need to know if migration is going into COLO state for incoming side before start normal migration. Instead by using the VMStateDescription to send colo_state from source side to destination side, we use MIG_CMD_ENABLE_COLO to indicate whether COLO is enabled or not. Signed-off-by: zhanghailia

Re: [Qemu-devel] [PATCH v2 10/23] RISC-V: Hold rcu_read_lock when accessing memory

2018-03-10 Thread Philippe Mathieu-Daudé
Hi Michael, On 03/09/2018 05:12 AM, Michael Clark wrote: > From reading other code that accesses memory regions directly, > it appears that the rcu_read_lock needs to be held. Note: the > original code for accessing RAM directly was added because > there is no other way to use atomic_cmpxchg on gu

Re: [Qemu-devel] [PATCH v2 15/23] RISC-V: Use memory_region_is_ram in pte update

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 05:12 AM, Michael Clark wrote: > After reading cpu_physical_memory_write and friends, it seems > that memory_region_is_ram is a more appropriate interface, > and matches the intent of the code that is calling it. > > Cc: Sagar Karandikar > Cc: Bastian Koppelmann > Signed-off-by: Mi

Re: [Qemu-devel] [PATCH v2 04/23] RISC-V: Use ROM base address and size from memmap

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 05:12 AM, Michael Clark wrote: > Another case of replaceing hard coded constants, this time > referring to the definition in the virt machine's memmap. > > Cc: Sagar Karandikar > Cc: Bastian Koppelmann > Signed-off-by: Michael Clark > Signed-off-by: Palmer Dabbelt Reviewed-by: P

Re: [Qemu-devel] [PATCH v2 03/23] RISC-V: Make virt board description match spike

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 05:12 AM, Michael Clark wrote: > This makes 'qemu-system-riscv64 -machine help' output more tidy > and consistent. > > Cc: Sagar Karandikar > Cc: Bastian Koppelmann > Signed-off-by: Michael Clark > Signed-off-by: Palmer Dabbelt Reviewed-by: Philippe Mathieu-Daudé > --- > hw/r

Re: [Qemu-devel] [PATCH v2 01/23] RISC-V: Make virt create_fdt interface consistent

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 05:12 AM, Michael Clark wrote: > create_fdt sets the fdt variable on RISCVVirtState and this is > used to access the fdt. This reverts a change introduced in > https://github.com/riscv/riscv-qemu/pull/109 which introduced > a redundant return value, overlooking the RISCVVirtState > st

Re: [Qemu-devel] [PATCH v2 16/23] RISC-V: Remove EM_RISCV ELF_MACHINE indirection

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 05:12 AM, Michael Clark wrote: > Pointless indirection. Other ports use EM_ constants directly. > > Cc: Sagar Karandikar > Cc: Bastian Koppelmann > Signed-off-by: Michael Clark > Signed-off-by: Palmer Dabbelt Reviewed-by: Philippe Mathieu-Daudé > --- > hw/riscv/sifive_e.c | 2

Re: [Qemu-devel] [PATCH v2 14/23] RISC-V: Make virt header comment title consistent

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 05:12 AM, Michael Clark wrote: > Cc: Sagar Karandikar > Cc: Bastian Koppelmann > Signed-off-by: Michael Clark > Signed-off-by: Palmer Dabbelt Reviewed-by: Philippe Mathieu-Daudé > --- > include/hw/riscv/virt.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff -

Re: [Qemu-devel] [PATCH v2 13/23] RISC-V: Make some header guards more specific

2018-03-10 Thread Philippe Mathieu-Daudé
On 03/09/2018 05:12 AM, Michael Clark wrote: > Cc: Sagar Karandikar > Cc: Bastian Koppelmann > Signed-off-by: Michael Clark > Signed-off-by: Palmer Dabbelt Reviewed-by: Philippe Mathieu-Daudé > --- > include/hw/riscv/spike.h | 4 ++-- > include/hw/riscv/virt.h | 4 ++-- > 2 files changed,

Re: [Qemu-devel] [PATCH v2 23/23] RISC-V: Convert cpu definition towards future model

2018-03-10 Thread Philippe Mathieu-Daudé
Hi Michael, On 03/09/2018 05:12 AM, Michael Clark wrote: > - Model borrowed from target/sh4/cpu.c > - Rewrote riscv_cpu_list to use object_class_get_list > - Dropped 'struct RISCVCPUInfo' and used TypeInfo array > - Replaced riscv_cpu_register_types with DEFINE_TYPES > - Marked base class as abst

Re: [Qemu-devel] [PATCH v2] fsl-imx6: Swap Ethernet interrupt defines

2018-03-10 Thread Peter Maydell
On 9 March 2018 at 21:47, Guenter Roeck wrote: > The sabrelite machine model used by qemu-system-arm is based on the > Freescale/NXP i.MX6Q processor. This SoC has an on-board ethernet > controller which is supported in QEMU using the imx_fec.c module > (actually called imx.enet for this model.) >

Re: [Qemu-devel] [PULL] RISC-V: Fix riscv_isa_string g_new0 size calculation

2018-03-10 Thread Peter Maydell
On 9 March 2018 at 20:54, Michael Clark wrote: > Apologies for jumping the gun again with a PR before a review. It was most > likely because I thought it was a critical bug fix. > > I'm incorporating Eric Blake's feedback. For one-off "this is a build fix" bugfix patches, the simplest thing is ju

[Qemu-devel] [PATCH] migration: Fix rate limiting issue on RDMA migration

2018-03-10 Thread Lidong Chen
RDMA migration implement save_page function for QEMUFile, but ram_control_save_page do not increase bytes_xfer. So when doing RDMA migration, it will use whole bandwidth. Signed-off-by: Lidong Chen --- migration/qemu-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migr

Re: [Qemu-devel] [PATCH v3 0/2] block latency histogram

2018-03-10 Thread no-reply
Hi, This series failed docker-quick@centos6 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180309144918.44975-1-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH v3 0/2] bloc

Re: [Qemu-devel] [PATCH v8 19/23] SiFive RISC-V UART Device

2018-03-10 Thread Mark Cave-Ayland
On 10/03/18 03:02, Michael Clark wrote: On Sat, Mar 10, 2018 at 1:39 AM, Philippe Mathieu-Daudé wrote: On 03/02/2018 02:51 PM, Michael Clark wrote: QEMU model of the UART on the SiFive E300 and U500 series SOCs. BBL supports the SiFive UART for early console access via the SBI (Supervisor Bi

Re: [Qemu-devel] [PATCH 02/25] hw/dma/i8257: Rename DMA_init() to i8257_dma_init()

2018-03-10 Thread Mark Cave-Ayland
On 09/03/18 20:51, Artyom Tarasenko wrote: On Fri, Mar 9, 2018 at 1:19 PM, Mark Cave-Ayland wrote: On 09/03/18 10:43, Philippe Mathieu-Daudé wrote: Hi Mark, On 03/09/2018 11:32 AM, Mark Cave-Ayland wrote: On 08/03/18 22:39, Philippe Mathieu-Daudé wrote: - Move the header from hw/isa/ to

[Qemu-devel] [PATCH v5 06/21] iotests: add pause_wait

2018-03-10 Thread John Snow
Split out the pause command into the actual pause and the wait. Not every usage presently needs to resubmit a pause request. The intent with the next commit will be to explicitly disallow redundant or meaningless pause/resume requests, so the tests need to become more judicious to reflect that. S

[Qemu-devel] [PATCH v5 00/21] blockjobs: add explicit job management

2018-03-10 Thread John Snow
This series seeks to address two distinct but closely related issues concerning the job management API. (1) For jobs that complete when a monitor is not attached and receiving events or notifications, there's no way to discern the job's final return code. Jobs must remain in the query list

[Qemu-devel] [PATCH v5 03/21] Blockjobs: documentation touchup

2018-03-10 Thread John Snow
Trivial; Document what the job creation flags do, and some general tidying. Signed-off-by: John Snow --- include/block/blockjob.h | 8 include/block/blockjob_int.h | 4 +++- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/include/block/blockjob.h b/include/block/block

[Qemu-devel] [PATCH v5 08/21] blockjobs: add ABORTING state

2018-03-10 Thread John Snow
Add a new state ABORTING. This makes transitions from normative states to error states explicit in the STM, and serves as a disambiguation for which states may complete normally when normal end-states (CONCLUDED) are added in future commits. Notably, Paused/Standby jobs do not transition directly

[Qemu-devel] [PATCH v5 21/21] tests/test-blockjob: test cancellations

2018-03-10 Thread John Snow
Whatever the state a blockjob is in, it should be able to be canceled by the block layer. Signed-off-by: John Snow --- tests/test-blockjob.c | 233 +- 1 file changed, 229 insertions(+), 4 deletions(-) diff --git a/tests/test-blockjob.c b/tests/tes

[Qemu-devel] [PATCH v5 14/21] blockjobs: add block_job_txn_apply function

2018-03-10 Thread John Snow
Simply apply a function transaction-wide. A few more uses of this in forthcoming patches. Signed-off-by: John Snow --- blockjob.c | 25 - 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/blockjob.c b/blockjob.c index 0c64fadc6d..7e03824751 100644 --- a/block

[Qemu-devel] [PATCH v5 20/21] iotests: test manual job dismissal

2018-03-10 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/056 | 187 + tests/qemu-iotests/056.out | 4 +- 2 files changed, 189 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index 04f2c3c841..223292175a 100755 --

[Qemu-devel] [PATCH v5 10/21] blockjobs: add NULL state

2018-03-10 Thread John Snow
Add a new state that specifically demarcates when we begin to permanently demolish a job after it has performed all work. This makes the transition explicit in the STM table and highlights conditions under which a job may be demolished. Alongside this state, add a new helper command "block_job_dec

[Qemu-devel] [PATCH v5 15/21] blockjobs: add prepare callback

2018-03-10 Thread John Snow
Some jobs upon finalization may need to perform some work that can still fail. If these jobs are part of a transaction, it's important that these callbacks fail the entire transaction. We allow for a new callback in addition to commit/abort/clean that allows us the opportunity to have fairly late-

[Qemu-devel] [PATCH v5 16/21] blockjobs: add waiting status

2018-03-10 Thread John Snow
For jobs that are stuck waiting on others in a transaction, it would be nice to know that they are no longer "running" in that sense, but instead are waiting on other jobs in the transaction. Jobs that are "waiting" in this sense cannot be meaningfully altered any longer as they have left their ru

  1   2   >