[PULL v3 04/19] block/qcow2: proper locking on bitmap add/remove paths

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy qmp_block_dirty_bitmap_add and do_block_dirty_bitmap_remove do acquire aio context since 0a6c86d024c52b. But this is not enough: we also must lock qcow2 mutex when access in-image metadata. Especially it concerns freeing qcow2 clusters. To achieve this, move qc

[PULL v3 07/19] block/dirty-bitmap: drop BdrvDirtyBitmap.mutex

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy mutex field is just a pointer to bs->dirty_bitmap_mutex, so no needs to store it in BdrvDirtyBitmap when we have bs pointer in it (since previous patch). Drop mutex field. Constantly use bdrv_dirty_bitmaps_lock/unlock in block/dirty-bitmap.c to make it more obv

[PULL v3 02/19] block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy block/dirty-bitmap.c seems to be more appropriate for it and bdrv_remove_persistent_dirty_bitmap already in it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190920082543.23444-2-vsement...@virtuozzo.com Signed-off-by: John

[PULL v3 03/19] block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy It's more comfortable to not deal with local_err. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190920082543.23444-3-vsement...@virtuozzo.com Signed-off-by: John Snow --- block/qcow2.h| 5 ++--- include/bl

[PULL v3 10/19] block: reverse order for reopen commits

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy It's needed to fix reopening qcow2 with bitmaps to RW. Currently it can't work, as qcow2 needs write access to file child, to mark bitmaps in-image with IN_USE flag. But usually children goes after parents in reopen queue and file child is still RO on qcow2 reop

[PULL v3 08/19] block/dirty-bitmap: refactor bdrv_dirty_bitmap_next

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy bdrv_dirty_bitmap_next is always used in same pattern. So, split it into _next and _first, instead of combining two functions into one and add FOR_EACH_DIRTY_BITMAP macro. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190916

[PULL v3 11/19] iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Reopening bitmaps to RW was broken prior to previous commit. Check that it works now. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190927122355.7344-4-vsement...@virtuozzo.com Signed-off-by: John Snow --- tests/qemu-iotests/165 | 57

[PULL v3 06/19] block/dirty-bitmap: add bs link

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Add bs field to BdrvDirtyBitmap structure. Drop BlockDriverState parameter from bitmap APIs where possible. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190916141911.5255-3-vsement...@virtuozzo.com [Rebased on top of block-

[PULL v3 09/19] block: switch reopen queue from QSIMPLEQ to QTAILQ

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy We'll need reverse-foreach in the following commit, QTAILQ support it, so move to QTAILQ. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-id: 20190927122355.7344-2-vsement...@virtuozzo.com Signed-off-by: John Snow --- include/bloc

Re: [PATCH 2/3] hw/timer/xilinx_timer.c: Switch to transaction-based ptimer API

2019-10-17 Thread Alistair Francis
On Thu, Oct 17, 2019 at 6:50 AM Peter Maydell wrote: > > Switch the xilinx_timer code away from bottom-half based ptimers to > the new transaction-based ptimer API. This just requires adding > begin/commit calls around the various places that modify the ptimer > state, and using the new ptimer_in

[PULL v3 15/19] iotests: add test 260 to check bitmap life after snapshot + commit

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190927122355.7344-8-vsement...@virtuozzo.com [Maintainer edit: removed 260 from auto group per Peter Maydell. --js] Signed-off-by: John Snow --- tests/qemu-iotests/260 | 89

[PULL v3 13/19] block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint()

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy The function is unused, drop it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Message-id: 20190927122355.7344-6-vsement...@virtuozzo.com Signed-off-by: John Snow --- block/qcow2.h| 2 -- block/qcow2-bitmap.c | 15 +

[PULL v3 12/19] block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Firstly, no reason to optimize failure path. Then, function name is ambiguous: it checks for readonly and similar things, but someone may think that it will ignore normal bitmaps which was just unchanged, and this is in bad relation with the fact that we should

[PULL v3 16/19] block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy - Correct check for write access to file child, and in correct place (only if we want to write). - Support reopen rw -> rw (which will be used in following commit), for example, !bdrv_dirty_bitmap_readonly() is not a corruption if bitmap is marked IN_USE i

[PULL v3 14/19] block/qcow2-bitmap: do not remove bitmaps on reopen-ro

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy qcow2_reopen_bitmaps_ro wants to store bitmaps and then mark them all readonly. But the latter don't work, as qcow2_store_persistent_dirty_bitmaps removes bitmaps after storing. It's OK for inactivation but bad idea for reopen-ro. And this leads to the following

Re: [PATCH 3/3] hw/dma/xilinx_axidma.c: Switch to transaction-based ptimer API

2019-10-17 Thread Alistair Francis
On Thu, Oct 17, 2019 at 6:53 AM Peter Maydell wrote: > > Switch the xilinx_axidma code away from bottom-half based ptimers to > the new transaction-based ptimer API. This just requires adding > begin/commit calls around the various places that modify the ptimer > state, and using the new ptimer_i

[PULL v3 17/19] qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit

2019-10-17 Thread John Snow
From: Vladimir Sementsov-Ogievskiy The only reason I can imagine for this strange code at the very-end of bdrv_reopen_commit is the fact that bs->read_only updated after calling drv->bdrv_reopen_commit in bdrv_reopen_commit. And in the same time, prior to previous commit, qcow2_reopen_bitmaps_rw

Re: [PATCH 1/3] hw/net/fsl_etsec/etsec.c: Switch to transaction-based ptimer API

2019-10-17 Thread Alistair Francis
On Thu, Oct 17, 2019 at 6:42 AM Peter Maydell wrote: > > Switch the fsl_etsec code away from bottom-half based ptimers to > the new transaction-based ptimer API. This just requires adding > begin/commit calls around the various places that modify the ptimer > state, and using the new ptimer_init(

[PULL v3 19/19] dirty-bitmaps: remove deprecated autoload parameter

2019-10-17 Thread John Snow
This parameter has been deprecated since 2.12.0 and is eligible for removal. Remove this parameter as it is actually completely ignored; let's not give false hope. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20191002232411.29968-1-js...

Re: Python 2 and test/vm/netbsd

2019-10-17 Thread Eduardo Habkost
On Wed, Oct 16, 2019 at 07:41:24PM -0300, Eduardo Habkost wrote: > On Wed, Oct 16, 2019 at 08:11:57AM +0200, Thomas Huth wrote: > > On 16/10/2019 05.00, Eduardo Habkost wrote: > > > On Tue, Sep 17, 2019 at 08:31:40PM -0300, Eduardo Habkost wrote: > > >> On Mon, Jul 01, 2019 at 07:25:27PM -0300, Edu

Re: [PATCH 00/97] Patch Round-up for stable 4.0.1, freeze on 2019-10-10

2019-10-17 Thread Michael Roth
Quoting Philippe Mathieu-Daudé (2019-10-09 09:23:22) > On 10/9/19 4:17 PM, Michael Roth wrote: > > Quoting Philippe Mathieu-Daudé (2019-10-08 08:04:52) > >> Hi Michael, > >> > >> On 10/2/19 1:44 AM, Michael Roth wrote: > >>> Hi everyone, > >>> > >>> The following new patches are queued for QEMU sta

[ANNOUNCE] QEMU 4.0.1 Stable released

2019-10-17 Thread Michael Roth
Hi everyone, I am pleased to announce that the QEMU v4.0.1 stable release is now available: You can grab the tarball from our download page here: https://www.qemu.org/download/#source v4.0.1 is now tagged in the official qemu.git repository, and the stable-4.0 branch has been updated accordin

Re: [PATCH 00/10] image-fuzzer: Port to Python 3

2019-10-17 Thread John Snow
On 10/17/19 5:29 PM, Eduardo Habkost wrote: > On Thu, Oct 17, 2019 at 05:11:29PM -0400, John Snow wrote: >> >> >> On 10/16/19 3:24 PM, Eduardo Habkost wrote: >>> This series ports image-fuzzer to Python 3. >>> >>> Eduardo Habkost (10): >>> image-fuzzer: Open image files in binary mode >>> im

Re: Python 2 and test/vm/netbsd

2019-10-17 Thread Eduardo Habkost
On Thu, Oct 17, 2019 at 07:05:41PM -0300, Eduardo Habkost wrote: > On Wed, Oct 16, 2019 at 07:41:24PM -0300, Eduardo Habkost wrote: > > On Wed, Oct 16, 2019 at 08:11:57AM +0200, Thomas Huth wrote: > > > On 16/10/2019 05.00, Eduardo Habkost wrote: > > > > On Tue, Sep 17, 2019 at 08:31:40PM -0300, Ed

[PATCH v2 01/10] hw/misc/bcm2835_thermal: Add a dummy BCM2835 thermal sensor

2019-10-17 Thread Philippe Mathieu-Daudé
We will soon implement the SYS_timer. This timer is used by Linux in the thermal subsystem, so once available, the subsystem will be enabled and poll the temperature sensors. We need to provide the minimum required to keep Linux booting. Add a dummy thermal sensor returning ~25°C based on: https:/

[PATCH v2 00/10] hw/arm/raspi: Add thermal/timer, improve multicore address space

2019-10-17 Thread Philippe Mathieu-Daudé
From: PhilMD Hi, Some patches from v1 are already merged. This v2 addresses the review comment from v1, and add patches to clean the memory space when using multiple cores. Laurent, if you test U-Boot with this patchset again, do you mind replying with a "Tested-by:" tag? The next patchset is

[PATCH v2 03/10] hw/timer/bcm2835: Add the BCM2835 SYS_timer

2019-10-17 Thread Philippe Mathieu-Daudé
Add the 64-bit free running timer. Do not model the COMPARE register (no IRQ generated). This timer is used by Linux kernel and recently U-Boot: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/clocksource/bcm2835_timer.c?h=v3.7 https://github.com/u-boot/u-boot/blob/v

[PATCH v2 08/10] hw/arm/raspi: Use AddressSpace when using arm_boot::write_secondary_boot

2019-10-17 Thread Philippe Mathieu-Daudé
write_secondary_boot() is used in SMP configurations where the CPU address space might not be the main System Bus. The rom_add_blob_fixed_as() function allow us to specify an address space. Use it to write each boot blob in the corresponding CPU address space. Signed-off-by: Philippe Mathieu-Daudé

[PATCH v2 02/10] hw/arm/bcm2835_peripherals: Use the thermal sensor block

2019-10-17 Thread Philippe Mathieu-Daudé
Map the thermal sensor in the BCM2835 block. Reviewed-by: Alistair Francis Signed-off-by: Philippe Mathieu-Daudé --- hw/arm/bcm2835_peripherals.c | 13 + include/hw/arm/bcm2835_peripherals.h | 2 ++ include/hw/arm/raspi_platform.h | 1 + 3 files changed, 16 insertions

[PATCH v2 05/10] hw/arm/bcm2836: Make the SoC code modular

2019-10-17 Thread Philippe Mathieu-Daudé
This file creates the BCM2836/BCM2837 blocks. The biggest differences with the BCM2838 we are going to add, are the base addresses of the interrupt controller and the peripherals. Add these addresses in the BCM283XInfo structure to make this block more modular. Remove the MCORE_OFFSET offset as it

[PATCH v2 09/10] hw/arm/raspi: Make the board code modular

2019-10-17 Thread Philippe Mathieu-Daudé
Our code currently create the raspi2 (based on the BCM2836) and the raspi3 (on the BCM2837). Similarly, the raspi4 is based on the BCM2838. To be able to add the new board, make the current code more modular: - Dynamically fills the 'board-rev' value - Allow DRAM sizes different than 1 GiB Rename

[PATCH v2 10/10] hw/arm/highbank: Use AddressSpace when using write_secondary_boot()

2019-10-17 Thread Philippe Mathieu-Daudé
write_secondary_boot() is used in SMP configurations where the CPU address space might not be the main System Bus. The rom_add_blob_fixed_as() function allow us to specify an address space. Use it to write each boot blob in the corresponding CPU address space. Signed-off-by: Philippe Mathieu-Daudé

[PATCH v2 04/10] hw/arm/bcm2835_peripherals: Use the SYS_timer

2019-10-17 Thread Philippe Mathieu-Daudé
Connect the recently added SYS_timer. Now U-Boot does not hang anymore polling a free running counter stuck at 0. This timer is also used by the Linux kernel thermal subsystem. Signed-off-by: Philippe Mathieu-Daudé --- v2: Remove spurious error check (Alex) --- hw/arm/bcm2835_peripherals.c

[PATCH v2 06/10] hw/arm/bcm2836: Create VideoCore address space in the SoC

2019-10-17 Thread Philippe Mathieu-Daudé
Currently the VideoCore is created in the Peripheral container as the 'GPU bus'. It is created there because the peripherals using DMA use physical addresses from the VideoCore bus. However the VideoCore is a GPU core placed at the same hierarchical level than the ARM cores. To match the datasheet

[PATCH v2 07/10] hw/arm/bcm2836: Use per CPU address spaces

2019-10-17 Thread Philippe Mathieu-Daudé
Currently all CPUs access the main system bus. Let each CPU have his own address space. Before: address-space: cpu-secure-memory-0 - (prio 0, i/o): system -3fff (prio 0, ram): ram 3f00-3fff (prio

Re: [PATCH] util/cacheinfo: fix crash when compiling with uClibc

2019-10-17 Thread Carlos Santos
On Thu, Oct 17, 2019 at 9:47 AM Peter Maydell wrote: > > On Thu, 17 Oct 2019 at 13:39, wrote: > > > > From: Carlos Santos > > > > uClibc defines _SC_LEVEL1_ICACHE_LINESIZE and _SC_LEVEL1_DCACHE_LINESIZE > > but the corresponding sysconf calls returns -1, which is a valid result, > > meaning that

Re: [PATCH v2 04/10] hw/arm/bcm2835_peripherals: Use the SYS_timer

2019-10-17 Thread Alistair Francis
On Thu, Oct 17, 2019 at 4:05 PM Philippe Mathieu-Daudé wrote: > > Connect the recently added SYS_timer. > Now U-Boot does not hang anymore polling a free running counter > stuck at 0. > This timer is also used by the Linux kernel thermal subsystem. > > Signed-off-by: Philippe Mathieu-Daudé Revie

[PATCH] Fix unsigned integer underflow in fd-trans.c

2019-10-17 Thread Shu-Chun Weng
In any of these `*_for_each_*` functions, the last entry in the buffer (so the "remaining length in the buffer" `len` is equal to the length of the entry `nlmsg_len`/`nla_len`/etc) has size that is not a multiple of the alignment, the aligned lengths `*_ALIGN(*_len)` will be greater than `len`. Sin

[PATCH 4/6] migration/postcopy: set all_zero to true on the first target page

2019-10-17 Thread Wei Yang
For the first target page, all_zero is set to true for this round check. After target_pages introduced, we could leverage this variable instead of checking the address offset. Signed-off-by: Wei Yang --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migrati

[PATCH 1/6] migration/postcopy: reduce memset when it is zero page and matches_target_page_size

2019-10-17 Thread Wei Yang
In this case, page_buffer content would not be used. Skip this to save some time. Signed-off-by: Wei Yang --- migration/ram.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index 99a98b2da4..7938a643d9 100644 --- a/migration/ram.c ++

[PATCH 2/6] migration/postcopy: wait for decompress thread in precopy

2019-10-17 Thread Wei Yang
Compress is not supported with postcopy, it is safe to wait for decompress thread just in precopy. This is a preparation for later patch. Signed-off-by: Wei Yang --- migration/ram.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 793

[PATCH 3/6] migration/postcopy: count target page number to decide the place_needed

2019-10-17 Thread Wei Yang
In postcopy, it requires to place whole host page instead of target page. Currently, it relies on the page offset to decide whether this is the last target page. We also can count the target page number during the iteration. When the number of target page equals (host page size / target page size)

[PATCH 0/6] migration/postcopy: enable compress during postcopy

2019-10-17 Thread Wei Yang
This patch set tries enable compress during postcopy. postcopy requires to place a whole host page, while migration thread migrate memory in target page size. This makes postcopy need to collect all target pages in one host page before placing via userfaultfd. To enable compress during postcopy,

[PATCH 6/6] migration/postcopy: enable compress during postcopy

2019-10-17 Thread Wei Yang
postcopy requires to place a whole host page, while migration thread migrate memory in target page size. This makes postcopy need to collect all target pages in one host page before placing via userfaultfd. To enable compress during postcopy, there are two problems to solve: 1. Random order f

[PATCH 5/6] migration/postcopy: enable random order target page arrival

2019-10-17 Thread Wei Yang
After using number of target page received to track one host page, we could have the capability to handle random order target page arrival in one host page. This is a preparation for enabling compress during postcopy. Signed-off-by: Wei Yang --- migration/ram.c | 16 +++- 1 file cha

Re: [PATCH RESEND v6 1/2] x86/cpu: Add support for UMONITOR/UMWAIT/TPAUSE

2019-10-17 Thread Tao Xu
Hi Paolo, Ping :) On 10/11/2019 3:49 PM, Tao Xu wrote: On 10/11/2019 3:41 PM, Xu, Tao3 wrote: [...] diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 11b9c854b5..a465c893b5 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -401,6 +401,12 @@ uint32_t kvm_arch_get_supported_cpuid(KV

Re: [PATCH] vhost-user-scsi: implement handle_output

2019-10-17 Thread Yongji Xie
On Fri, 18 Oct 2019 at 01:17, Felipe Franciosi wrote: > > Originally, vhost-user-scsi did not implement a handle_output callback > as that didn't seem necessary. Turns out it is. > > Depending on which other devices are presented to a VM, SeaBIOS may > decide to map vhost-user-scsi devices on the

Re: [PATCH v9 04/15] hw/i386/pc: replace use of strtol with qemu_strtol in x86_load_linux()

2019-10-17 Thread Markus Armbruster
Sergio Lopez writes: > Markus Armbruster writes: > >> Philippe Mathieu-Daudé writes: >> >>> Hi Sergio, >>> >>> On 10/15/19 1:23 PM, Sergio Lopez wrote: Follow checkpatch.pl recommendation and replace the use of strtol with qemu_strtol in x86_load_linux(). >>> >>> "with qemu_strtoui" >

Re: [PATCH] hw/s390x: Emit a warning if user tried to enable USB

2019-10-17 Thread Markus Armbruster
Cornelia Huck writes: > On Thu, 17 Oct 2019 16:40:56 +0200 > Thomas Huth wrote: > >> On 17/10/2019 16.34, Cornelia Huck wrote: >> > On Thu, 17 Oct 2019 16:21:23 +0200 >> > Thomas Huth wrote: >> > >> >> There is no USB on s390x, so running qemu-system-s390x with >> >> "-machine ...,usb=on" is

Re: [PATCH 1/2] spapr: Introduce a interrupt presenter reset handler

2019-10-17 Thread Cédric Le Goater
On 17/10/2019 16:42, Cédric Le Goater wrote: > The interrupt presenters are not reseted today. I should have added : ... when CPU are hot-plugged. > Extend the sPAPR IRQ > backend with a new cpu_intc_reset() handler which will be called by > the CPU reset handler. > > spapr_realize_vcpu() is m

Re: iotest failure -- test possibly not using sufficiently unique temp filename?

2019-10-17 Thread Thomas Huth
On 17/10/2019 18.41, Peter Maydell wrote: > On Fri, 27 Sep 2019 at 17:44, Max Reitz wrote: >> >> On 27.09.19 18:39, Peter Maydell wrote: >>> Hi; I just saw this iotest failure (on an s390x box, as it happens): >>> >>> TESTiotest-qcow2: 130 [fail] >>> QEMU -- >>> "/home/linux1/qemu/b

Re: [PATCH] hw/s390x: Emit a warning if user tried to enable USB

2019-10-17 Thread Thomas Huth
On 17/10/2019 20.18, Philippe Mathieu-Daudé wrote: > On 10/17/19 4:40 PM, Thomas Huth wrote: >> On 17/10/2019 16.34, Cornelia Huck wrote: >>> On Thu, 17 Oct 2019 16:21:23 +0200 >>> Thomas Huth wrote: >>> There is no USB on s390x, so running qemu-system-s390x with "-machine ...,usb=on" is

Re: [PATCH v2 7/7] libqos: add VIRTIO PCI 1.0 support

2019-10-17 Thread Thomas Huth
On 17/10/2019 18.18, Thomas Huth wrote: > On 17/10/2019 18.07, Stefan Hajnoczi wrote: >> On Thu, Oct 17, 2019 at 04:52:54PM +0200, Thomas Huth wrote: >>> On 11/10/2019 10.56, Stefan Hajnoczi wrote: Implement the VIRTIO 1.0 virtio-pci interface. The main change here is that the register l

Re: [PATCH v2 7/7] libqos: add VIRTIO PCI 1.0 support

2019-10-17 Thread Thomas Huth
On 18/10/2019 08.48, Thomas Huth wrote: > On 17/10/2019 18.18, Thomas Huth wrote: >> On 17/10/2019 18.07, Stefan Hajnoczi wrote: >>> On Thu, Oct 17, 2019 at 04:52:54PM +0200, Thomas Huth wrote: On 11/10/2019 10.56, Stefan Hajnoczi wrote: > Implement the VIRTIO 1.0 virtio-pci interface. Th

<    1   2   3   4   5