Re: [Qemu-devel] [PATCH 15/18] Boot Linux Console Test: add a test for aarch64 + virt

[Daniel P . Berrangé]

On Fri, Jun 07, 2019 at 03:58:57PM -0300, Eduardo Habkost wrote:
> CCing Daniel, who wrote commit 6ab3fc32ea64.
> 
> On Fri, Jun 07, 2019 at 11:44:32AM -0400, Cleber Rosa wrote:
> > On Fri, Jun 07, 2019 at 12:42:14AM -0300, Eduardo Habkost wrote:
> > > On Fri, Jun 07, 2019 at 12:26:48AM -0300, Eduardo Habkost wrote:
> > > > On Fri, Feb 01, 2019 at 11:10:31AM -0500, Cleber Rosa wrote:
> > > > > 
> > > > > 
> > > > > On 1/31/19 4:26 PM, Cleber Rosa wrote:
> > > > > > 
> > > > > > 
> > > > > > On 1/31/19 3:21 PM, Cleber Rosa wrote:
> > > > > >>
> > > > > >>
> > > > > >> On 1/31/19 3:02 PM, Wainer dos Santos Moschetta wrote:
> > > > > >>>
> > > > > >>> On 01/17/2019 04:56 PM, Cleber Rosa wrote:
> > > > >  Just like the previous tests, boots a Linux kernel on a aarch64 
> > > > >  target
> > > > >  using the virt machine.
> > > > > 
> > > > >  One special option added is the CPU type, given that the kernel
> > > > >  selected fails to boot on the virt machine's default CPU 
> > > > >  (cortex-a15).
> > > > > 
> > > > >  Signed-off-by: Cleber Rosa 
> > > > >  ---
> > > > >    .travis.yml    |  2 +-
> > > > >    tests/acceptance/boot_linux_console.py | 20 
> > > > >  
> > > > >    2 files changed, 21 insertions(+), 1 deletion(-)
> > > > > 
> > > > >  diff --git a/.travis.yml b/.travis.yml
> > > > >  index 54100eea5a..595e8c0b6c 100644
> > > > >  --- a/.travis.yml
> > > > >  +++ b/.travis.yml
> > > > >  @@ -187,7 +187,7 @@ matrix:
> > > > >      # Acceptance (Functional) tests
> > > > >    - env:
> > > > >  -    - CONFIG="--python=/usr/bin/python3
> > > > >  --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu"
> > > > >  +    - CONFIG="--python=/usr/bin/python3
> > > > >  --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu"
> > > > > 
> > > > >    - TEST_CMD="make check-acceptance"
> > > > >      addons:
> > > > >    apt:
> > > > >  diff --git a/tests/acceptance/boot_linux_console.py
> > > > >  b/tests/acceptance/boot_linux_console.py
> > > > >  index f3ccd23a7a..107700b517 100644
> > > > >  --- a/tests/acceptance/boot_linux_console.py
> > > > >  +++ b/tests/acceptance/boot_linux_console.py
> > > > >  @@ -138,3 +138,23 @@ class BootLinuxConsole(Test):
> > > > >    self.vm.launch()
> > > > >    console_pattern = 'Kernel command line: %s' %
> > > > >  kernel_command_line
> > > > >    self.wait_for_console_pattern(console_pattern)
> > > > >  +
> > > > >  +    def test_aarch64_virt(self):
> > > > > >>>
> > > > > >>> That test case fails on my system (Fedora 29 x86_64). Avocado 
> > > > > >>> seems
> > > > > >>> unable to kill the VM so it  reaches the timeout.
> > > > > >>>
> > > > > >>> I compiled QEMU with default configuration:
> > > > > >>>
> > > > > >>> $ configure --python=/usr/bin/python3 --target-list=x86_64-softmmu
> > > > > >>> --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,ppc64-softmmu,aarch64-softmmu)
> > > > > >>>
> > > > > >>>
> > > > > >>> Follows a snippet of the Avocado's job.log file:
> > > > > >>> 
> > > > > >>> 2019-01-31 14:41:34,912 test L0602 INFO | START
> > > > > >>> 07-/root/src/qemu/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt
> > > > > >>>
> > > > > >>> 2019-01-31 14:41:34,912 test L0298 DEBUG| DATA
> > > > > >>> (filename=output.expected) => NOT FOUND (data sources: variant, 
> > > > > >>> test, file)
> > > > > >>> 2019-01-31 14:41:34,913 parameters   L0146 DEBUG| PARAMS 
> > > > > >>> (key=arch,
> > > > > >>> path=*, default=aarch64) => 'aarch64'
> > > > > >>> 2019-01-31 14:41:34,913 parameters   L0146 DEBUG| PARAMS
> > > > > >>> (key=qemu_bin, path=*, 
> > > > > >>> default=aarch64-softmmu/qemu-system-aarch64) =>
> > > > > >>> 'aarch64-softmmu/qemu-system-aarch64'
> > > > > >>> 2019-01-31 14:41:34,915 download L0070 INFO | Fetching
> > > > > >>> https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz
> > > > > >>> -> /var/lib/avocado/data/cache/by_name/vmlinuz.3upct2pr
> > > > > >>> 2019-01-31 14:41:35,490 download L0054 DEBUG| Retrieved 
> > > > > >>> URL
> > > > > >>> "https://sjc.edge.kernel.org/fedora-buffet/fedora/linux/releases/29/Server/aarch64/os/images/pxeboot/vmlinuz":
> > > > > >>> content-length 8623423, date: "Thu, 31 Jan 2019 19:41:35 GMT",
> > > > > >>> last-modified: "Sun, 21 Oct 2018 00:43:09 GMT"
> > > > > >>> 2019-01-31 14:41:41,765 qemu L0317 DEBUG| VM launch 
> > > > > >>> command:
> > > > > >>> 'aarch64-softmmu/qemu-system-aarch64 -chardev
> > > > > >>> socket,id=mon,path=/var/tmp/tmpizirkcud/qemu-32609-monitor.sock 
> > > > > >>> -mon
> > > > > >>> chardev=mon,mode=control -display none

Re: [Qemu-devel] [PATCH 7/8] VNC Acceptance test: check protocol version

[Daniel P . Berrangé]

On Fri, Jun 07, 2019 at 02:12:07PM -0400, Cleber Rosa wrote:
> On Fri, Jun 07, 2019 at 06:29:15PM +0100, Daniel P. Berrangé wrote:
> > On Fri, Jun 07, 2019 at 11:22:22AM -0400, Cleber Rosa wrote:
> > > This goes a bit further than the other tests, and does a basic (read
> > > only) interaction with the VNC protocol.
> > > 
> > > This is not a enough to perform a handshake, but enough to make sure
> > > that the socket is somewhat operational and that the expected initial
> > > step of the handshake is performed by the server and that the version
> > > matches.
> > 
> > The GTK-VNC project provides a low level library libgvnc that can
> > be used to talk the RFB protocol in a fairly fine grained manner.
> > This is built using GObject, so is accessible from Python thanks
> > to GObject Introspection.
> >
> 
> Interesting.
> 
> > We could use libgvnc for exercising the VNC server instead of writing
> > custom RFB code. For your simple test just sending/receiving the
> > version it won't save much, but if we ever want to test TLS or
> > SASL integration, it would save alot of work dealing wth the auth
> > handshake and subsequent encryption needs.
> >
> 
> Absolutely.
> 
> > The main limitation it would have is that it would only work well
> > for sending "well formed" RFB protocol messages. If we ever want to
> > send intentionally evil/bad RFB data to check QEMU's VNC server
> > security hardening it would be harder.
> >
> 
> Right.  Still, there's a lot that can be done until we eventually
> exaust all possibilities and look into sending bad messages.
> 
> > As the maintainer of GTK-VNC though, I would be open to adding more
> > APIs to the low level gvnc library to facilitate QEMU's testing
> > needs if we want.
> >
> 
> I personally need to get acquainted with the currently available APIs
> first, but it looks like you alread have ideas for extensions that
> would come in handy.
> 
> Also, the one concern I have is how to deploy the library and Python
> bindings so that we can host those more advanced tests and still allow
> for a "make check-acceptance"-like experience.  What I mean is, I
> expect the Python bindings to be easily installed by pip, by I'd be
> (positively) surprised if the libgvnc would also have such an easy
> bootstrap.
> 
> Any ideas on this?

IMHO we shouldn't try to install anything from pip. It should all be
available from distro repos already. In fact if you have "virt-install"
or "virt-manager" installed, you should already have the required
python bits.

On RHEL/Fedora you need python3-gobject to get the GObject introspection
support and then all you need is the native gvnc library. There is no
python library for gvnc because GObject introspection makes this
redundant - the python API is dynamically created at runtime using
FFI to access the native library APIs.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH] RFC: qio: Improve corking of TLS sessions

[Daniel P . Berrangé]

On Fri, Jun 07, 2019 at 05:14:14PM -0500, Eric Blake wrote:
> Our current implementation of qio_channel_set_cork() is pointless for
> TLS sessions: we block the underlying channel, but still hand things
> piecemeal to gnutls which then produces multiple encryption packets.
> Better is to directly use gnutls corking, which collects multiple
> inputs into a single encryption packet.
> 
> Signed-off-by: Eric Blake 
> 
> ---
> 
> RFC because unfortunately, I'm not sure I like the results.  My test
> case (using latest nbdkit.git) involves sending 10G of random data
> over NBD using parallel writes (and doing nothing on the server end;
> this is all about timing the data transfer):
> 
> $ dd if=/dev/urandom of=rand bs=1M count=10k
> $ time nbdkit -p 10810 --tls=require --tls-verify-peer \
>--tls-psk=/tmp/keys.psk --filter=stats null 10g statsfile=/dev/stderr \
>--run '~/qemu/qemu-img convert -f raw -W -n --target-image-opts \
>  --object tls-creds-psk,id=tls0,endpoint=client,dir=/tmp,username=eblake \
>  rand 
> driver=nbd,server.type=inet,server.host=localhost,server.port=10810,tls-creds=tls0'
> 
> Pre-patch, I measured:
> real  0m34.536s
> user  0m29.264s
> sys   0m4.014s
> 
> while post-patch, it changed to:
> real  0m36.055s
> user  0m27.685s
> sys   0m10.138s
> 
> Less time spent in user space, but for this particular qemu-img
> behavior (writing 2M chunks at a time), gnutls is now uncorking huge
> packets and the kernel is doing enough extra work that the overall
> program actually takes longer. :(

I wonder if the problem is that we're hitting a tradeoff between
time spent in encryption vs time spent in network I/O.

When we don't cork, the kernel has already sent the first packet
during the time gnutls is burning CPU encrypting the second packet.

When we do cork gnutls has to encrypt both packets before the kernel
can send anything.

> For smaller I/O patterns, the effects of corking are more likely to be
> beneficial, but I don't have a ready test case to produce that pattern
> (short of creating a guest running fio on a device backed by nbd).

I think it would probably be useful to see guest kernels with fio
and definitely see results for something closer to sector sized
I/O.

2 MB chunks is pretty unrealistic for a guest workload where there
will be lots of sector sized I/O. Sure QEMU / guest OS can merge
sectors, but it still feels like most I/O is going to be much smaller
than 2 MB.


> diff --git a/io/channel.c b/io/channel.c
> index 2a26c2a2c0b9..3510912465ac 100644
> --- a/io/channel.c
> +++ b/io/channel.c
> @@ -379,7 +379,16 @@ void qio_channel_set_cork(QIOChannel *ioc,
>  QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
> 
>  if (klass->io_set_cork) {
> -klass->io_set_cork(ioc, enabled);
> +int r = klass->io_set_cork(ioc, enabled);
> +
> +while (r == QIO_CHANNEL_ERR_BLOCK) {
> +if (qemu_in_coroutine()) {
> +qio_channel_yield(ioc, G_IO_OUT);
> +} else {
> +qio_channel_wait(ioc, G_IO_OUT);
> +}
> +r = klass->io_set_cork(ioc, enabled);
> +}
>  }

Interesting - did you actually hit this EAGAIN behaviour ? I wouldn't
have expected anything to be pending in gnutls that needs retry. 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[Qemu-devel] [PATCH v11 0/7] virtio pmem driver

[Pankaj Gupta]

 This patch series is ready to be merged via nvdimm tree
 as discussed with Dan. We have ack/review on XFS, EXT4 &
 VIRTIO patches. Need an ack on device mapper change in
 patch 4.

 Mike, Can you please review and ack patch4.

 This version does not has any additonal code change from v10 
 and is only rebase of v10 on Linux 5.2-rc4 which is required 
 for patch4. Keeping all the existing r-o-bs. Jakob CCed also
 tested the patch series and confirmed the working of v9.
 ---

 This patch series has implementation for "virtio pmem". 
 "virtio pmem" is fake persistent memory(nvdimm) in guest 
 which allows to bypass the guest page cache. This also
 implements a VIRTIO based asynchronous flush mechanism.  
 
 Sharing guest kernel driver in this patchset with the 
 changes suggested in v4. Tested with Qemu side device 
 emulation [5] for virtio-pmem. Documented the impact of
 possible page cache side channel attacks with suggested
 countermeasures.

 Details of project idea for 'virtio pmem' flushing interface 
 is shared [3] & [4].

 Implementation is divided into two parts:
 New virtio pmem guest driver and qemu code changes for new 
 virtio pmem paravirtualized device.

1. Guest virtio-pmem kernel driver
-
   - Reads persistent memory range from paravirt device and 
 registers with 'nvdimm_bus'.  
   - 'nvdimm/pmem' driver uses this information to allocate 
 persistent memory region and setup filesystem operations 
 to the allocated memory. 
   - virtio pmem driver implements asynchronous flushing 
 interface to flush from guest to host.

2. Qemu virtio-pmem device
-
   - Creates virtio pmem device and exposes a memory range to 
 KVM guest. 
   - At host side this is file backed memory which acts as 
 persistent memory. 
   - Qemu side flush uses aio thread pool API's and virtio 
 for asynchronous guest multi request handling. 

 Virtio-pmem security implications and countermeasures:
 -

 In previous posting of kernel driver, there was discussion [7]
 on possible implications of page cache side channel attacks with 
 virtio pmem. After thorough analysis of details of known side 
 channel attacks, below are the suggestions:

 - Depends entirely on how host backing image file is mapped 
   into guest address space. 

 - virtio-pmem device emulation, by default shared mapping is used
   to map host backing file. It is recommended to use separate
   backing file at host side for every guest. This will prevent
   any possibility of executing common code from multiple guests
   and any chance of inferring guest local data based based on 
   execution time.

 - If backing file is required to be shared among multiple guests 
   it is recommended to don't support host page cache eviction 
   commands from the guest driver. This will avoid any possibility
   of inferring guest local data or host data from another guest. 

 - Proposed device specification [6] for virtio-pmem device with 
   details of possible security implications and suggested 
   countermeasures for device emulation.

 Virtio-pmem errors handling:
 
  Checked behaviour of virtio-pmem for below types of errors
  Need suggestions on expected behaviour for handling these errors?

  - Hardware Errors: Uncorrectable recoverable Errors: 
  a] virtio-pmem: 
- As per current logic if error page belongs to Qemu process, 
  host MCE handler isolates(hwpoison) that page and send SIGBUS. 
  Qemu SIGBUS handler injects exception to KVM guest. 
- KVM guest then isolates the page and send SIGBUS to guest 
  userspace process which has mapped the page. 
  
  b] Existing implementation for ACPI pmem driver: 
- Handles such errors with MCE notifier and creates a list 
  of bad blocks. Read/direct access DAX operation return EIO 
  if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.  
- Similar functionality can be reused in virtio-pmem with MCE 
  notifier but without scrubbing(no ACPI/ARS)? Need inputs to 
  confirm if this behaviour is ok or needs any change?

Changes from PATCH v10: [1] 
 - Rebased on Linux-5.2-rc4

Changes from PATCH v9: [2] 
 - Kconfig help text add two spaces - Randy
 - Fixed libnvdimm 'bio' include warning - Dan
 - virtio-pmem, separate request/resp struct and 
   move to uapi file with updated license - DavidH
 - Use virtio32* type for req/resp endianess - DavidH
 - Added tested-by & ack-by of Jakob
 - Rebased to 5.2-rc1

Changes from PATCH v8:
 - Set device mapper synchronous if all target devices support - Dan
 - Move virtio_pmem.h to nvdimm directory  - Dan
 - Style, indentation & better error messages in patch 2 - DavidH
 - Added MST's ack in patch 2.

Changes from PATCH v7:
 - Corrected pending request queue logic (patch 2) - Jakub Staroń
 - Used unsigned long flags for passin

[Qemu-devel] [PATCH v11 1/7] libnvdimm: nd_region flush callback support

[Pankaj Gupta]

This patch adds functionality to perform flush from guest
to host over VIRTIO. We are registering a callback based
on 'nd_region' type. virtio_pmem driver requires this special
flush function. For rest of the region types we are registering
existing flush function. Report error returned by host fsync
failure to userspace.

Signed-off-by: Pankaj Gupta 
---
 drivers/acpi/nfit/core.c |  4 ++--
 drivers/nvdimm/claim.c   |  6 --
 drivers/nvdimm/nd.h  |  1 +
 drivers/nvdimm/pmem.c| 13 -
 drivers/nvdimm/region_devs.c | 26 --
 include/linux/libnvdimm.h|  9 -
 6 files changed, 47 insertions(+), 12 deletions(-)

diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c
index f1ed0befe303..9ddd8667153e 100644
--- a/drivers/acpi/nfit/core.c
+++ b/drivers/acpi/nfit/core.c
@@ -2434,7 +2434,7 @@ static void write_blk_ctl(struct nfit_blk *nfit_blk, 
unsigned int bw,
offset = to_interleave_offset(offset, mmio);
 
writeq(cmd, mmio->addr.base + offset);
-   nvdimm_flush(nfit_blk->nd_region);
+   nvdimm_flush(nfit_blk->nd_region, NULL);
 
if (nfit_blk->dimm_flags & NFIT_BLK_DCR_LATCH)
readq(mmio->addr.base + offset);
@@ -2483,7 +2483,7 @@ static int acpi_nfit_blk_single_io(struct nfit_blk 
*nfit_blk,
}
 
if (rw)
-   nvdimm_flush(nfit_blk->nd_region);
+   nvdimm_flush(nfit_blk->nd_region, NULL);
 
rc = read_blk_stat(nfit_blk, lane) ? -EIO : 0;
return rc;
diff --git a/drivers/nvdimm/claim.c b/drivers/nvdimm/claim.c
index fb667bf469c7..13510bae1e6f 100644
--- a/drivers/nvdimm/claim.c
+++ b/drivers/nvdimm/claim.c
@@ -263,7 +263,7 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns,
struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
unsigned int sz_align = ALIGN(size + (offset & (512 - 1)), 512);
sector_t sector = offset >> 9;
-   int rc = 0;
+   int rc = 0, ret = 0;
 
if (unlikely(!size))
return 0;
@@ -301,7 +301,9 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns,
}
 
memcpy_flushcache(nsio->addr + offset, buf, size);
-   nvdimm_flush(to_nd_region(ndns->dev.parent));
+   ret = nvdimm_flush(to_nd_region(ndns->dev.parent), NULL);
+   if (ret)
+   rc = ret;
 
return rc;
 }
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index a5ac3b240293..0c74d2428bd7 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -159,6 +159,7 @@ struct nd_region {
struct badblocks bb;
struct nd_interleave_set *nd_set;
struct nd_percpu_lane __percpu *lane;
+   int (*flush)(struct nd_region *nd_region, struct bio *bio);
struct nd_mapping mapping[0];
 };
 
diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 0279eb1da3ef..c757a47183b8 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -192,6 +192,7 @@ static blk_status_t pmem_do_bvec(struct pmem_device *pmem, 
struct page *page,
 
 static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)
 {
+   int ret = 0;
blk_status_t rc = 0;
bool do_acct;
unsigned long start;
@@ -201,7 +202,7 @@ static blk_qc_t pmem_make_request(struct request_queue *q, 
struct bio *bio)
struct nd_region *nd_region = to_region(pmem);
 
if (bio->bi_opf & REQ_PREFLUSH)
-   nvdimm_flush(nd_region);
+   ret = nvdimm_flush(nd_region, bio);
 
do_acct = nd_iostat_start(bio, &start);
bio_for_each_segment(bvec, bio, iter) {
@@ -216,7 +217,10 @@ static blk_qc_t pmem_make_request(struct request_queue *q, 
struct bio *bio)
nd_iostat_end(bio, start);
 
if (bio->bi_opf & REQ_FUA)
-   nvdimm_flush(nd_region);
+   ret = nvdimm_flush(nd_region, bio);
+
+   if (ret)
+   bio->bi_status = errno_to_blk_status(ret);
 
bio_endio(bio);
return BLK_QC_T_NONE;
@@ -469,7 +473,6 @@ static int pmem_attach_disk(struct device *dev,
}
dax_write_cache(dax_dev, nvdimm_has_cache(nd_region));
pmem->dax_dev = dax_dev;
-
gendev = disk_to_dev(disk);
gendev->groups = pmem_attribute_groups;
 
@@ -527,14 +530,14 @@ static int nd_pmem_remove(struct device *dev)
sysfs_put(pmem->bb_state);
pmem->bb_state = NULL;
}
-   nvdimm_flush(to_nd_region(dev->parent));
+   nvdimm_flush(to_nd_region(dev->parent), NULL);
 
return 0;
 }
 
 static void nd_pmem_shutdown(struct device *dev)
 {
-   nvdimm_flush(to_nd_region(dev->parent));
+   nvdimm_flush(to_nd_region(dev->parent), NULL);
 }
 
 static void nd_pmem_notify(struct device *dev, enum nvdimm_event event)
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index b4ef7d9ff22e..e5b59708865e 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/dr

[Qemu-devel] [PATCH v11 2/7] virtio-pmem: Add virtio pmem driver

[Pankaj Gupta]

This patch adds virtio-pmem driver for KVM guest.

Guest reads the persistent memory range information from
Qemu over VIRTIO and registers it on nvdimm_bus. It also
creates a nd_region object with the persistent memory
range information so that existing 'nvdimm/pmem' driver
can reserve this into system memory map. This way
'virtio-pmem' driver uses existing functionality of pmem
driver to register persistent memory compatible for DAX
capable filesystems.

This also provides function to perform guest flush over
VIRTIO from 'pmem' driver when userspace performs flush
on DAX memory range.

Signed-off-by: Pankaj Gupta 
Reviewed-by: Yuval Shaia 
Acked-by: Michael S. Tsirkin 
Acked-by: Jakub Staron 
Tested-by: Jakub Staron 
---
 drivers/nvdimm/Makefile  |   1 +
 drivers/nvdimm/nd_virtio.c   | 124 +++
 drivers/nvdimm/virtio_pmem.c | 122 ++
 drivers/nvdimm/virtio_pmem.h |  55 ++
 drivers/virtio/Kconfig   |  11 +++
 include/uapi/linux/virtio_ids.h  |   1 +
 include/uapi/linux/virtio_pmem.h |  35 +
 7 files changed, 349 insertions(+)
 create mode 100644 drivers/nvdimm/nd_virtio.c
 create mode 100644 drivers/nvdimm/virtio_pmem.c
 create mode 100644 drivers/nvdimm/virtio_pmem.h
 create mode 100644 include/uapi/linux/virtio_pmem.h

diff --git a/drivers/nvdimm/Makefile b/drivers/nvdimm/Makefile
index 6f2a088afad6..cefe233e0b52 100644
--- a/drivers/nvdimm/Makefile
+++ b/drivers/nvdimm/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_ND_BTT) += nd_btt.o
 obj-$(CONFIG_ND_BLK) += nd_blk.o
 obj-$(CONFIG_X86_PMEM_LEGACY) += nd_e820.o
 obj-$(CONFIG_OF_PMEM) += of_pmem.o
+obj-$(CONFIG_VIRTIO_PMEM) += virtio_pmem.o nd_virtio.o
 
 nd_pmem-y := pmem.o
 
diff --git a/drivers/nvdimm/nd_virtio.c b/drivers/nvdimm/nd_virtio.c
new file mode 100644
index ..efc535723517
--- /dev/null
+++ b/drivers/nvdimm/nd_virtio.c
@@ -0,0 +1,124 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * virtio_pmem.c: Virtio pmem Driver
+ *
+ * Discovers persistent memory range information
+ * from host and provides a virtio based flushing
+ * interface.
+ */
+#include "virtio_pmem.h"
+#include "nd.h"
+
+ /* The interrupt handler */
+void host_ack(struct virtqueue *vq)
+{
+   struct virtio_pmem *vpmem = vq->vdev->priv;
+   struct virtio_pmem_request *req_data, *req_buf;
+   unsigned long flags;
+   unsigned int len;
+
+   spin_lock_irqsave(&vpmem->pmem_lock, flags);
+   while ((req_data = virtqueue_get_buf(vq, &len)) != NULL) {
+   req_data->done = true;
+   wake_up(&req_data->host_acked);
+
+   if (!list_empty(&vpmem->req_list)) {
+   req_buf = list_first_entry(&vpmem->req_list,
+   struct virtio_pmem_request, list);
+   req_buf->wq_buf_avail = true;
+   wake_up(&req_buf->wq_buf);
+   list_del(&req_buf->list);
+   }
+   }
+   spin_unlock_irqrestore(&vpmem->pmem_lock, flags);
+}
+EXPORT_SYMBOL_GPL(host_ack);
+
+ /* The request submission function */
+int virtio_pmem_flush(struct nd_region *nd_region)
+{
+   struct virtio_device *vdev = nd_region->provider_data;
+   struct virtio_pmem *vpmem  = vdev->priv;
+   struct virtio_pmem_request *req_data;
+   struct scatterlist *sgs[2], sg, ret;
+   unsigned long flags;
+   int err, err1;
+
+   might_sleep();
+   req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
+   if (!req_data)
+   return -ENOMEM;
+
+   req_data->done = false;
+   init_waitqueue_head(&req_data->host_acked);
+   init_waitqueue_head(&req_data->wq_buf);
+   INIT_LIST_HEAD(&req_data->list);
+   req_data->req.type = cpu_to_virtio32(vdev, VIRTIO_PMEM_REQ_TYPE_FLUSH);
+   sg_init_one(&sg, &req_data->req, sizeof(req_data->req));
+   sgs[0] = &sg;
+   sg_init_one(&ret, &req_data->resp.ret, sizeof(req_data->resp));
+   sgs[1] = &ret;
+
+   spin_lock_irqsave(&vpmem->pmem_lock, flags);
+/*
+ * If virtqueue_add_sgs returns -ENOSPC then req_vq virtual
+ * queue does not have free descriptor. We add the request
+ * to req_list and wait for host_ack to wake us up when free
+ * slots are available.
+ */
+   while ((err = virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data,
+   GFP_ATOMIC)) == -ENOSPC) {
+
+   dev_err(&vdev->dev, "failed to send command to virtio pmem 
device, no free slots in the virtqueue\n");
+   req_data->wq_buf_avail = false;
+   list_add_tail(&req_data->list, &vpmem->req_list);
+   spin_unlock_irqrestore(&vpmem->pmem_lock, flags);
+
+   /* A host response results in "host_ack" getting called */
+   wait_event(req_data->wq_buf, req_data->wq_buf_avail);
+   spin_lock_irqsave(&vpmem->pmem_lock, flag

Re: [Qemu-devel] [PATCH 2/7] iotests: amendment for benchmark output of 039 061 137

[Daniel P . Berrangé]

On Sun, Jun 09, 2019 at 09:35:43PM +0300, Andrey Shinkevich wrote:
> After including the Valgrind into the QEMU processes wrappers in the
> common.rc script, the output for the tests 039 061 137 has been
> changed and is to be amended.

The change to common.rc was done in the previous patch, so this
fixup should be there too. Separating the fixup will impact people
doing "git bisect" to track down bugs.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[Qemu-devel] [PATCH v11 3/7] libnvdimm: add dax_dev sync flag

[Pankaj Gupta]

This patch adds 'DAXDEV_SYNC' flag which is set
for nd_region doing synchronous flush. This later
is used to disable MAP_SYNC functionality for
ext4 & xfs filesystem for devices don't support
synchronous flush.

Signed-off-by: Pankaj Gupta 
---
 drivers/dax/bus.c|  2 +-
 drivers/dax/super.c  | 19 ++-
 drivers/md/dm.c  |  3 ++-
 drivers/nvdimm/pmem.c|  5 -
 drivers/nvdimm/region_devs.c |  7 +++
 include/linux/dax.h  |  9 +++--
 include/linux/libnvdimm.h|  1 +
 7 files changed, 40 insertions(+), 6 deletions(-)

diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index 2109cfe80219..5f184e751c82 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -388,7 +388,7 @@ struct dev_dax *__devm_create_dev_dax(struct dax_region 
*dax_region, int id,
 * No 'host' or dax_operations since there is no access to this
 * device outside of mmap of the resulting character device.
 */
-   dax_dev = alloc_dax(dev_dax, NULL, NULL);
+   dax_dev = alloc_dax(dev_dax, NULL, NULL, DAXDEV_F_SYNC);
if (!dax_dev)
goto err;
 
diff --git a/drivers/dax/super.c b/drivers/dax/super.c
index bbd57ca0634a..93b3718b5cfa 100644
--- a/drivers/dax/super.c
+++ b/drivers/dax/super.c
@@ -186,6 +186,8 @@ enum dax_device_flags {
DAXDEV_ALIVE,
/* gate whether dax_flush() calls the low level flush routine */
DAXDEV_WRITE_CACHE,
+   /* flag to check if device supports synchronous flush */
+   DAXDEV_SYNC,
 };
 
 /**
@@ -354,6 +356,18 @@ bool dax_write_cache_enabled(struct dax_device *dax_dev)
 }
 EXPORT_SYMBOL_GPL(dax_write_cache_enabled);
 
+bool dax_synchronous(struct dax_device *dax_dev)
+{
+   return test_bit(DAXDEV_SYNC, &dax_dev->flags);
+}
+EXPORT_SYMBOL_GPL(dax_synchronous);
+
+void set_dax_synchronous(struct dax_device *dax_dev)
+{
+   set_bit(DAXDEV_SYNC, &dax_dev->flags);
+}
+EXPORT_SYMBOL_GPL(set_dax_synchronous);
+
 bool dax_alive(struct dax_device *dax_dev)
 {
lockdep_assert_held(&dax_srcu);
@@ -508,7 +522,7 @@ static void dax_add_host(struct dax_device *dax_dev, const 
char *host)
 }
 
 struct dax_device *alloc_dax(void *private, const char *__host,
-   const struct dax_operations *ops)
+   const struct dax_operations *ops, unsigned long flags)
 {
struct dax_device *dax_dev;
const char *host;
@@ -531,6 +545,9 @@ struct dax_device *alloc_dax(void *private, const char 
*__host,
dax_add_host(dax_dev, host);
dax_dev->ops = ops;
dax_dev->private = private;
+   if (flags & DAXDEV_F_SYNC)
+   set_dax_synchronous(dax_dev);
+
return dax_dev;
 
  err_dev:
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 1fb1333fefec..7eee7ddc73a8 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1970,7 +1970,8 @@ static struct mapped_device *alloc_dev(int minor)
sprintf(md->disk->disk_name, "dm-%d", minor);
 
if (IS_ENABLED(CONFIG_DAX_DRIVER)) {
-   md->dax_dev = alloc_dax(md, md->disk->disk_name, &dm_dax_ops);
+   md->dax_dev = alloc_dax(md, md->disk->disk_name,
+   &dm_dax_ops, 0);
if (!md->dax_dev)
goto bad;
}
diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 0279eb1da3ef..bdbd2b414d3d 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -365,6 +365,7 @@ static int pmem_attach_disk(struct device *dev,
struct gendisk *disk;
void *addr;
int rc;
+   unsigned long flags = 0UL;
 
pmem = devm_kzalloc(dev, sizeof(*pmem), GFP_KERNEL);
if (!pmem)
@@ -462,7 +463,9 @@ static int pmem_attach_disk(struct device *dev,
nvdimm_badblocks_populate(nd_region, &pmem->bb, &bb_res);
disk->bb = &pmem->bb;
 
-   dax_dev = alloc_dax(pmem, disk->disk_name, &pmem_dax_ops);
+   if (is_nvdimm_sync(nd_region))
+   flags = DAXDEV_F_SYNC;
+   dax_dev = alloc_dax(pmem, disk->disk_name, &pmem_dax_ops, flags);
if (!dax_dev) {
put_disk(disk);
return -ENOMEM;
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index b4ef7d9ff22e..f3ea5369d20a 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -1197,6 +1197,13 @@ int nvdimm_has_cache(struct nd_region *nd_region)
 }
 EXPORT_SYMBOL_GPL(nvdimm_has_cache);
 
+bool is_nvdimm_sync(struct nd_region *nd_region)
+{
+   return is_nd_pmem(&nd_region->dev) &&
+   !test_bit(ND_REGION_ASYNC, &nd_region->flags);
+}
+EXPORT_SYMBOL_GPL(is_nvdimm_sync);
+
 struct conflict_context {
struct nd_region *nd_region;
resource_size_t start, size;
diff --git a/include/linux/dax.h b/include/linux/dax.h
index 0dd316a74a29..2b106752b1b8 100644
--- a/include/linux/dax.h
+++ b/include/linux/dax.h
@@ -7,6 +7,9 @@
 #include 
 #include 
 
+/* Flag for sync

[Qemu-devel] [PATCH v11 5/7] dax: check synchronous mapping is supported

[Pankaj Gupta]

This patch introduces 'daxdev_mapping_supported' helper
which checks if 'MAP_SYNC' is supported with filesystem
mapping. It also checks if corresponding dax_device is
synchronous. Virtio pmem device is asynchronous and
does not not support VM_SYNC.

Suggested-by: Jan Kara 
Signed-off-by: Pankaj Gupta 
Reviewed-by: Jan Kara 
---
 include/linux/dax.h | 17 +
 1 file changed, 17 insertions(+)

diff --git a/include/linux/dax.h b/include/linux/dax.h
index 2b106752b1b8..267251a394fa 100644
--- a/include/linux/dax.h
+++ b/include/linux/dax.h
@@ -42,6 +42,18 @@ void dax_write_cache(struct dax_device *dax_dev, bool wc);
 bool dax_write_cache_enabled(struct dax_device *dax_dev);
 bool dax_synchronous(struct dax_device *dax_dev);
 void set_dax_synchronous(struct dax_device *dax_dev);
+/*
+ * Check if given mapping is supported by the file / underlying device.
+ */
+static inline bool daxdev_mapping_supported(struct vm_area_struct *vma,
+   struct dax_device *dax_dev)
+{
+   if (!(vma->vm_flags & VM_SYNC))
+   return true;
+   if (!IS_DAX(file_inode(vma->vm_file)))
+   return false;
+   return dax_synchronous(dax_dev);
+}
 #else
 static inline struct dax_device *dax_get_by_host(const char *host)
 {
@@ -69,6 +81,11 @@ static inline bool dax_write_cache_enabled(struct dax_device 
*dax_dev)
 {
return false;
 }
+static inline bool daxdev_mapping_supported(struct vm_area_struct *vma,
+   struct dax_device *dax_dev)
+{
+   return !(vma->vm_flags & VM_SYNC);
+}
 #endif
 
 struct writeback_control;
-- 
2.20.1

[Qemu-devel] [PATCH v11 4/7] dm: enable synchronous dax

[Pankaj Gupta]

 This patch sets dax device 'DAXDEV_SYNC' flag if all the target
 devices of device mapper support synchrononous DAX. If device
 mapper consists of both synchronous and asynchronous dax devices,
 we don't set 'DAXDEV_SYNC' flag.

Signed-off-by: Pankaj Gupta 
---
 drivers/md/dm-table.c | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
index 350cf0451456..c5160d846fe6 100644
--- a/drivers/md/dm-table.c
+++ b/drivers/md/dm-table.c
@@ -890,10 +890,17 @@ static int device_supports_dax(struct dm_target *ti, 
struct dm_dev *dev,
start, len);
 }
 
+static int device_synchronous(struct dm_target *ti, struct dm_dev *dev,
+  sector_t start, sector_t len, void *data)
+{
+   return dax_synchronous(dev->dax_dev);
+}
+
 bool dm_table_supports_dax(struct dm_table *t, int blocksize)
 {
struct dm_target *ti;
unsigned i;
+   bool dax_sync = true;
 
/* Ensure that all targets support DAX. */
for (i = 0; i < dm_table_get_num_targets(t); i++) {
@@ -906,7 +913,14 @@ bool dm_table_supports_dax(struct dm_table *t, int 
blocksize)
!ti->type->iterate_devices(ti, device_supports_dax,
&blocksize))
return false;
+
+   /* Check devices support synchronous DAX */
+   if (dax_sync &&
+   !ti->type->iterate_devices(ti, device_synchronous, NULL))
+   dax_sync = false;
}
+   if (dax_sync)
+   set_dax_synchronous(t->md->dax_dev);
 
return true;
 }
-- 
2.20.1

[Qemu-devel] [PATCH v11 6/7] ext4: disable map_sync for async flush

[Pankaj Gupta]

Dont support 'MAP_SYNC' with non-DAX files and DAX files
with asynchronous dax_device. Virtio pmem provides
asynchronous host page cache flush mechanism. We don't
support 'MAP_SYNC' with virtio pmem and ext4.

Signed-off-by: Pankaj Gupta 
Reviewed-by: Jan Kara 
---
 fs/ext4/file.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 98ec11f69cd4..dee549339e13 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -360,15 +360,17 @@ static const struct vm_operations_struct ext4_file_vm_ops 
= {
 static int ext4_file_mmap(struct file *file, struct vm_area_struct *vma)
 {
struct inode *inode = file->f_mapping->host;
+   struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
+   struct dax_device *dax_dev = sbi->s_daxdev;
 
-   if (unlikely(ext4_forced_shutdown(EXT4_SB(inode->i_sb
+   if (unlikely(ext4_forced_shutdown(sbi)))
return -EIO;
 
/*
-* We don't support synchronous mappings for non-DAX files. At least
-* until someone comes with a sensible use case.
+* We don't support synchronous mappings for non-DAX files and
+* for DAX files if underneath dax_device is not synchronous.
 */
-   if (!IS_DAX(file_inode(file)) && (vma->vm_flags & VM_SYNC))
+   if (!daxdev_mapping_supported(vma, dax_dev))
return -EOPNOTSUPP;
 
file_accessed(file);
-- 
2.20.1

[Qemu-devel] [PATCH v11 7/7] xfs: disable map_sync for async flush

[Pankaj Gupta]

Dont support 'MAP_SYNC' with non-DAX files and DAX files
with asynchronous dax_device. Virtio pmem provides
asynchronous host page cache flush mechanism. We don't
support 'MAP_SYNC' with virtio pmem and xfs.

Signed-off-by: Pankaj Gupta 
Reviewed-by: Darrick J. Wong 
---
 fs/xfs/xfs_file.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index a7ceae90110e..f17652cca5ff 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1203,11 +1203,14 @@ xfs_file_mmap(
struct file *filp,
struct vm_area_struct *vma)
 {
+   struct dax_device   *dax_dev;
+
+   dax_dev = xfs_find_daxdev_for_inode(file_inode(filp));
/*
-* We don't support synchronous mappings for non-DAX files. At least
-* until someone comes with a sensible use case.
+* We don't support synchronous mappings for non-DAX files and
+* for DAX files if underneath dax_device is not synchronous.
 */
-   if (!IS_DAX(file_inode(filp)) && (vma->vm_flags & VM_SYNC))
+   if (!daxdev_mapping_supported(vma, dax_dev))
return -EOPNOTSUPP;
 
file_accessed(filp);
-- 
2.20.1

Re: [Qemu-devel] [PATCH v3 3/8] tests: Add migration multifd test

[Juan Quintela]

Wei Yang  wrote:
> On Wed, May 15, 2019 at 02:15:39PM +0200, Juan Quintela wrote:
>>We set multifd-channels.
>>
>>Reviewed-by: Dr. David Alan Gilbert 
>>Reviewed-by: Thomas Huth 
>>Signed-off-by: Juan Quintela 
>>---
>> tests/migration-test.c | 48 ++
>> 1 file changed, 48 insertions(+)
>>
>>diff --git a/tests/migration-test.c b/tests/migration-test.c
>>index 0b25aa3d6c..ff480e0682 100644
>>--- a/tests/migration-test.c
>>+++ b/tests/migration-test.c
>>@@ -1028,6 +1028,53 @@ static void test_precopy_tcp(void)
>> g_free(uri);
>> }
>> 
>>+static void test_multifd_tcp(void)
>>+{
>>+char *uri;
>>+QTestState *from, *to;
>>+
>>+if (test_migrate_start(&from, &to, "tcp:127.0.0.1:0", false, false)) {
>>+return;
>>+}
>>+
>>+/*
>>+ * We want to pick a speed slow enough that the test completes
>>+ * quickly, but that it doesn't complete precopy even on a slow
>>+ * machine, so also set the downtime.
>>+ */
>>+/* 1 ms should make it not converge*/
>>+migrate_set_parameter_int(from, "downtime-limit", 1);
>>+/* 1GB/s */
>>+migrate_set_parameter_int(from, "max-bandwidth", 10);
>>+
>>+migrate_set_parameter_int(from, "multifd-channels", 2);
>>+migrate_set_parameter_int(to, "multifd-channels", 2);
>>+
>>+migrate_set_capability(from, "multifd", "true");
>>+migrate_set_capability(to, "multifd", "true");
>
> Hi, Juan
>
> In case to test multifd, what's the minimum configuration? Set multifd
> capability at both side is fine?

It is required.  For older configuration, we are a bit more lax about
that requirement.  For anything new, we want it to be set in both sides.

Thanks, Juan.

Re: [Qemu-devel] [PATCH v3 3/8] tests: Add migration multifd test

[Juan Quintela]

Wei Yang  wrote:
> On Wed, May 15, 2019 at 02:15:39PM +0200, Juan Quintela wrote:
>>We set multifd-channels.
>>
>>+wait_for_serial("dest_serial");
>>+wait_for_migration_complete(from);
>>+
>>+test_migrate_end(from, to, true);
>
> Miss free uri here?
>
> g_free(uri);

You are right.
Fixed.

>
> And
>
> Tested-by: Wei Yang 

Thanks.

Re: [Qemu-devel] [PATCH 2/5] block/dirty-bitmap: Refactor bdrv_can_store_new_bitmap

[Vladimir Sementsov-Ogievskiy]

08.06.2019 1:08, John Snow wrote:
> 
> 
> On 6/7/19 2:17 PM, Vladimir Sementsov-Ogievskiy wrote:
>> 07.06.2019 21:10, John Snow wrote:
>>>
>>>
>>> On 6/7/19 10:29 AM, Vladimir Sementsov-Ogievskiy wrote:
 06.06.2019 21:41, John Snow wrote:
> Instead of bdrv_can_store_new_bitmap, rework this as
> bdrv_add_persistent_dirty_bitmap. This makes a more obvious symmetry
> with bdrv_remove_persistent_dirty_bitmap. Most importantly, we are free
> to modify the driver state because we know we ARE adding a bitmap
> instead of simply being asked if we CAN store one.
>
> As part of this symmetry, move this function next to
> bdrv_remove_persistent_bitmap in block/dirty-bitmap.c.
>
> To cement this semantic distinction, use a bitmap object instead of the
> (name, granularity) pair as this allows us to set persistence as a
> condition of the "add" succeeding.
>
> Notably, the qcow2 implementation still does not actually store the new
> bitmap at add time, but merely ensures that it will be able to at store
> time.
>
> Signed-off-by: John Snow 
> ---
> block/qcow2.h|  5 ++---
> include/block/block.h|  2 --
> include/block/block_int.h|  5 ++---
> include/block/dirty-bitmap.h |  3 +++
> block.c  | 22 -
> block/dirty-bitmap.c | 38 
> block/qcow2-bitmap.c | 24 ++-
> block/qcow2.c|  2 +-
> blockdev.c   | 19 +++---
> 9 files changed, 68 insertions(+), 52 deletions(-)
>
> diff --git a/block/qcow2.h b/block/qcow2.h
> index fc1b0d3c1e..95d723d3c0 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -742,9 +742,8 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, 
> Error **errp);
> int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
> void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error 
> **errp);
> int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
> -bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs,
> -  const char *name,
> -  uint32_t granularity,
> +int qcow2_add_persistent_dirty_bitmap(BlockDriverState *bs,
> +  BdrvDirtyBitmap *bitmap,
>   Error **errp);
> void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs,
>   const char *name,
> diff --git a/include/block/block.h b/include/block/block.h
> index f9415ed740..6d520f3b59 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -683,8 +683,6 @@ void bdrv_add_child(BlockDriverState *parent, 
> BlockDriverState *child,
> Error **errp);
> void bdrv_del_child(BlockDriverState *parent, BdrvChild *child, Error 
> **errp);
> 
> -bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char 
> *name,
> - uint32_t granularity, Error **errp);
> /**
>  *
>  * bdrv_register_buf/bdrv_unregister_buf:
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index 06df2bda1b..93bbb66cd0 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -537,9 +537,8 @@ struct BlockDriver {
>  * field of BlockDirtyBitmap's in case of success.
>  */
> int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp);
> -bool (*bdrv_can_store_new_dirty_bitmap)(BlockDriverState *bs,
> -const char *name,
> -uint32_t granularity,
> +int (*bdrv_add_persistent_dirty_bitmap)(BlockDriverState *bs,
> +BdrvDirtyBitmap *bitmap,
> Error **errp);
> void (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
> const char *name,
> diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
> index 8044ace63e..c37edbe05f 100644
> --- a/include/block/dirty-bitmap.h
> +++ b/include/block/dirty-bitmap.h
> @@ -38,6 +38,9 @@ int bdrv_dirty_bitmap_check(const BdrvDirtyBitmap 
> *bitmap, uint32_t flags,
> Error **errp);
> void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap 
> *bitmap);
> void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs);
> +int bdrv_add_persistent_dirty_bitmap(Blo

Re: [Qemu-devel] [PATCH 2/7] iotests: amendment for benchmark output of 039 061 137

[Andrey Shinkevich]

On 10/06/2019 12:10, Daniel P. Berrangé wrote:
> On Sun, Jun 09, 2019 at 09:35:43PM +0300, Andrey Shinkevich wrote:
>> After including the Valgrind into the QEMU processes wrappers in the
>> common.rc script, the output for the tests 039 061 137 has been
>> changed and is to be amended.
> 
> The change to common.rc was done in the previous patch, so this
> fixup should be there too. Separating the fixup will impact people
> doing "git bisect" to track down bugs.
> 
> 
> Regards,
> Daniel
> 
Yes, I will in v2, thank you.
Andrey

Re: [Qemu-devel] [PATCH 2/4] block/dirty-bitmap: add hide/unhide API

[Vladimir Sementsov-Ogievskiy]

08.06.2019 1:39, John Snow wrote:
> 
> 
> On 6/3/19 8:00 AM, Vladimir Sementsov-Ogievskiy wrote:
>> Add functionality to make bitmap temporary anonymous. It will be used
>> to implement bitmap remove transaction action. We need hide bitmap
>> persistence too, as there are should not be unnamed persistent bitmaps.
>>
> 
> Ah, so this effectively ... "hides" a bitmap from any further
> transaction actions. It also "hides" it from getting flushed to disk...
> sort of?
> 
> The outer loop in store works with bdrv_dirty_bitmap_next, and we'll
> skip this bitmap because it's anonymous/not persistent.
> 
> There's a second loop where we iterate bm_list, and we'll skip storing
> this bitmap because that entry won't have an in-memory bitmap associated
> with it in bm_list.
> 
> ...But then we'll call update_ext_header_and_dir with the stale entries
> in bm_list?

Hidden is a temprory state, so, we should not go to close() in this state.
It's a state inside a transaction.

Stale entries in list will be marked IN_USE anyway, so there is no damage 
anyway.

> 
>> Signed-off-by: Vladimir Sementsov-Ogievskiy 
>> ---
>>   include/block/dirty-bitmap.h |  2 ++
>>   block/dirty-bitmap.c | 26 ++
>>   2 files changed, 28 insertions(+)
>>
>> diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
>> index 8044ace63e..542e437123 100644
>> --- a/include/block/dirty-bitmap.h
>> +++ b/include/block/dirty-bitmap.h
>> @@ -116,5 +116,7 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap 
>> *bitmap,
>>   BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
>> BdrvDirtyBitmap *bitmap,
>> Error **errp);
>> +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap);
>> +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap);
>>   
>>   #endif
>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>> index 49646a30e6..592964635e 100644
>> --- a/block/dirty-bitmap.c
>> +++ b/block/dirty-bitmap.c
>> @@ -35,6 +35,10 @@ struct BdrvDirtyBitmap {
>>   bool busy;  /* Bitmap is busy, it can't be used via 
>> QMP */
>>   BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
>>   char *name; /* Optional non-empty unique ID */
>> +char *hidden_name;  /* Backup of @name for removal transaction
>> +   action. Used for hide/unhide API. */
>> +bool hidden_persistent; /* Backup of @persistent for removal 
>> transaction
>> +   action. */
>>   int64_t size;   /* Size of the bitmap, in bytes */
>>   bool disabled;  /* Bitmap is disabled. It ignores all 
>> writes to
>>  the device */
>> @@ -849,3 +853,25 @@ out:
>>   qemu_mutex_unlock(src->mutex);
>>   }
>>   }
>> +
>> +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap)
>> +{
>> +qemu_mutex_lock(bitmap->mutex);
>> +assert(!bitmap->hidden_name);
>> +bitmap->hidden_name = bitmap->name;
>> +bitmap->hidden_persistent = bitmap->persistent;
>> +bitmap->name = NULL;
>> +bitmap->persistent = false;
>> +qemu_mutex_unlock(bitmap->mutex);
>> +}
>> +
>> +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap)
>> +{
>> +qemu_mutex_lock(bitmap->mutex);
>> +assert(!bitmap->name);
>> +bitmap->name = bitmap->hidden_name;
>> +bitmap->persistent = bitmap->hidden_persistent;
>> +bitmap->hidden_name = NULL;
>> +bitmap->hidden_persistent = false;
>> +qemu_mutex_unlock(bitmap->mutex);
>> +}
>>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v3 5/8] migration: Add multifd-compress parameter

[Juan Quintela]

Markus Armbruster  wrote:
> Juan Quintela  writes:
>> +case MIGRATION_PARAMETER_MULTIFD_COMPRESS:
>> +p->has_multifd_compress = true;
>> +visit_type_MultifdCompress(v, param, &compress_type, &err);
>> +if (err) {
>> +break;
>> +}
>> +if (compress_type < 0 || compress_type >= MULTIFD_COMPRESS__MAX) {
>> +error_setg(&err, "Invalid multifd_compress option %s", 
>> valuestr);
>> +break;
>> +}
>
> This should never happen.  If you want to check anyway, make it an
> assertion.
>
> Just in case you don't believe me, or are curious:
>
> visit_type_MultifdCompress() wraps around visit_type_enum(), passing it
> &MultifdCompress_lookup.
>
> Since @v is an input visitor, visit_type_enum() wraps around
> input_type_enum().
>
> input_type_enum() computes the value to store in @compress_type with
> qapi_enum_parse().
>
> To get here, visit_type_MultifdCompress() must have succeeded,
> i.e. visit_type_enum(), input_type_enum() and qapi_enum_parse() all
> succeded.
>
> On success, qapi_enum_parse() returns one of the values in
> MultifdCompress_lookup, i.e. a member of enum MultifdCompress other than
> MULTIFD_COMPRESS__MAX.

Fixed, thanks.

>> @@ -3353,6 +3362,9 @@ void migration_global_dump(Monitor *mon)
>>  #define DEFINE_PROP_MIG_CAP(name, x) \
>>  DEFINE_PROP_BOOL(name, MigrationState, enabled_capabilities[x], false)
>>  
>> +#define DEFINE_PROP_MULTIFD_COMPRESS(_n, _s, _f, _d) \
>> +DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_multifd_compress, 
>> MultifdCompress)
>> +
>
> Did you forget to move this?

It appears that yes.  I tried to move the other part out of
qdev-properties, failed.  And then forgot to move this bit.

Thanks.

Later, Juan.

Re: [Qemu-devel] [PATCH 3/4] qapi: implement block-dirty-bitmap-remove transaction action

[Vladimir Sementsov-Ogievskiy]

08.06.2019 1:57, John Snow wrote:
> 
> 
> On 6/3/19 8:00 AM, Vladimir Sementsov-Ogievskiy wrote:
>> It is used to do transactional movement of the bitmap (which is
>> possible in conjunction with merge command). Transactional bitmap
>> movement is needed in scenarios with external snapshot, when we don't
>> want to leave copy of the bitmap in the base image.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy 
>> ---
>>   qapi/transaction.json |  2 ++
>>   blockdev.c| 74 +++
>>   2 files changed, 70 insertions(+), 6 deletions(-)
>>
>> diff --git a/qapi/transaction.json b/qapi/transaction.json
>> index 95edb78227..da95b804aa 100644
>> --- a/qapi/transaction.json
>> +++ b/qapi/transaction.json
>> @@ -45,6 +45,7 @@
>>   #
>>   # - @abort: since 1.6
>>   # - @block-dirty-bitmap-add: since 2.5
>> +# - @block-dirty-bitmap-remove: since 4.1
>>   # - @block-dirty-bitmap-clear: since 2.5
>>   # - @block-dirty-bitmap-enable: since 4.0
>>   # - @block-dirty-bitmap-disable: since 4.0
>> @@ -61,6 +62,7 @@
>> 'data': {
>>  'abort': 'Abort',
>>  'block-dirty-bitmap-add': 'BlockDirtyBitmapAdd',
>> +   'block-dirty-bitmap-remove': 'BlockDirtyBitmap',
>>  'block-dirty-bitmap-clear': 'BlockDirtyBitmap',
>>  'block-dirty-bitmap-enable': 'BlockDirtyBitmap',
>>  'block-dirty-bitmap-disable': 'BlockDirtyBitmap',
>> diff --git a/blockdev.c b/blockdev.c
>> index 5b3eef0d3e..0d9aa7f0a1 100644
>> --- a/blockdev.c
>> +++ b/blockdev.c
>> @@ -2135,6 +2135,46 @@ static void 
>> block_dirty_bitmap_merge_prepare(BlkActionState *common,
>>   errp);
>>   }
>>   
>> +static BdrvDirtyBitmap *do_block_dirty_bitmap_remove(
>> +const char *node, const char *name, bool release,
>> +BlockDriverState **bitmap_bs, Error **errp);
>> +
>> +static void block_dirty_bitmap_remove_prepare(BlkActionState *common,
>> +  Error **errp)
>> +{
>> +BlockDirtyBitmap *action;
>> +BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
>> + common, common);
>> +
>> +if (action_check_completion_mode(common, errp) < 0) {
>> +return;
>> +}
>> +
>> +action = common->action->u.block_dirty_bitmap_remove.data;
>> +
>> +state->bitmap = do_block_dirty_bitmap_remove(action->node, action->name,
>> + false, &state->bs, errp);
>> +if (state->bitmap) {
>> +bdrv_dirty_bitmap_hide(state->bitmap);
>> +}
>> +}
>> +
>> +static void block_dirty_bitmap_remove_abort(BlkActionState *common)
>> +{
>> +BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
>> + common, common);
>> +
>> +bdrv_dirty_bitmap_unhide(state->bitmap);
>> +}
>> +
>> +static void block_dirty_bitmap_remove_commit(BlkActionState *common)
>> +{
>> +BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
>> + common, common);
>> +
>> +bdrv_release_dirty_bitmap(state->bs, state->bitmap);
>> +}
>> +
>>   static void abort_prepare(BlkActionState *common, Error **errp)
>>   {
>>   error_setg(errp, "Transaction aborted using Abort action");
>> @@ -2212,6 +2252,12 @@ static const BlkActionOps actions[] = {
>>   .commit = block_dirty_bitmap_free_backup,
>>   .abort = block_dirty_bitmap_restore,
>>   },
>> +[TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_REMOVE] = {
>> +.instance_size = sizeof(BlockDirtyBitmapState),
>> +.prepare = block_dirty_bitmap_remove_prepare,
>> +.commit = block_dirty_bitmap_remove_commit,
>> +.abort = block_dirty_bitmap_remove_abort,
>> +},
>>   /* Where are transactions for MIRROR, COMMIT and STREAM?
>>* Although these blockjobs use transaction callbacks like the backup 
>> job,
>>* these jobs do not necessarily adhere to transaction semantics.
>> @@ -2870,20 +2916,21 @@ void qmp_block_dirty_bitmap_add(const char *node, 
>> const char *name,
>>   bdrv_dirty_bitmap_set_persistence(bitmap, persistent);
>>   }
>>   
>> -void qmp_block_dirty_bitmap_remove(const char *node, const char *name,
>> -   Error **errp)
>> +static BdrvDirtyBitmap *do_block_dirty_bitmap_remove(
>> +const char *node, const char *name, bool release,
>> +BlockDriverState **bitmap_bs, Error **errp)
> 
> Hm, why does the hide feature need to copy the persistent bit when we're
> removing it here anyway?
> 
> If release is false, we still call bdrv_remove_persistent_dirty_bitmap,
> yeah?
> 
> And when we go to undo it, we won't have undone the persistent removal,
> right?

Hm, good question. I remember there was bad thing on storing persistent bitmap,
as this code is not prepared for unnamed persistent bitmaps. Aha, I was wrong in
my previous a

Re: [Qemu-devel] [PATCH v3 5/8] migration: Add multifd-compress parameter

[Juan Quintela]

Wei Yang  wrote:
> On Wed, May 15, 2019 at 02:15:41PM +0200, Juan Quintela wrote:
>>Signed-off-by: Juan Quintela 
>>diff --git a/tests/migration-test.c b/tests/migration-test.c
>>index 65d5e256a7..8a1ccc2516 100644
>>--- a/tests/migration-test.c
>>+++ b/tests/migration-test.c
>
> Well, may I suggest to split the test into another one?

In next patch, I get another multifd method.  Think of this patch as a
"rename", more than something different.

Thanks, Juan.

Re: [Qemu-devel] [PATCH 2/4] block/dirty-bitmap: add hide/unhide API

[Vladimir Sementsov-Ogievskiy]

10.06.2019 12:33, Vladimir Sementsov-Ogievskiy wrote:
> 08.06.2019 1:39, John Snow wrote:
>>
>>
>> On 6/3/19 8:00 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> Add functionality to make bitmap temporary anonymous. It will be used
>>> to implement bitmap remove transaction action. We need hide bitmap
>>> persistence too, as there are should not be unnamed persistent bitmaps.
>>>
>>
>> Ah, so this effectively ... "hides" a bitmap from any further
>> transaction actions. It also "hides" it from getting flushed to disk...
>> sort of?
>>
>> The outer loop in store works with bdrv_dirty_bitmap_next, and we'll
>> skip this bitmap because it's anonymous/not persistent.
>>
>> There's a second loop where we iterate bm_list, and we'll skip storing
>> this bitmap because that entry won't have an in-memory bitmap associated
>> with it in bm_list.
>>
>> ...But then we'll call update_ext_header_and_dir with the stale entries
>> in bm_list?
> 
> Hidden is a temprory state, so, we should not go to close() in this state.
> It's a state inside a transaction.
> 
> Stale entries in list will be marked IN_USE anyway, so there is no damage 
> anyway.

I'm wrong, storing inside transaction is exactly what test does.. Hmm, so, 
seems you
are right, we are storing stale IN_USE bitmaps. No serious damage, but bad 
design:
creating inconsistent bitmaps in each snapshot. I need to fix it in v2 somehow.

> 
>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy 
>>> ---
>>>   include/block/dirty-bitmap.h |  2 ++
>>>   block/dirty-bitmap.c | 26 ++
>>>   2 files changed, 28 insertions(+)
>>>
>>> diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
>>> index 8044ace63e..542e437123 100644
>>> --- a/include/block/dirty-bitmap.h
>>> +++ b/include/block/dirty-bitmap.h
>>> @@ -116,5 +116,7 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap 
>>> *bitmap,
>>>   BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
>>>     BdrvDirtyBitmap *bitmap,
>>>     Error **errp);
>>> +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap);
>>> +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap);
>>>   #endif
>>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>>> index 49646a30e6..592964635e 100644
>>> --- a/block/dirty-bitmap.c
>>> +++ b/block/dirty-bitmap.c
>>> @@ -35,6 +35,10 @@ struct BdrvDirtyBitmap {
>>>   bool busy;  /* Bitmap is busy, it can't be used via 
>>> QMP */
>>>   BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
>>>   char *name; /* Optional non-empty unique ID */
>>> +    char *hidden_name;  /* Backup of @name for removal transaction
>>> +   action. Used for hide/unhide API. */
>>> +    bool hidden_persistent; /* Backup of @persistent for removal 
>>> transaction
>>> +   action. */
>>>   int64_t size;   /* Size of the bitmap, in bytes */
>>>   bool disabled;  /* Bitmap is disabled. It ignores all 
>>> writes to
>>>  the device */
>>> @@ -849,3 +853,25 @@ out:
>>>   qemu_mutex_unlock(src->mutex);
>>>   }
>>>   }
>>> +
>>> +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap)
>>> +{
>>> +    qemu_mutex_lock(bitmap->mutex);
>>> +    assert(!bitmap->hidden_name);
>>> +    bitmap->hidden_name = bitmap->name;
>>> +    bitmap->hidden_persistent = bitmap->persistent;
>>> +    bitmap->name = NULL;
>>> +    bitmap->persistent = false;
>>> +    qemu_mutex_unlock(bitmap->mutex);
>>> +}
>>> +
>>> +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap)
>>> +{
>>> +    qemu_mutex_lock(bitmap->mutex);
>>> +    assert(!bitmap->name);
>>> +    bitmap->name = bitmap->hidden_name;
>>> +    bitmap->persistent = bitmap->hidden_persistent;
>>> +    bitmap->hidden_name = NULL;
>>> +    bitmap->hidden_persistent = false;
>>> +    qemu_mutex_unlock(bitmap->mutex);
>>> +}
>>>
> 
> 


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH] target/mips: Add implementation of DSPRAM

[Mateja Marjanovic]

On 8.6.19. 07:32, Aleksandar Markovic wrote:



On Jun 6, 2019 3:49 PM, "Mateja Marjanovic" 
mailto:mateja.marjano...@rt-rk.com>> wrote:

>
> From: Mateja Marjanovic >

>
> Add support for DSPRAM (Data Scratch Pad RAM). It still needs
> some minor fixing, but the structure is right.
>
> Signed-off-by: Mateja Marjanovic >

> ---

Please consider splitting the patch into several logical units.

Adding Philippe.


I will do that in v2.

Regards,
Mateja


Thanks,
Aleksandar

>  default-configs/mips-softmmu-common.mak |   1 +
>  hw/mips/cps.c                           |  28 +-
>  hw/misc/Makefile.objs                   |   1 +
>  hw/misc/mips_dspram.c                   | 153 


>  include/hw/mips/cps.h                   |   2 +
>  include/hw/misc/mips_dspram.h           |  46 ++
>  target/mips/cpu.h                       |   9 +-
>  target/mips/internal.h                  |   3 +-
>  target/mips/op_helper.c                 |  14 +++
>  target/mips/translate.c                 |   8 ++
>  target/mips/translate_init.inc.c        |   2 +
>  11 files changed, 262 insertions(+), 5 deletions(-)
>  create mode 100644 hw/misc/mips_dspram.c
>  create mode 100644 include/hw/misc/mips_dspram.h
>
> diff --git a/default-configs/mips-softmmu-common.mak 
b/default-configs/mips-softmmu-common.mak

> index ded7498..d3f85b0 100644
> --- a/default-configs/mips-softmmu-common.mak
> +++ b/default-configs/mips-softmmu-common.mak
> @@ -35,6 +35,7 @@ CONFIG_ISA_TESTDEV=y
>  CONFIG_EMPTY_SLOT=y
>  CONFIG_MIPS_CPS=y
>  CONFIG_MIPS_ITU=y
> +CONFIG_MIPS_DSPRAM=y
>  CONFIG_I2C=y
>  CONFIG_R4K=y
>  CONFIG_MALTA=y
> diff --git a/hw/mips/cps.c b/hw/mips/cps.c
> index fc97f59..0205556 100644
> --- a/hw/mips/cps.c
> +++ b/hw/mips/cps.c
> @@ -90,7 +90,8 @@ static void mips_cps_realize(DeviceState *dev, 
Error **errp)

>
>      cpu = MIPS_CPU(first_cpu);
>      env = &cpu->env;
> -    saar_present = (bool)env->saarp;
> +    saar_present = env->saarp;
> +    bool dspram_present = env->dspramp;
>
>      /* Inter-Thread Communication Unit */
>      if (itu_present) {
> @@ -102,7 +103,8 @@ static void mips_cps_realize(DeviceState *dev, 
Error **errp)
>          object_property_set_bool(OBJECT(&s->itu), saar_present, 
"saar-present",

>                                   &err);
>          if (saar_present) {
> -            qdev_prop_set_ptr(DEVICE(&s->itu), "saar", (void 
*)&env->CP0_SAAR);

> +            qdev_prop_set_ptr(DEVICE(&s->itu), "saar",
> +                              (void *) &env->CP0_SAAR[0]);
>          }
>          object_property_set_bool(OBJECT(&s->itu), true, "realized", 
&err);

>          if (err != NULL) {
> @@ -113,6 +115,28 @@ static void mips_cps_realize(DeviceState *dev, 
Error **errp)

>          memory_region_add_subregion(&s->container, 0,
> sysbus_mmio_get_region(SYS_BUS_DEVICE(&s->itu), 0));
>      }
> +    env->dspram = g_new0(MIPSDSPRAMState, 1);
> +
> +    /* Data Scratch Pad RAM */
> +    if (dspram_present) {
> +        if (!saar_present) {
> +            error_report("%s: DSPRAM requires SAAR registers", 
__func__);

> +            return;
> +        }
> +        object_initialize(&s->dspram, sizeof(MIPSDSPRAMState),
> +                          TYPE_MIPS_DSPRAM);
> +        qdev_set_parent_bus(DEVICE(&s->dspram), sysbus_get_default());
> +        qdev_prop_set_ptr(DEVICE(&s->dspram), "saar",
> +                          &env->CP0_SAAR[1]);
> + object_property_set_bool(OBJECT(&s->dspram), true, "realized", &err);
> +        if (err != NULL) {
> +            error_report("%s: DSPRAM initialisation failed", __func__);
> +            error_propagate(errp, err);
> +            return;
> +        }
> +        memory_region_add_subregion(&s->container, 0,
> + sysbus_mmio_get_region(SYS_BUS_DEVICE(&s->dspram), 0));
> +    }
>
>      /* Cluster Power Controller */
>      object_initialize(&s->cpc, sizeof(s->cpc), TYPE_MIPS_CPC);
> diff --git a/hw/misc/Makefile.objs b/hw/misc/Makefile.objs
> index 74c91d2..37c4108 100644
> --- a/hw/misc/Makefile.objs
> +++ b/hw/misc/Makefile.objs
> @@ -60,6 +60,7 @@ obj-$(CONFIG_STM32F2XX_SYSCFG) += stm32f2xx_syscfg.o
>  obj-$(CONFIG_MIPS_CPS) += mips_cmgcr.o
>  obj-$(CONFIG_MIPS_CPS) += mips_cpc.o
>  obj-$(CONFIG_MIPS_ITU) += mips_itu.o
> +obj-$(CONFIG_MIPS_DSPRAM) += mips_dspram.o
>  obj-$(CONFIG_MPS2_FPGAIO) += mps2-fpgaio.o
>  obj-$(CONFIG_MPS2_SCC) += mps2-scc.o
>
> diff --git a/hw/misc/mips_dspram.c b/hw/misc/mips_dspram.c
> new file mode 100644
> index 000..9bc155b
> --- /dev/null
> +++ b/hw/misc/mips_dspram.c
> @@ -0,0 +1,153 @@
> +/*
> + * Data Scratch Pad RAM
> + *
> + * Copyright (c) 2017 Imagination Technologies
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2 of the License, or (at your 

Re: [Qemu-devel] [PATCH v3 5/8] migration: Add multifd-compress parameter

[Juan Quintela]

Wei Yang  wrote:
> On Wed, May 15, 2019 at 02:15:41PM +0200, Juan Quintela wrote:
>>Signed-off-by: Juan Quintela 
>>@@ -1821,6 +1826,18 @@ void hmp_migrate_set_parameter(Monitor *mon, const 
>>QDict *qdict)
>> p->has_multifd_channels = true;
>> visit_type_int(v, param, &p->multifd_channels, &err);
>> break;
>>+case MIGRATION_PARAMETER_MULTIFD_COMPRESS:
>>+p->has_multifd_compress = true;
>>+visit_type_MultifdCompress(v, param, &compress_type, &err);
>>+if (err) {
>>+break;
>>+}
>>+if (compress_type < 0 || compress_type >= MULTIFD_COMPRESS__MAX) {
>
> A warning during build:
>
> hmp.c:1835:27: warning: comparison of unsigned enum expression < 0 is
> always false [-Wtautological-compare]
> if (compress_type < 0 || compress_type >= MULTIFD_COMPRESS__MAX) {

Fixed, see Markus reason for dropping this bit.

/me wonders why this didn't failed my compilation 

Thanks, Juan.

Re: [Qemu-devel] [PATCH 2/4] block/dirty-bitmap: add hide/unhide API

[Vladimir Sementsov-Ogievskiy]

10.06.2019 12:42, Vladimir Sementsov-Ogievskiy wrote:
> 10.06.2019 12:33, Vladimir Sementsov-Ogievskiy wrote:
>> 08.06.2019 1:39, John Snow wrote:
>>>
>>>
>>> On 6/3/19 8:00 AM, Vladimir Sementsov-Ogievskiy wrote:
 Add functionality to make bitmap temporary anonymous. It will be used
 to implement bitmap remove transaction action. We need hide bitmap
 persistence too, as there are should not be unnamed persistent bitmaps.

>>>
>>> Ah, so this effectively ... "hides" a bitmap from any further
>>> transaction actions. It also "hides" it from getting flushed to disk...
>>> sort of?
>>>
>>> The outer loop in store works with bdrv_dirty_bitmap_next, and we'll
>>> skip this bitmap because it's anonymous/not persistent.
>>>
>>> There's a second loop where we iterate bm_list, and we'll skip storing
>>> this bitmap because that entry won't have an in-memory bitmap associated
>>> with it in bm_list.
>>>
>>> ...But then we'll call update_ext_header_and_dir with the stale entries
>>> in bm_list?
>>
>> Hidden is a temprory state, so, we should not go to close() in this state.
>> It's a state inside a transaction.
>>
>> Stale entries in list will be marked IN_USE anyway, so there is no damage 
>> anyway.
> 
> I'm wrong, storing inside transaction is exactly what test does.. Hmm, so, 
> seems you
> are right, we are storing stale IN_USE bitmaps. No serious damage, but bad 
> design:
> creating inconsistent bitmaps in each snapshot. I need to fix it in v2 
> somehow.

Ahaha, and I'm again wrong, as my series do right thing: remove bitmap from the 
image
in .prepare, so there would not be stale entries, all is OK.

> 
>>
>>>
 Signed-off-by: Vladimir Sementsov-Ogievskiy 
 ---
   include/block/dirty-bitmap.h |  2 ++
   block/dirty-bitmap.c | 26 ++
   2 files changed, 28 insertions(+)

 diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
 index 8044ace63e..542e437123 100644
 --- a/include/block/dirty-bitmap.h
 +++ b/include/block/dirty-bitmap.h
 @@ -116,5 +116,7 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap 
 *bitmap,
   BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
     BdrvDirtyBitmap 
 *bitmap,
     Error **errp);
 +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap);
 +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap);
   #endif
 diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
 index 49646a30e6..592964635e 100644
 --- a/block/dirty-bitmap.c
 +++ b/block/dirty-bitmap.c
 @@ -35,6 +35,10 @@ struct BdrvDirtyBitmap {
   bool busy;  /* Bitmap is busy, it can't be used via 
 QMP */
   BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
   char *name; /* Optional non-empty unique ID */
 +    char *hidden_name;  /* Backup of @name for removal transaction
 +   action. Used for hide/unhide API. */
 +    bool hidden_persistent; /* Backup of @persistent for removal 
 transaction
 +   action. */
   int64_t size;   /* Size of the bitmap, in bytes */
   bool disabled;  /* Bitmap is disabled. It ignores all 
 writes to
  the device */
 @@ -849,3 +853,25 @@ out:
   qemu_mutex_unlock(src->mutex);
   }
   }
 +
 +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap)
 +{
 +    qemu_mutex_lock(bitmap->mutex);
 +    assert(!bitmap->hidden_name);
 +    bitmap->hidden_name = bitmap->name;
 +    bitmap->hidden_persistent = bitmap->persistent;
 +    bitmap->name = NULL;
 +    bitmap->persistent = false;
 +    qemu_mutex_unlock(bitmap->mutex);
 +}
 +
 +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap)
 +{
 +    qemu_mutex_lock(bitmap->mutex);
 +    assert(!bitmap->name);
 +    bitmap->name = bitmap->hidden_name;
 +    bitmap->persistent = bitmap->hidden_persistent;
 +    bitmap->hidden_name = NULL;
 +    bitmap->hidden_persistent = false;
 +    qemu_mutex_unlock(bitmap->mutex);
 +}

>>
>>
> 
> 


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v3 5/8] migration: Add multifd-compress parameter

[Juan Quintela]

Wei Yang  wrote:
> On Wed, May 15, 2019 at 02:15:41PM +0200, Juan Quintela wrote:
>>diff --git a/qapi/migration.json b/qapi/migration.json
>>index 9cfbaf8c6c..8ec1944b7a 100644
>>--- a/qapi/migration.json
>>+++ b/qapi/migration.json
>>@@ -482,6 +482,19 @@
>> # TODO either fuse back into MigrationParameters, or make
>>@@ -707,7 +726,8 @@
>> '*multifd-channels': 'int',
>> '*xbzrle-cache-size': 'size',
>> '*max-postcopy-bandwidth': 'size',
>>- '*max-cpu-throttle': 'int' } }
>>+ '*max-cpu-throttle': 'int',
>
> A tab at the beginning, it would be better to fix this :-)

The wonders of magit+emacs, I didn't saw this one.

Fixed.

Thanks, Juan.

Re: [Qemu-devel] [PATCH 2/4] block/dirty-bitmap: add hide/unhide API

[Vladimir Sementsov-Ogievskiy]

10.06.2019 12:42, Vladimir Sementsov-Ogievskiy wrote:
> 10.06.2019 12:33, Vladimir Sementsov-Ogievskiy wrote:
>> 08.06.2019 1:39, John Snow wrote:
>>>
>>>
>>> On 6/3/19 8:00 AM, Vladimir Sementsov-Ogievskiy wrote:
 Add functionality to make bitmap temporary anonymous. It will be used
 to implement bitmap remove transaction action. We need hide bitmap
 persistence too, as there are should not be unnamed persistent bitmaps.

>>>
>>> Ah, so this effectively ... "hides" a bitmap from any further
>>> transaction actions. It also "hides" it from getting flushed to disk...
>>> sort of?
>>>
>>> The outer loop in store works with bdrv_dirty_bitmap_next, and we'll
>>> skip this bitmap because it's anonymous/not persistent.
>>>
>>> There's a second loop where we iterate bm_list, and we'll skip storing
>>> this bitmap because that entry won't have an in-memory bitmap associated
>>> with it in bm_list.
>>>
>>> ...But then we'll call update_ext_header_and_dir with the stale entries
>>> in bm_list?
>>
>> Hidden is a temprory state, so, we should not go to close() in this state.
>> It's a state inside a transaction.
>>
>> Stale entries in list will be marked IN_USE anyway, so there is no damage 
>> anyway.
> 
> I'm wrong, storing inside transaction is exactly what test does.. Hmm, so, 
> seems you
> are right, we are storing stale IN_USE bitmaps. No serious damage, but bad 
> design:
> creating inconsistent bitmaps in each snapshot. I need to fix it in v2 
> somehow.

Ahaha, and I'm wrong again, as my series do right thing: remove bitmap from the 
image
in .prepare, so there will be no stale entries..

> 
>>
>>>
 Signed-off-by: Vladimir Sementsov-Ogievskiy 
 ---
   include/block/dirty-bitmap.h |  2 ++
   block/dirty-bitmap.c | 26 ++
   2 files changed, 28 insertions(+)

 diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
 index 8044ace63e..542e437123 100644
 --- a/include/block/dirty-bitmap.h
 +++ b/include/block/dirty-bitmap.h
 @@ -116,5 +116,7 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap 
 *bitmap,
   BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
     BdrvDirtyBitmap 
 *bitmap,
     Error **errp);
 +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap);
 +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap);
   #endif
 diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
 index 49646a30e6..592964635e 100644
 --- a/block/dirty-bitmap.c
 +++ b/block/dirty-bitmap.c
 @@ -35,6 +35,10 @@ struct BdrvDirtyBitmap {
   bool busy;  /* Bitmap is busy, it can't be used via 
 QMP */
   BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
   char *name; /* Optional non-empty unique ID */
 +    char *hidden_name;  /* Backup of @name for removal transaction
 +   action. Used for hide/unhide API. */
 +    bool hidden_persistent; /* Backup of @persistent for removal 
 transaction
 +   action. */
   int64_t size;   /* Size of the bitmap, in bytes */
   bool disabled;  /* Bitmap is disabled. It ignores all 
 writes to
  the device */
 @@ -849,3 +853,25 @@ out:
   qemu_mutex_unlock(src->mutex);
   }
   }
 +
 +void bdrv_dirty_bitmap_hide(BdrvDirtyBitmap *bitmap)
 +{
 +    qemu_mutex_lock(bitmap->mutex);
 +    assert(!bitmap->hidden_name);
 +    bitmap->hidden_name = bitmap->name;
 +    bitmap->hidden_persistent = bitmap->persistent;
 +    bitmap->name = NULL;
 +    bitmap->persistent = false;
 +    qemu_mutex_unlock(bitmap->mutex);
 +}
 +
 +void bdrv_dirty_bitmap_unhide(BdrvDirtyBitmap *bitmap)
 +{
 +    qemu_mutex_lock(bitmap->mutex);
 +    assert(!bitmap->name);
 +    bitmap->name = bitmap->hidden_name;
 +    bitmap->persistent = bitmap->hidden_persistent;
 +    bitmap->hidden_name = NULL;
 +    bitmap->hidden_persistent = false;
 +    qemu_mutex_unlock(bitmap->mutex);
 +}

>>
>>
> 
> 


-- 
Best regards,
Vladimir

Re: [Qemu-devel] QEMU/KVM migration backwards compatibility broken?

[Dr. David Alan Gilbert]

* Liran Alon (liran.a...@oracle.com) wrote:
> 
> > On 6 Jun 2019, at 16:31, Dr. David Alan Gilbert  wrote:
> > 
> >>> 
> >>> So we still need to tie subsections to machine types; that way
> >>> you don't send them to old qemu's and there for you don't have the
> >>> problem of the qemu receiving something it doesn't know.
> >> 
> >> I agree that if there is no way to skip a VMState subsection in the 
> >> stream, then we must
> >> have a way to specify to source QEMU to prevent sending this subsection to 
> >> destination…
> >> 
> >> I would suggest though that instead of having a flag tied to machine-type, 
> >> we will have a QMP command
> >> that can specify names of subsections we explicitly wish to be skipped 
> >> sending to destination even if their .needed() method returns true.
> > 
> > I don't like the thought of generically going behind the devices back;
> > it's pretty rare to have to do this, so adding a qmp command to tweak
> > properties that we've already got seems to make more sense to me.
> > 
> >> This seems like a more explicit approach and doesn’t come with the 
> >> down-side of forever not migrating this VMState subsection
> > Dave
> 
> If I understand you correctly, this is what you propose:
> 1) Have a .post_load() method for VMState subsections that depend on kernel 
> capability to fail migration in case capability do not exist.

Yes (wehther it fails or prints a warning depends on how significant the
capability is; if it's a guest crash then fail is probably best).

> 2) For specific problematic VMState subsections, add property such that it’s 
> .needed() method will return false in case the property is set to false 
> (value is true by default).
> 3) Have a QMP command that allows dynamically changing the value of these 
> properties.
> 4) Properties values are still tied to machine-type? I think not right?

Property values are initialised from the machine type; in your case
where you want to upgrade to use a new feature then you can use
(3) to change it.

> I instead propose the following:
> 1) Same as (1) above.
> 2) Add a MigrationParameter (and matching MigrationCapability) named 
> “avoid_state” that specifies list of subsection names to avoid sending in 
> migration even if their .needed() method will return false. i.e. We will 
> modify migration/vmstate.c to not even call .needed() method of such 
> subsection.
> 
> I believe the second proposal have the following advantages:
> 1) Less error-prone: .needed() methods are written only once and don’t need 
> to take into account additional properties when calculating if they are 
> required or not. Just depend on guest state.
> 2) Generic: We don’t require additional patch to add a new property to 
> support avoiding sending some subsection in case it doesn’t matter for some 
> workload. As we have discovered only late after msr_smi_count was added (by 
> me) at that point. Second approach allows avoid sending any subsection that 
> is deemed not important to guest workload by migration admin.
> 3) Not tied to machine-type: Properties are usually tied to machine-type as 
> they need to remain same forever for the lifetime of the guest. However, 
> migration parameters are per-migration and are meant to be tweaked and 
> changed. This allows a guest that used to run on old QEMU and moved to new 
> QEMU to now have better state saved for it’s next future migrations.
> 
> Currently we indeed have very rare cases like this ([git grep \"x-migrate | 
> wc -l] product only 4 results…) but I’m not sure it’s not only because we 
> haven’t analysed carefully the case of
> restored properties that it’s property depend on kernel capability.
> 
> As a start thought, we can start by at least agreeing to implement (1) and 
> consider the property VS MigrationParameter discussion for a later time.
> 
> What do you think?

I still don't like exposing a list of migration subsections into an
interface.

Dave

> -Liran
> 
> 
> 
> 
> 
> 
> 
> 
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] qgraph

[Natalia Fursova]

Hi there!
Read please our correspondence and answer on last question (Will it ok for
QOM conception?)
Also forwarding to QOM and QMP maintainers. Can you make comments on this
discussion?


Best regards,
Natalia

-Original Message-
From: Natalia Fursova [mailto:natalia.furs...@ispras.ru] 
Sent: Wednesday, June 05, 2019 5:23 PM
To: 'Paolo Bonzini'; 'qemu-devel@nongnu.org'; 'Паша'
Subject: RE: [Qemu-devel] qgraph

I see.
We need this opportunity and want to implement it. Will it ok for QOM
conception?



Best regards,
Natalia

-Original Message-
From: Paolo Bonzini [mailto:pbonz...@redhat.com] 
Sent: Wednesday, June 05, 2019 4:07 PM
To: Natalia Fursova; qemu-devel@nongnu.org; 'Паша'
Subject: Re: [Qemu-devel] qgraph

On 05/06/19 14:34, Natalia Fursova wrote:
> 
> Thank you for your answer. I would like to clarify something about the qmp
> commands.
> For example, consider SCSI controller "lsi53c895a". For getting
information
> we use two commands: "device-list-properties" and "qom-list-properties".
> Output consists of many properties, but there is no information about
> provided buses by this device. Is there a qmp command which provides this
> information?

Unfortunately there is no information in QMP about buses that are
provided.   qom-list-types gives the buses that are requested.

Paolo

Re: [Qemu-devel] [PATCH v3 6/8] migration: Make none operations into its own structure

[Juan Quintela]

Wei Yang  wrote:
> On Wed, May 15, 2019 at 02:15:42PM +0200, Juan Quintela wrote:
>>+
>>+MultifdMethods multifd_none_ops = {
>>+.send_prepare = none_send_prepare,
>>+.send_write = none_send_write,
>>+.recv_pages = none_recv_pages
>>+};
>>+
>> static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
>> {
>> MultiFDInit_t msg;
>>@@ -904,6 +938,8 @@ struct {
>> uint64_t packet_num;
>> /* send channels ready */
>> QemuSemaphore channels_ready;
>>+/* multifd ops */
>>+MultifdMethods *ops;
>> } *multifd_send_state;
>> 
>> /*
>>@@ -1093,6 +1129,8 @@ static void *multifd_send_thread(void *opaque)
>> /* initial packet */
>> p->num_packets = 1;
>> 
>>+multifd_send_state->ops = &multifd_none_ops;
>>+
>
> I am afraid it is not a good practice to assign ops when each thread starts
> work.

Agreed.

Thanks, Juan.

Re: [Qemu-devel] [PATCH v3 6/8] migration: Make none operations into its own structure

[Juan Quintela]

"Dr. David Alan Gilbert"  wrote:
> * Juan Quintela (quint...@redhat.com) wrote:
>> It will be used later.
>
> 'none' is confusing - I think this is no-compression specifically -
> right?
> I'd be happy with something abbreviated like 'nocomp'

Got into nocomp.
>> @@ -1093,6 +1129,8 @@ static void *multifd_send_thread(void *opaque)
>>  /* initial packet */
>>  p->num_packets = 1;
>>  
>> +multifd_send_state->ops = &multifd_none_ops;
>> +
>
> I agree with Wei Yang that is a bad idea; that should be done once
> before the first thread is started.

Also fixed.

Thanks, Juan.

Re: [Qemu-devel] [PULL 00/12] Miscellaneous patches for 2019-06-07

[Peter Maydell]

On Fri, 7 Jun 2019 at 19:21, Markus Armbruster  wrote:
>
> The following changes since commit 0d74f3b4277a7ecb0ccb80c865797d11f8e466f5:
>
>   Merge remote-tracking branch 
> 'remotes/vivier2/tags/trivial-branch-pull-request' into staging (2019-06-06 
> 14:09:14 +0100)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/armbru.git tags/pull-misc-2019-06-07
>
> for you to fetch changes up to 516741ebae8e2baef456c10897569725663157dd:
>
>   MAINTAINERS: Polish headline decorations (2019-06-07 16:30:59 +0200)
>
> 
> Miscellaneous patches for 2019-06-07
>
> 
> Markus Armbruster (11):
>   qemu-common: Move tcg_enabled() etc. to sysemu/tcg.h
>   qemu-common: Move qemu_isalnum() etc. to qemu/ctype.h
>   Include qemu/module.h where needed, drop it from qemu-common.h
>   Include qemu-common.h exactly where needed
>   Normalize position of header guard
>   scripts/clean-header-guards: Fix handling of trailing comments
>   Clean up a few header guard symbols
>   Supply missing header guards
>   Clean up a header guard symbols (again)
>   MAINTAINERS: Improve section headlines
>   MAINTAINERS: Polish headline decorations
>
> Philippe Mathieu-Daudé (1):
>   MAINTAINERS: Remove duplicate entries of qemu-devel@nongnu.org
>

Compile failure, I'm afraid, on OSX, NetBSD, FreeBSD and OpenBSD:

/Users/pm215/src/qemu-for-merges/net/tap-bsd.c:68:9: error: implicit
declaration of function 'TFR' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
TFR(fd = open(dname, O_RDWR));
^
/Users/pm215/src/qemu-for-merges/net/tap-bsd.c:68:9: error: this
function declaration is not a prototype [-Werror,-Wstrict-prototypes]

thanks
-- PMM

Re: [Qemu-devel] [PULL 0/2] Machine Core queue, 2019-06-07

[Peter Maydell]

On Fri, 7 Jun 2019 at 19:32, Eduardo Habkost  wrote:
>
> The following changes since commit 185b7ccc11354cbd69b6d53bf8d831dd964f6c88:
>
>   Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20190607-2' into 
> staging (2019-06-07 15:24:13 +0100)
>
> are available in the Git repository at:
>
>   git://github.com/ehabkost/qemu.git tags/machine-next-pull-request
>
> for you to fetch changes up to 754f1dbcf1d4d543167e3212366055761c30bcf4:
>
>   accel: Remove unused AccelClass::opt_name attribute (2019-06-07 15:28:46 
> -0300)
>
> 
> Machine Core queue, 2019-06-07
>
> * Improve CPU hotplug error message
>   (Laurent Vivier)
> * Remove unused AccelClass::opt_name field
>   (Wainer dos Santos Moschetta)
>
> 


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.1
for any user-visible changes.

-- PMM

Re: [Qemu-devel] [PATCH v2] migration/xbzrle: update cache and current_data in one place

[Dr. David Alan Gilbert]

* Wei Yang (richardw.y...@linux.intel.com) wrote:
> When we are not in the last_stage, we need to update the cache if page
> is not the same.
> 
> Currently this procedure is scattered in two places and mixed with
> encoding status check.
> 
> This patch extract this general step out to make the code a little bit
> easy to read.
> 
> Signed-off-by: Wei Yang 
> 

Reviewed-by: Dr. David Alan Gilbert 

> ---
> v2: give more comment on the behavior
> ---
>  migration/ram.c | 25 +++--
>  1 file changed, 15 insertions(+), 10 deletions(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index e9b40d636d..17cc9b2b44 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -1594,25 +1594,30 @@ static int save_xbzrle_page(RAMState *rs, uint8_t 
> **current_data,
>  encoded_len = xbzrle_encode_buffer(prev_cached_page, XBZRLE.current_buf,
> TARGET_PAGE_SIZE, XBZRLE.encoded_buf,
> TARGET_PAGE_SIZE);
> +
> +/*
> + * Update the cache contents, so that it corresponds to the data
> + * sent, in all cases except where we skip the page.
> + */
> +if (!last_stage && encoded_len != 0) {
> +memcpy(prev_cached_page, XBZRLE.current_buf, TARGET_PAGE_SIZE);
> +/*
> + * In the case where we couldn't compress, ensure that the caller
> + * sends the data from the cache, since the guest might have
> + * changed the RAM since we copied it.
> + */
> +*current_data = prev_cached_page;
> +}
> +
>  if (encoded_len == 0) {
>  trace_save_xbzrle_page_skipping();
>  return 0;
>  } else if (encoded_len == -1) {
>  trace_save_xbzrle_page_overflow();
>  xbzrle_counters.overflow++;
> -/* update data in the cache */
> -if (!last_stage) {
> -memcpy(prev_cached_page, *current_data, TARGET_PAGE_SIZE);
> -*current_data = prev_cached_page;
> -}
>  return -1;
>  }
>  
> -/* we need to update the data in the cache, in order to get the same 
> data */
> -if (!last_stage) {
> -memcpy(prev_cached_page, XBZRLE.current_buf, TARGET_PAGE_SIZE);
> -}
> -
>  /* Send XBZRLE based compressed page */
>  bytes_xbzrle = save_page_header(rs, rs->f, block,
>  offset | RAM_SAVE_FLAG_XBZRLE);
> -- 
> 2.19.1
> 
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

[Qemu-devel] [PATCH] monitor: increase amount of data for monitor to read

[Denis Plotnikov]

Right now QMP and HMP monitors read 1 byte at a time from the socket, which
is very inefficient. With 100+ VMs on the host this easily reasults in
a lot of unnecessary system calls and CPU usage in the system.

This patch changes the amount of data to read to 4096 bytes, which matches
buffer size on the channel level. Fortunately, monitor protocol is
synchronous right now thus we should not face side effects in reality.

Signed-off-by: Denis V. Lunev 
Signed-off-by: Denis Plotnikov 
---
 include/monitor/monitor.h | 2 +-
 monitor.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index c1b40a9cac..afa1ed34a4 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -14,7 +14,7 @@ extern __thread Monitor *cur_mon;
 #define MONITOR_USE_CONTROL   0x04
 #define MONITOR_USE_PRETTY0x08
 
-#define QMP_REQ_QUEUE_LEN_MAX 8
+#define QMP_REQ_QUEUE_LEN_MAX 4096
 
 bool monitor_cur_is_qmp(void);
 
diff --git a/monitor.c b/monitor.c
index 4807bbe811..a08e020b61 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4097,7 +4097,7 @@ static int monitor_can_read(void *opaque)
 {
 Monitor *mon = opaque;
 
-return !atomic_mb_read(&mon->suspend_cnt);
+return !atomic_mb_read(&mon->suspend_cnt) ? 4096 : 0;
 }
 
 /*
-- 
2.17.0

[Qemu-devel] [PATCH 2/7] configure: set $PYTHON to a full path

[Paolo Bonzini]

This will make it possible to replace it in a shebang line.

Signed-off-by: Paolo Bonzini 
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 203dece..0814a5f 100755
--- a/configure
+++ b/configure
@@ -918,7 +918,7 @@ for binary in "${PYTHON-python3}" python python2
 do
 if has "$binary"
 then
-python="$binary"
+python=$(type -p "$binary")
 break
 fi
 done
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[Paolo Bonzini]

Hi all,

this is just a simple example of introducing Meson into the build system,
converting a few contrib/ subprojects to use it instead of Makefile.objs.
There are certainly many rough edges, but I wanted to throw this out to
show what it could look like.

I chose contrib/ because it's a simple example of how the current build
system causes clutter in the toplevel Makefiles and how interrelated the
Makefile, Makefile.objs and contrib/*/Makefile.objs file are even for
such simple executables.

Paolo

Paolo Bonzini (7):
  configure: do not include $(...) variables in config-host.mak
  configure: set $PYTHON to a full path
  configure: integrate Meson in the build system
  libvhost-user: convert to Meson
  vhost-user-blk: convert to Meson
  vhost-user-scsi: convert to Meson
  rdmacm-mux: convert to Meson

 .gitignore|   5 +-
 Makefile  |  26 +-
 Makefile.objs |   6 -
 configure |  66 ++-
 contrib/libvhost-user/Makefile.objs   |   1 -
 contrib/libvhost-user/meson.build |   7 +
 contrib/rdmacm-mux/Makefile.objs  |   3 -
 contrib/rdmacm-mux/meson.build|   6 +
 contrib/vhost-user-blk/Makefile.objs  |   1 -
 contrib/vhost-user-blk/meson.build|   3 +
 contrib/vhost-user-scsi/Makefile.objs |   1 -
 contrib/vhost-user-scsi/meson.build   |   3 +
 meson.build   |  20 +
 scripts/ninjatool.py  | 964 ++
 14 files changed, 1066 insertions(+), 46 deletions(-)
 delete mode 100644 contrib/libvhost-user/Makefile.objs
 create mode 100644 contrib/libvhost-user/meson.build
 delete mode 100644 contrib/rdmacm-mux/Makefile.objs
 create mode 100644 contrib/rdmacm-mux/meson.build
 delete mode 100644 contrib/vhost-user-blk/Makefile.objs
 create mode 100644 contrib/vhost-user-blk/meson.build
 delete mode 100644 contrib/vhost-user-scsi/Makefile.objs
 create mode 100644 contrib/vhost-user-scsi/meson.build
 create mode 100644 meson.build
 create mode 100644 scripts/ninjatool.py

-- 
1.8.3.1

[Qemu-devel] [PATCH 6/7] vhost-user-scsi: convert to Meson

[Paolo Bonzini]

The libiscsi pkg-config information is extracted from config-host.mak and
used to link vhost-user-blk.

Signed-off-by: Paolo Bonzini 
---
 .gitignore| 2 +-
 Makefile  | 3 ---
 Makefile.objs | 3 ---
 contrib/vhost-user-scsi/Makefile.objs | 1 -
 contrib/vhost-user-scsi/meson.build   | 3 +++
 meson.build   | 3 +++
 6 files changed, 7 insertions(+), 8 deletions(-)
 delete mode 100644 contrib/vhost-user-scsi/Makefile.objs
 create mode 100644 contrib/vhost-user-scsi/meson.build

diff --git a/.gitignore b/.gitignore
index 20637a1..3934eff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -63,7 +63,7 @@
 /qemu-version.h.tmp
 /module_block.h
 /scsi/qemu-pr-helper
-/vhost-user-scsi
+/contrib/vhost-user-scsi/vhost-user-scsi
 /contrib/vhost-user-blk/vhost-user-blk
 /fsdev/virtfs-proxy-helper
 *.tmp
diff --git a/Makefile b/Makefile
index da290fa..bff097c 100644
--- a/Makefile
+++ b/Makefile
@@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
 rdmacm-mux-obj-y \
-vhost-user-scsi-obj-y \
 vhost-user-input-obj-y \
 vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
@@ -629,8 +628,6 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) 
$(COMMON_LDADDS)
 ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
$(call LINK, $^)
 endif
-vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) 
contrib/libvhost-user/libvhost-user.a
-   $(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index 644e2bd..adf2f92 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -116,9 +116,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
-vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
-vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
 rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
diff --git a/contrib/vhost-user-scsi/Makefile.objs 
b/contrib/vhost-user-scsi/Makefile.objs
deleted file mode 100644
index e83a38a..000
--- a/contrib/vhost-user-scsi/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-vhost-user-scsi-obj-y = vhost-user-scsi.o
diff --git a/contrib/vhost-user-scsi/meson.build 
b/contrib/vhost-user-scsi/meson.build
new file mode 100644
index 000..b80398c
--- /dev/null
+++ b/contrib/vhost-user-scsi/meson.build
@@ -0,0 +1,3 @@
+executable('vhost-user-scsi', files('vhost-user-scsi.c'),
+   link_with: libvhost_user,
+   dependencies: [glib, libiscsi])
diff --git a/meson.build b/meson.build
index c4cad8f..bc7fbea 100644
--- a/meson.build
+++ b/meson.build
@@ -10,6 +10,9 @@ add_project_arguments(config_host['QEMU_INCLUDES'].split(),
 
 glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
   link_args: config_host['GLIB_LIBS'].split())
+libiscsi = declare_dependency(compile_args: 
config_host['LIBISCSI_CFLAGS'].split(),
+  link_args: config_host['LIBISCSI_LIBS'].split())
 
 subdir('contrib/libvhost-user')
 subdir('contrib/vhost-user-blk')
+subdir('contrib/vhost-user-scsi')
-- 
1.8.3.1

[Qemu-devel] [PATCH 1/7] configure: do not include $(...) variables in config-host.mak

[Paolo Bonzini]

This ensures that Meson will be able to reuse the results of
the tests that are performed in the configure script.

Signed-off-by: Paolo Bonzini 
---
 configure | 32 
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/configure b/configure
index 6cdcfb2..203dece 100755
--- a/configure
+++ b/configure
@@ -597,7 +597,7 @@ QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv 
-std=gnu99 $QEMU_CFLAGS"
 QEMU_CFLAGS="-Wall -Wundef -Wwrite-strings -Wmissing-prototypes $QEMU_CFLAGS"
 QEMU_CFLAGS="-Wstrict-prototypes -Wredundant-decls $QEMU_CFLAGS"
 QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
$QEMU_CFLAGS"
-QEMU_INCLUDES="-iquote . -iquote \$(SRC_PATH) -iquote \$(SRC_PATH)/accel/tcg 
-iquote \$(SRC_PATH)/include"
+QEMU_INCLUDES="-iquote . -iquote ${source_path} -iquote 
${source_path}/accel/tcg -iquote ${source_path}/include"
 if test "$debug_info" = "yes"; then
 CFLAGS="-g $CFLAGS"
 LDFLAGS="-g $LDFLAGS"
@@ -896,7 +896,7 @@ Linux)
   linux="yes"
   linux_user="yes"
   kvm="yes"
-  QEMU_INCLUDES="-I\$(SRC_PATH)/linux-headers -I$PWD/linux-headers 
$QEMU_INCLUDES"
+  QEMU_INCLUDES="-I${source_path}/linux-headers -I$PWD/linux-headers 
$QEMU_INCLUDES"
   supported_os="yes"
   libudev="yes"
 ;;
@@ -4090,8 +4090,8 @@ EOF
   symlink "$source_path/dtc/Makefile" "dtc/Makefile"
   symlink "$source_path/dtc/scripts" "dtc/scripts"
   fi
-  fdt_cflags="-I\$(SRC_PATH)/dtc/libfdt"
-  fdt_ldflags="-L\$(BUILD_DIR)/dtc/libfdt"
+  fdt_cflags="-I${source_path}/dtc/libfdt"
+  fdt_ldflags="-L$PWD/dtc/libfdt"
   fdt_libs="$fdt_libs"
   elif test "$fdt" = "yes" ; then
   # Not a git build & no libfdt found, prompt for system install
@@ -5040,13 +5040,13 @@ case "$capstone" in
   git_submodules="${git_submodules} capstone"
 fi
 mkdir -p capstone
-QEMU_CFLAGS="$QEMU_CFLAGS -I\$(SRC_PATH)/capstone/include"
+QEMU_CFLAGS="$QEMU_CFLAGS -I${source_path}/capstone/include"
 if test "$mingw32" = "yes"; then
   LIBCAPSTONE=capstone.lib
 else
   LIBCAPSTONE=libcapstone.a
 fi
-LIBS="-L\$(BUILD_DIR)/capstone -lcapstone $LIBS"
+LIBS="-L$PWD/capstone -lcapstone $LIBS"
 ;;
 
   system)
@@ -5951,8 +5951,8 @@ case "$slirp" in
   git_submodules="${git_submodules} slirp"
 fi
 mkdir -p slirp
-slirp_cflags="-I\$(SRC_PATH)/slirp/src -I\$(BUILD_DIR)/slirp/src"
-slirp_libs="-L\$(BUILD_DIR)/slirp -lslirp"
+slirp_cflags="-I${source_path}/slirp/src -I$PWD/slirp/src"
+slirp_libs="-L$PWD/slirp -lslirp"
 ;;
 
   system)
@@ -7307,21 +7307,21 @@ if test "$sheepdog" = "yes" ; then
 fi
 
 if test "$tcg_interpreter" = "yes"; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/tci $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/tci $QEMU_INCLUDES"
 elif test "$ARCH" = "sparc64" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/sparc $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/sparc $QEMU_INCLUDES"
 elif test "$ARCH" = "s390x" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/s390 $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/s390 $QEMU_INCLUDES"
 elif test "$ARCH" = "x86_64" || test "$ARCH" = "x32" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/i386 $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/i386 $QEMU_INCLUDES"
 elif test "$ARCH" = "ppc64" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/ppc $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/ppc $QEMU_INCLUDES"
 elif test "$ARCH" = "riscv32" || test "$ARCH" = "riscv64" ; then
-  QEMU_INCLUDES="-I\$(SRC_PATH)/tcg/riscv $QEMU_INCLUDES"
+  QEMU_INCLUDES="-I${source_path}/tcg/riscv $QEMU_INCLUDES"
 else
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/\$(ARCH) $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/\$(ARCH) $QEMU_INCLUDES"
 fi
-QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg $QEMU_INCLUDES"
+QEMU_INCLUDES="-iquote ${source_path}/tcg $QEMU_INCLUDES"
 
 echo "TOOLS=$tools" >> $config_host_mak
 echo "ROMS=$roms" >> $config_host_mak
-- 
1.8.3.1

[Qemu-devel] [PATCH 7/7] rdmacm-mux: convert to Meson

[Paolo Bonzini]

We can use config-host.mak to decide whether the tool has to be built,
apart from that the conversion is straightforward.

Signed-off-by: Paolo Bonzini 
---
 .gitignore   | 1 +
 Makefile | 5 -
 Makefile.objs| 1 -
 contrib/rdmacm-mux/Makefile.objs | 3 ---
 contrib/rdmacm-mux/meson.build   | 6 ++
 meson.build  | 2 ++
 6 files changed, 9 insertions(+), 9 deletions(-)
 delete mode 100644 contrib/rdmacm-mux/Makefile.objs
 create mode 100644 contrib/rdmacm-mux/meson.build

diff --git a/.gitignore b/.gitignore
index 3934eff..b8d38a8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -63,6 +63,7 @@
 /qemu-version.h.tmp
 /module_block.h
 /scsi/qemu-pr-helper
+/contrib/rdmacm-mux/rdmacm-mux
 /contrib/vhost-user-scsi/vhost-user-scsi
 /contrib/vhost-user-blk/vhost-user-blk
 /fsdev/virtfs-proxy-helper
diff --git a/Makefile b/Makefile
index bff097c..713f301 100644
--- a/Makefile
+++ b/Makefile
@@ -425,7 +425,6 @@ dummy := $(call unnest-vars,, \
 elf2dmp-obj-y \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
-rdmacm-mux-obj-y \
 vhost-user-input-obj-y \
 vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
@@ -629,10 +628,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) 
$(COMMON_LDADDS)
$(call LINK, $^)
 endif
 
-rdmacm-mux$(EXESUF): LIBS += "-libumad"
-rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
-   $(call LINK, $^)
-
 vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) 
contrib/libvhost-user/libvhost-user.a
$(call LINK, $^)
 
diff --git a/Makefile.objs b/Makefile.objs
index adf2f92..cf02f63 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
 
diff --git a/contrib/rdmacm-mux/Makefile.objs b/contrib/rdmacm-mux/Makefile.objs
deleted file mode 100644
index 3df744a..000
--- a/contrib/rdmacm-mux/Makefile.objs
+++ /dev/null
@@ -1,3 +0,0 @@
-ifdef CONFIG_PVRDMA
-rdmacm-mux-obj-y = main.o
-endif
diff --git a/contrib/rdmacm-mux/meson.build b/contrib/rdmacm-mux/meson.build
new file mode 100644
index 000..8451756
--- /dev/null
+++ b/contrib/rdmacm-mux/meson.build
@@ -0,0 +1,6 @@
+if config_host['CONFIG_PVRDMA'] == 'y'
+  # if not found, CONFIG_PVRDMA should not be set
+  libumad = cc.find_library('ibumad', required: true)
+  executable('rdmacm-mux', files('main.c'),
+ dependencies: [glib, libumad])
+endif
diff --git a/meson.build b/meson.build
index bc7fbea..2fc6111 100644
--- a/meson.build
+++ b/meson.build
@@ -1,4 +1,5 @@
 project('qemu', 'c', meson_version: '>=0.50.999')
+cc = meson.get_compiler('c')
 
 kconfig = import('unstable-kconfig')
 config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak')
@@ -14,5 +15,6 @@ libiscsi = declare_dependency(compile_args: 
config_host['LIBISCSI_CFLAGS'].split
   link_args: config_host['LIBISCSI_LIBS'].split())
 
 subdir('contrib/libvhost-user')
+subdir('contrib/rdmacm-mux')
 subdir('contrib/vhost-user-blk')
 subdir('contrib/vhost-user-scsi')
-- 
1.8.3.1

[Qemu-devel] [PATCH 3/7] configure: integrate Meson in the build system

[Paolo Bonzini]

The Meson build system is integrated in the existing configure/make steps
by invoking Meson from the configure script and converting Meson's build.ninja
rules to an included Makefile.

Signed-off-by: Paolo Bonzini 
---
 Makefile |   9 +
 configure|  30 ++
 meson.build  |   9 +
 scripts/ninjatool.py | 964 +++
 4 files changed, 1012 insertions(+)
 create mode 100644 meson.build
 create mode 100644 scripts/ninjatool.py

diff --git a/Makefile b/Makefile
index 8e2fc66..b8f802c 100644
--- a/Makefile
+++ b/Makefile
@@ -48,6 +48,15 @@ git-submodule-update:
 endif
 endif
 
+export NINJA=./ninjatool
+Makefile.ninja: build.ninja ninjatool
+   ./ninjatool -t ninja2make --omit dist uninstall < $< > $@
+-include Makefile.ninja
+
+ninjatool: $(SRC_PATH)/scripts/ninjatool.py
+   sed -e '1c\' -e '#! $(PYTHON)' $< > $@
+   chmod +x $@
+
 .git-submodule-status: git-submodule-update config-host.mak
 
 # Check that we're not trying to do an out-of-tree build from
diff --git a/configure b/configure
index 0814a5f..b8c3c58 100755
--- a/configure
+++ b/configure
@@ -493,6 +493,7 @@ docker="no"
 debug_mutex="no"
 libpmem=""
 default_devices="yes"
+meson=meson
 
 # cross compilers defaults, can be overridden with --cross-cc-ARCH
 cross_cc_aarch64="aarch64-linux-gnu-gcc"
@@ -983,6 +984,8 @@ for opt do
   ;;
   --python=*) python="$optarg"
   ;;
+  --meson=*) meson="$optarg"
+  ;;
   --gcov=*) gcov_tool="$optarg"
   ;;
   --smbd=*) smbd="$optarg"
@@ -1685,6 +1688,7 @@ Advanced options (experts only):
   --make=MAKE  use specified make [$make]
   --install=INSTALLuse specified install [$install]
   --python=PYTHON  use specified python [$python]
+  --meson=PYTHON   use specified meson [$meson]
   --smbd=SMBD  use specified smbd [$smbd]
   --with-git=GIT   use specified git [$git]
   --static enable static build [$static]
@@ -1850,6 +1854,11 @@ then
 error_exit "Python not found. Use --python=/path/to/python"
 fi
 
+if ! has "$meson"
+then
+error_exit "Meson not found. Use --meson=/path/to/meson"
+fi
+
 # Note that if the Python conditional here evaluates True we will exit
 # with status 1 which is a shell 'false' value.
 if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then
@@ -7983,6 +7992,27 @@ echo "# Automatically generated by configure - do not 
modify" > "$iotests_common
 echo >> "$iotests_common_env"
 echo "export PYTHON='$python'" >> "$iotests_common_env"
 
+# bootstrap ninjatool, we need it before Make runs
+if ! test -x ninjatool; then
+  sed -e '1c\' -e "#! $python" ${source_path}/scripts/ninjatool.py > ninjatool
+  chmod +x ninjatool
+fi
+rm -rf meson-private meson-info meson-logs
+NINJA=$PWD/ninjatool $python $meson setup \
+   --prefix "$prefix" \
+   --libdir "$libdir" \
+   --libexecdir "$libexecdir" \
+   --bindir "$bindir" \
+   --includedir "$includedir" \
+   --datadir "$datadir" \
+   --mandir "$mandir" \
+   --sysconfdir "$sysconfdir" \
+   --localstatedir "$local_statedir" \
+   $(test "$strip_opt" = yes && echo --strip) \
+   --buildtype $(if test "$debug" = yes; then echo debug; else echo 
release; fi) \
+   "$PWD" "$source_path"
+
+
 # Save the configure command line for later reuse.
 cat =0.50.999')
+
+kconfig = import('unstable-kconfig')
+config_host = kconfig.load(meson.current_build_dir() / 'config-host.mak')
+
+add_project_arguments(config_host['QEMU_CFLAGS'].split(),
+  language: 'c')
+add_project_arguments(config_host['QEMU_INCLUDES'].split(),
+  language: 'c')
diff --git a/scripts/ninjatool.py b/scripts/ninjatool.py
new file mode 100644
index 000..6d90919
--- /dev/null
+++ b/scripts/ninjatool.py
@@ -0,0 +1,964 @@
+#! /usr/bin/env python3
+
+"""Python module for parsing and processing .ninja files.
+
+   Author: Paolo Bonzini
+
+   Copyright (C) 2019 Red Hat, Inc."""
+
+from collections import namedtuple, defaultdict
+import sys
+import os
+import re
+import json
+import argparse
+
+
+class InvalidArgumentError(Exception):
+pass
+
+
+#  lexer and parser 
+
+BUILD = 1
+POOL = 2
+RULE = 3
+DEFAULT = 4
+EQUALS = 5
+COLON = 6
+PIPE = 7
+PIPE2 = 8
+IDENT = 9
+INCLUDE = 10
+INDENT = 11
+EOL = 12
+
+
+class LexerError(Exception):
+pass
+
+
+class ParseError(Exception):
+pass
+
+
+class NinjaParserEvents(object):
+def __init__(self, parser):
+self.parser = parser
+
+def literal_token(self, word, in_path=False):
+word = word.replace('$', '$$')
+if in_path:
+word = word.replace(' ', '$ ').replace(':', '$:')
+return word
+
+def variable_expansion_token(self, varname):
+  

[Qemu-devel] [PATCH 4/7] libvhost-user: convert to Meson

[Paolo Bonzini]

Because libqemuutil.a is not converted yet, selected files have to be
compiled twice, once with Meson and once with Makefile.objs.  Apart
from this the conversion is straightforward.

Signed-off-by: Paolo Bonzini 
---
 Makefile| 10 --
 Makefile.objs   |  1 -
 contrib/libvhost-user/Makefile.objs |  1 -
 contrib/libvhost-user/meson.build   |  7 +++
 meson.build |  2 ++
 5 files changed, 13 insertions(+), 8 deletions(-)
 delete mode 100644 contrib/libvhost-user/Makefile.objs
 create mode 100644 contrib/libvhost-user/meson.build

diff --git a/Makefile b/Makefile
index b8f802c..6a3461e 100644
--- a/Makefile
+++ b/Makefile
@@ -426,7 +426,6 @@ dummy := $(call unnest-vars,, \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
 rdmacm-mux-obj-y \
-libvhost-user-obj-y \
 vhost-user-scsi-obj-y \
 vhost-user-blk-obj-y \
 vhost-user-input-obj-y \
@@ -529,7 +528,6 @@ Makefile: $(version-obj-y)
 # Build libraries
 
 libqemuutil.a: $(util-obj-y) $(trace-obj-y) $(stub-obj-y)
-libvhost-user.a: $(libvhost-user-obj-y) $(util-obj-y) $(stub-obj-y)
 
 ##
 
@@ -632,21 +630,21 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) 
$(COMMON_LDADDS)
 ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
$(call LINK, $^)
 endif
-vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) libvhost-user.a
+vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) 
contrib/libvhost-user/libvhost-user.a
$(call LINK, $^)
-vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) libvhost-user.a
+vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) 
contrib/libvhost-user/libvhost-user.a
$(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
$(call LINK, $^)
 
-vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) $(libvhost-user-obj-y) 
libqemuutil.a libqemustub.a
+vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) 
contrib/libvhost-user/libvhost-user.a
$(call LINK, $^)
 
 ifdef CONFIG_VHOST_USER_INPUT
 ifdef CONFIG_LINUX
-vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) libvhost-user.a 
libqemuutil.a
+vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
 
 # build by default, do not install
diff --git a/Makefile.objs b/Makefile.objs
index c8337fa..4f2fa6a 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -116,7 +116,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-libvhost-user-obj-y = contrib/libvhost-user/
 vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
 vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
 vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
diff --git a/contrib/libvhost-user/Makefile.objs 
b/contrib/libvhost-user/Makefile.objs
deleted file mode 100644
index ef3778e..000
--- a/contrib/libvhost-user/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-libvhost-user-obj-y += libvhost-user.o libvhost-user-glib.o
diff --git a/contrib/libvhost-user/meson.build 
b/contrib/libvhost-user/meson.build
new file mode 100644
index 000..7ef610a
--- /dev/null
+++ b/contrib/libvhost-user/meson.build
@@ -0,0 +1,7 @@
+libvhost_user = static_library('vhost-user',
+   [files('libvhost-user.c', 'libvhost-user-glib.c'),
+meson.source_root() / 'stubs/error-printf.c',
+meson.source_root() / 'stubs/monitor.c',
+meson.source_root() / 'util/error.c',
+meson.source_root() / 'util/qemu-error.c',
+meson.source_root() / 'util/memfd.c'])
diff --git a/meson.build b/meson.build
index b683d70..a6748f9 100644
--- a/meson.build
+++ b/meson.build
@@ -7,3 +7,5 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(),
   language: 'c')
 add_project_arguments(config_host['QEMU_INCLUDES'].split(),
   language: 'c')
+
+subdir('contrib/libvhost-user')
-- 
1.8.3.1

[Qemu-devel] [PATCH 5/7] vhost-user-blk: convert to Meson

[Paolo Bonzini]

The GLib pkg-config information is extracted from config-host.mak and
used to link vhost-user-blk.

Signed-off-by: Paolo Bonzini 
---
 .gitignore   | 2 +-
 Makefile | 3 ---
 Makefile.objs| 1 -
 configure| 2 ++
 contrib/vhost-user-blk/Makefile.objs | 1 -
 contrib/vhost-user-blk/meson.build   | 3 +++
 meson.build  | 4 
 7 files changed, 10 insertions(+), 6 deletions(-)
 delete mode 100644 contrib/vhost-user-blk/Makefile.objs
 create mode 100644 contrib/vhost-user-blk/meson.build

diff --git a/.gitignore b/.gitignore
index fd6e6c3..20637a1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -64,7 +64,7 @@
 /module_block.h
 /scsi/qemu-pr-helper
 /vhost-user-scsi
-/vhost-user-blk
+/contrib/vhost-user-blk/vhost-user-blk
 /fsdev/virtfs-proxy-helper
 *.tmp
 *.[1-9]
diff --git a/Makefile b/Makefile
index 6a3461e..da290fa 100644
--- a/Makefile
+++ b/Makefile
@@ -427,7 +427,6 @@ dummy := $(call unnest-vars,, \
 ivshmem-server-obj-y \
 rdmacm-mux-obj-y \
 vhost-user-scsi-obj-y \
-vhost-user-blk-obj-y \
 vhost-user-input-obj-y \
 vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
@@ -632,8 +631,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) 
$(COMMON_LDADDS)
 endif
 vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) 
contrib/libvhost-user/libvhost-user.a
$(call LINK, $^)
-vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) 
contrib/libvhost-user/libvhost-user.a
-   $(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index 4f2fa6a..644e2bd 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -119,7 +119,6 @@ ivshmem-server-obj-$(CONFIG_IVSHMEM) = 
contrib/ivshmem-server/
 vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
 vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
 vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
-vhost-user-blk-obj-y = contrib/vhost-user-blk/
 rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
diff --git a/configure b/configure
index b8c3c58..f00a9e4 100755
--- a/configure
+++ b/configure
@@ -7364,6 +7364,8 @@ echo "CFLAGS_NOPIE=$CFLAGS_NOPIE" >> $config_host_mak
 echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
 echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak
 echo "QEMU_INCLUDES=$QEMU_INCLUDES" >> $config_host_mak
+echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
+echo "GLIB_LIBS=$glib_libs" >> $config_host_mak
 if test "$sparse" = "yes" ; then
   echo "CC   := REAL_CC=\"\$(CC)\" cgcc"   >> $config_host_mak
   echo "CPP  := REAL_CC=\"\$(CPP)\" cgcc"  >> $config_host_mak
diff --git a/contrib/vhost-user-blk/Makefile.objs 
b/contrib/vhost-user-blk/Makefile.objs
deleted file mode 100644
index 72e2cdc..000
--- a/contrib/vhost-user-blk/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-vhost-user-blk-obj-y = vhost-user-blk.o
diff --git a/contrib/vhost-user-blk/meson.build 
b/contrib/vhost-user-blk/meson.build
new file mode 100644
index 000..6c00b3b
--- /dev/null
+++ b/contrib/vhost-user-blk/meson.build
@@ -0,0 +1,3 @@
+executable('vhost-user-blk', files('vhost-user-blk.c'),
+   link_with: libvhost_user,
+   dependencies: glib)
diff --git a/meson.build b/meson.build
index a6748f9..c4cad8f 100644
--- a/meson.build
+++ b/meson.build
@@ -8,4 +8,8 @@ add_project_arguments(config_host['QEMU_CFLAGS'].split(),
 add_project_arguments(config_host['QEMU_INCLUDES'].split(),
   language: 'c')
 
+glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
+  link_args: config_host['GLIB_LIBS'].split())
+
 subdir('contrib/libvhost-user')
+subdir('contrib/vhost-user-blk')
-- 
1.8.3.1

Re: [Qemu-devel] [PULL 0/1] Usb 20190607 patches

[Peter Maydell]

On Fri, 7 Jun 2019 at 20:14, Gerd Hoffmann  wrote:
>
> The following changes since commit 442bac16a6cd708a9f87adb0a263f9d833f03ed5:
>
>   usb-tablet: fix serial compat property (2019-05-29 07:10:02 +0200)
>
> are available in the Git repository at:
>
>   git://git.kraxel.org/qemu tags/usb-20190607-pull-request
>
> for you to fetch changes up to e39b8a66d2af913e873256aa77ea71a179dfe5e0:
>
>   usb-mtp: refactor the flow of usb_mtp_write_data (2019-06-07 18:56:14 +0200)
>
> 
> usb-mtp: refactor the flow of usb_mtp_write_data
>
> 
>
> Bandan Das (1):
>   usb-mtp: refactor the flow of usb_mtp_write_data
>
>  hw/usb/dev-mtp.c | 57 +---
>  1 file changed, 30 insertions(+), 27 deletions(-)


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.1
for any user-visible changes.

-- PMM

Re: [Qemu-devel] qgraph

[Andreas Färber]

Hi Natalia,

Am 10.06.19 um 11:53 schrieb Natalia Fursova:
> Hi there!
> Read please our correspondence and answer on last question (Will it ok for
> QOM conception?)
> Also forwarding to QOM and QMP maintainers. Can you make comments on this
> discussion?

Your question doesn't make sense grammatically or conceptually. As Paolo
explained below, QOM is a pure object model, with object types/classes
and properties. Buses are just object instances attached as properties
and don't necessarily even need their own type of bus object (e.g, CPU).
An answer you don't like doesn't change by asking it to other people...
The information is all there, you just need to interpret it correctly.

There is no technical discussion (no concrete proposal of yours) to
comment on here, and kindly refer to last week's change of maintainers.

You would be better off just explaining what you really want to achieve.

Regards,
Andreas

> -Original Message-
> From: Natalia Fursova [mailto:natalia.furs...@ispras.ru] 
> Sent: Wednesday, June 05, 2019 5:23 PM
> To: 'Paolo Bonzini'; 'qemu-devel@nongnu.org'; 'Паша'
> Subject: RE: [Qemu-devel] qgraph
> 
> I see.
> We need this opportunity and want to implement it. Will it ok for QOM
> conception?
> 
> 
> 
> Best regards,
> Natalia
> 
> -Original Message-
> From: Paolo Bonzini [mailto:pbonz...@redhat.com] 
> Sent: Wednesday, June 05, 2019 4:07 PM
> To: Natalia Fursova; qemu-devel@nongnu.org; 'Паша'
> Subject: Re: [Qemu-devel] qgraph
> 
> On 05/06/19 14:34, Natalia Fursova wrote:
>>
>> Thank you for your answer. I would like to clarify something about the qmp
>> commands.
>> For example, consider SCSI controller "lsi53c895a". For getting
> information
>> we use two commands: "device-list-properties" and "qom-list-properties".
>> Output consists of many properties, but there is no information about
>> provided buses by this device. Is there a qmp command which provides this
>> information?
> 
> Unfortunately there is no information in QMP about buses that are
> provided.   qom-list-types gives the buses that are requested.
> 
> Paolo
> 


-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)

Re: [Qemu-devel] qgraph

[Paolo Bonzini]

On 10/06/19 13:57, Andreas Färber wrote:
> Your question doesn't make sense grammatically or conceptually. As Paolo
> explained below, QOM is a pure object model, with object types/classes
> and properties. Buses are just object instances attached as properties
> and don't necessarily even need their own type of bus object (e.g, CPU).
> An answer you don't like doesn't change by asking it to other people...
> The information is all there, you just need to interpret it correctly.
> 
> There is no technical discussion (no concrete proposal of yours) to
> comment on here, and kindly refer to last week's change of maintainers.
> 
> You would be better off just explaining what you really want to achieve.

Well, that was explained upthread---finding out what device can be
plugged where.

Let's see what is in QOM right now:

$ qemu-kvm -qmp unix:foo.sock,server,nowait -device virtio-scsi-pci,id=vs
$ ./qmp/qom-list -s ~/foo.sock /machine/peripheral/vs|less

There is a "virtio-bus" here, and iside it there is a scsi-bus.

$ ./qmp/qom-list -s ~/foo.sock /machine/peripheral/vs/virtio-bus/child[0]/
vs.0/

I guess you could add to virtio-scsi-pci a class property for the bus,
and then make "vs.0" an alias to that class property.  This would allow
you find buses by enumerating the class properties.

Paolo

Re: [Qemu-devel] kvm / virsh snapshot management

[Stefan Hajnoczi]

On Sat, Jun 01, 2019 at 08:12:01PM -0400, Gary Dale wrote:
> A while back I converted a raw disk image to qcow2 to be able to use
> snapshots. However I realize that I may not really understand exactly how
> snapshots work. In this particular case, I'm only talking about internal
> snapshots currently as there seems to be some differences of opinion as to
> whether internal or external are safer/more reliable. I'm also only talking
> about shutdown state snapshots, so it should just be the disk that is
> snapshotted.
> 
> As I understand it, the first snapshot freezes the base image and subsequent
> changes in the virtual machine's disk are stored elsewhere in the qcow2 file
> (remember, only internal snapshots). If I take a second snapshot, that
> freezes the first one, and subsequent changes are now in third location.
> Each new snapshot is incremental to the one that preceded it rather than
> differential to the base image. Each new snapshot is a child of the previous
> one.

Internal snapshots are not incremental or differential at the qcow2
level, they are simply a separate L1/L2 table pointing to data clusters.
In other words, they are an independent set of metadata showing the full
state of the image at the point of the snapshot.  qcow2 does not track
relationships between snapshots and parents/children.

> 
> One explanation I've seen of the process is if I delete a snapshot, the
> changes it contains are merged with its immediate child.

Nope.  Deleting a snapshot decrements the reference count on all its
data clusters.  If a data cluster's reference count reaches zero it will
be freed.  That's all, there is no additional data movement or
reorganization aside from this.

> So if I deleted the
> first snapshot, the base image stays the same but any data that has changed
> since the base image is now in the second snapshot's location. The merge
> with children explanation also implies that the base image is never touched
> even if the first snapshot is deleted.
> 
> But if I delete a snapshot that has no children, is that essentially the
> same as reverting to the point that snapshot was created and all subsequent
> disk changes are lost? Or does it merge down to the parent snapshot? If I
> delete all snapshots, would that revert to the base image?

No.  qcow2 has the concept of the current disk state of the running VM -
what you get when you boot the guest - and the snapshots - they are
read-only.

When you delete snapshots the current disk state (running VM) is
unaffected.

When you apply a snapshot this throws away the current disk state and
uses the snapshot as the new current disk state.  The read-only snapshot
itself is not modified in any way and you can apply the same snapshot
again as many times as you wish later.

> 
> I've seen it explained that a snapshot is very much like a timestamp so
> deleting a timestamp removes the dividing line between writes that occurred
> before and after that time, so that data is really only removed if I revert
> to some time stamp - all writes after that point are discarded. In this
> explanation, deleting the oldest timestamp is essentially updating the base
> image. Deleting all snapshots would leave me with the base image fully
> updated.
> 
> Frankly, the second explanation sounds more reasonable to me, without having
> to figure out how copy-on-write works,  But I'm dealing with important data
> here and I don't want to mess it up by mishandling the snapshots.
> 
> Can some provide a little clarity on this? Thanks!

If you want an analogy then git(1) is a pretty good one.  qcow2 internal
snapshots are like git tags.  Unlike branches, tags are immutable.  In
qcow2 you only have a master branch (the current disk state) from which
you can create a new tag or you can use git-checkout(1) to apply a
snapshot (discarding whatever your current disk state is).

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v4 0/3] numa: deprecate '-numa node, mem' and default memory distribution

[Igor Mammedov]

On Fri, 07 Jun 2019 19:28:58 +0200
Markus Armbruster  wrote:

> Igor Mammedov  writes:
> 
> > Changes since v3:
> >   - simplify series by dropping idea of showing property values in 
> > "qom-list-properties"
> > and use MachineInfo in QAPI schema instead  
> 
> Where did "[PATCH v3 1/6] pc: fix possible NULL pointer dereference in
> pc_machine_get_device_memory_region_size()" go?  It fixes a crash bug...

I'll post it as separate patch as it's not more related to this series

Re: [Qemu-devel] [PATCH v4 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Igor Mammedov]

On Fri, 07 Jun 2019 19:39:17 +0200
Markus Armbruster  wrote:

> Igor Mammedov  writes:
> 
> > Legacy '-numa node,mem' option has a number of issues and mgmt often
> > defaults to it. Unfortunately it's no possible to replace it with
> > an alternative '-numa memdev' without breaking migration compatibility.
> > What's possible though is to deprecate it, keeping option working with
> > old machine types only.
> >
> > In order to help users to find out if being deprecated CLI option
> > '-numa node,mem' is still supported by particular machine type, add new
> > "numa-mem-supported" property to MachineInfo description in QAPI schema.  
> 
> Suggest s/to MachineInfo description in QAPI schema/to output of
> query-machines/, because query-machines is the external interface people
> know.

fixed

> 
> > "numa-mem-supported" is set to 'true' for machines that currently support
> > NUMA, but it will be flipped to 'false' later on, once deprecation period
> > expires and kept 'true' only for old machine types that used to support
> > the legacy option so it won't break existing configuration that are using
> > it.
> >
> > Signed-off-by: Igor Mammedov 
> > ---
> >
> > Notes:
> > v4:
> >   * drop idea to use "qom-list-properties" and use MachineInfo instead
> > which could be inspected with 'query-machines'
> >
> >  include/hw/boards.h | 3 +++
> >  hw/arm/virt.c   | 1 +
> >  hw/i386/pc.c| 1 +
> >  hw/ppc/spapr.c  | 1 +
> >  qapi/misc.json  | 5 -
> >  vl.c| 1 +
> >  6 files changed, 11 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 6f7916f..86894b6 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -158,6 +158,8 @@ typedef struct {
> >   * @kvm_type:
> >   *Return the type of KVM corresponding to the kvm-type string option or
> >   *computed based on other criteria such as the host kernel 
> > capabilities.
> > + * @numa_mem_supported:
> > + *true if '--numa node.mem' option is supported and false otherwise
> >   */
> >  struct MachineClass {
> >  /*< private >*/
> > @@ -210,6 +212,7 @@ struct MachineClass {
> >  bool ignore_boot_device_suffixes;
> >  bool smbus_no_migration_support;
> >  bool nvdimm_supported;
> > +bool numa_mem_supported;
> >  
> >  HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> > DeviceState *dev);
> > diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> > index bf54f10..481a603 100644
> > --- a/hw/arm/virt.c
> > +++ b/hw/arm/virt.c
> > @@ -1943,6 +1943,7 @@ static void virt_machine_class_init(ObjectClass *oc, 
> > void *data)
> >  assert(!mc->get_hotplug_handler);
> >  mc->get_hotplug_handler = virt_machine_get_hotplug_handler;
> >  hc->plug = virt_machine_device_plug_cb;
> > +mc->numa_mem_supported = true;
> >  }
> >  
> >  static void virt_instance_init(Object *obj)
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 2632b73..05b8368 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -2747,6 +2747,7 @@ static void pc_machine_class_init(ObjectClass *oc, 
> > void *data)
> >  nc->nmi_monitor_handler = x86_nmi;
> >  mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
> >  mc->nvdimm_supported = true;
> > +mc->numa_mem_supported = true;
> >  
> >  object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int",
> >  pc_machine_get_device_memory_region_size, NULL,
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 2ef3ce4..265ecfb 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -4336,6 +4336,7 @@ static void spapr_machine_class_init(ObjectClass *oc, 
> > void *data)
> >   * in which LMBs are represented and hot-added
> >   */
> >  mc->numa_mem_align_shift = 28;
> > +mc->numa_mem_supported = true;
> >  
> >  smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF;
> >  smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;  
> 
> This is correct when the TYPE_VIRT_MACHINE, TYPE_PC_MACHINE and
> TYPE_SPAPR_MACHINE are exactly the machines supporting NUMA.  How could
> I check that?

We don't have an interface to communicate that to users as far as I know.

Eduardo pointed out to the patch that might serve as starting point for it 
though.

> 
> > diff --git a/qapi/misc.json b/qapi/misc.json
> > index 8b3ca4f..d0bdccb 100644
> > --- a/qapi/misc.json
> > +++ b/qapi/misc.json
> > @@ -2018,12 +2018,15 @@
> >  #
> >  # @hotpluggable-cpus: cpu hotplug via -device is supported (since 2.7.0)
> >  #
> > +# @numa-mem-supported: true if '-numa node,mem' option is supported by 
> > machine
> > +#  type and false otherwise (since 4.1)  
> 
> 
> "by the machine type", for consistency with @cpu-max.  Also, rather long
> line.

fixed

> 
> > +#
> >  # Since: 1.2.0
> >  ##
> >  { 'struct': 'MachineInfo',
> >'data': { 'name': 'str', '*alias': 'str',
> >  '*is-default'

Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonz...@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration
Type: series
Message-id: 1560165301-39026-1-git-send-email-pbonz...@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
   6960005212..19735c837a  master -> master
From https://github.com/patchew-project/qemu
 * [new tag]   
patchew/1560165301-39026-1-git-send-email-pbonz...@redhat.com -> 
patchew/1560165301-39026-1-git-send-email-pbonz...@redhat.com
Switched to a new branch 'test'
d681fb0695 rdmacm-mux: convert to Meson
2c12fcafd1 vhost-user-scsi: convert to Meson
7b87823a4c vhost-user-blk: convert to Meson
45233abe03 libvhost-user: convert to Meson
090f670ef8 configure: integrate Meson in the build system
9d9a4b1926 configure: set $PYTHON to a full path
d47d1b7f13 configure: do not include $(...) variables in config-host.mak

=== OUTPUT BEGIN ===
1/7 Checking commit d47d1b7f1366 (configure: do not include $(...) variables in 
config-host.mak)
2/7 Checking commit 9d9a4b192614 (configure: set $PYTHON to a full path)
3/7 Checking commit 090f670ef84c (configure: integrate Meson in the build 
system)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#104: 
new file mode 100644

WARNING: line over 80 characters
#283: FILE: scripts/ninjatool.py:160:
+raise LexerError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, 
msg))

WARNING: line over 80 characters
#286: FILE: scripts/ninjatool.py:163:
+raise ParseError("%s:%d: %s" % (self.stack[-1].filename, self.lineno, 
msg))

ERROR: line over 90 characters
#668: FILE: scripts/ninjatool.py:545:
+self.scope = BuildScope(self, out, iout, rule, in_, iin, orderdep, 
self.rule_vars[rule])

WARNING: line over 80 characters
#703: FILE: scripts/ninjatool.py:580:
+ARGS = argparse.ArgumentParser(description='Rewrite input build.ninja to 
stdout.')

WARNING: line over 80 characters
#738: FILE: scripts/ninjatool.py:615:
+def begin_build(self, outputs, implicit_outputs, rule, inputs, implicit, 
order_only):

WARNING: line over 80 characters
#843: FILE: scripts/ninjatool.py:720:
+ARGS = argparse.ArgumentParser(description='Convert build.ninja to a 
Makefile.')

ERROR: line over 90 characters
#897: FILE: scripts/ninjatool.py:774:
+self.print(r'ninja-clean:: ninja-clean-start; $(if $V,,@)rm -f 
${ninja-depfiles}')

ERROR: line over 90 characters
#911: FILE: scripts/ninjatool.py:788:
+self.print(r"ninja-command = $(if $V,,@printf '%s\n' '$(subst 
','\'',${.var.description})' && )${.var.command}")

ERROR: line over 90 characters
#937: FILE: scripts/ninjatool.py:814:
+self.print('ninja-targets-%s := %s' % (rule, ' 
'.join(self.rule_targets[rule])))

ERROR: line over 90 characters
#939: FILE: scripts/ninjatool.py:816:
+self.print('$(ninja-targets-%s): .var.%s = %s' % (rule, name, 
self.mangle_vars_for_make(value)))

WARNING: line over 80 characters
#940: FILE: scripts/ninjatool.py:817:
+self.print('ninja-outputdirs += $(sort $(dir 
${ninja-targets-%s}))' % rule)

WARNING: line over 80 characters
#951: FILE: scripts/ninjatool.py:828:
+default_targets = sorted(self.all_outs - self.all_ins, 
key=natural_sort_key)

WARNING: line over 80 characters
#960: FILE: scripts/ninjatool.py:837:
+phony_targets = sorted(self.all_phony - self.all_ins, 
key=natural_sort_key)

WARNING: line over 80 characters
#969: FILE: scripts/ninjatool.py:846:
+# These have to be printed last, because they override rule 
variables

WARNING: line over 80 characters
#970: FILE: scripts/ninjatool.py:847:
+# but we cannot print rule variables until we know the list of 
targets

WARNING: line over 80 characters
#972: FILE: scripts/ninjatool.py:849:
+self.build_vars[self.current_targets][name] = 
self.scope.build_vars[name]

WARNING: line over 80 characters
#991: FILE: scripts/ninjatool.py:868:
+ self.scope.in_ + self.scope.iin + 
self.scope.orderdep)

WARNING: line over 80 characters
#998: FILE: scripts/ninjatool.py:875:
+self.print('%s: %s | %s; ${ninja-command}' % (targets, inputs, 
orderonly))

ERROR: line over 90 characters
#1024: FILE: scripts/ninjatool.py:901:
+def __init__(self, option_strings, dest, choices, metavar='TOOL', 
nargs=None, **kwargs):

WARNING: line over 80 characters
#1057: FILE: scripts/ninjatool.py:934:
+parser = argparse.ArgumentParser(description='Process and transform 
build.ninja files.',

total: 6 errors, 15 warnings, 1048 lines che

Re: [Qemu-devel] [PATCH v6 6/7] block/nbd-client: nbd reconnect

[Vladimir Sementsov-Ogievskiy]

07.06.2019 6:17, Eric Blake wrote:
>> +typedef struct NBDConnection {
>> +BlockDriverState *bs;
>> +SocketAddress *saddr;
>> +const char *export;
>> +QCryptoTLSCreds *tlscreds;
>> +const char *hostname;
>> +const char *x_dirty_bitmap;
>> +} NBDConnection;
> Can we put this type in a header, and use it instead of passing lots of
> individual parameters to nbd_client_connect()?  Probably as a separate
> pre-requisite cleanup patch.
> 


Hmm, and then, include it into BDRVNBDState? I don't remember why didn't do
it, and now I don't see any reason for it. We store this information anyway
for the whole life of the driver..

So, if I'm going to refactor it, I have a question: is there a reason for
layered BDRVNBDState:

typedef struct BDRVNBDState {
 NBDClientSession client;

 /* For nbd_refresh_filename() */
 SocketAddress *saddr;
 char *export, *tlscredsid;
} BDRVNBDState;

and use nbd_get_client_session everywhere instead of simple converting void 
*opaque
to State pointer like in qcow2 for example?

The only reason I can imagine is not to share the whole BDRVNBDState, and keep
things we are using only in nbd.c to be available only in nbd.c.. But I've put
saddr and export to NBDConnection to be used in nbd-client.c anyway. So, I think
it's a good moment to flatten and share BDRVNBDState and drop 
nbd_get_client_session.

-- 
Best regards,
Vladimir

Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonz...@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

Configure options:
--enable-werror --target-list=x86_64-softmmu --prefix=/tmp/qemu-test/install 
--python=/usr/bin/python3 --enable-debug --enable-sanitizers --cxx=clang++ 
--cc=clang --host-cc=clang

ERROR: Meson not found. Use --meson=/path/to/meson

# QEMU configure log Mon Jun 10 12:40:26 UTC 2019
# Configured with: '/tmp/qemu-test/src/configure' '--enable-werror' 
'--target-list=x86_64-softmmu' '--prefix=/tmp/qemu-test/install' 
'--python=/usr/bin/python3' '--enable-debug' '--enable-sanitizers' 
'--cxx=clang++' '--cc=clang' '--host-cc=clang'
---
funcs: do_compiler do_cc compile_object check_define main
lines: 92 122 622 691 0
clang -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
-Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings 
-Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -c -o 
config-temp/qemu-conf.o config-temp/qemu-conf.c
config-temp/qemu-conf.c:2:2: error: __i386__ not defined
#error __i386__ not defined
 ^
1 error generated.
---
funcs: do_compiler do_cc compile_object check_define main
lines: 92 122 622 694 0
clang -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
-Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings 
-Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -c -o 
config-temp/qemu-conf.o config-temp/qemu-conf.c
config-temp/qemu-conf.c:2:2: error: __ILP32__ not defined
#error __ILP32__ not defined
 ^
1 error generated.


The full log is available at
http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonz...@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH v5] qemu-io: add pattern file for write command

[Denis Plotnikov]

On 31.05.2019 18:13, Eric Blake wrote:
> On 5/31/19 2:46 AM, Denis Plotnikov wrote:
>> The patch allows to provide a pattern file for write
>> command. There was no similar ability before.
>>
>> Signed-off-by: Denis Plotnikov 
>> ---
>> v5:
>>* file name initiated with null to make compilers happy
>>
> 
>> +static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
>> + char *file_name)
>> +{
>> +char *buf, *buf_pos;
>> +FILE *f = fopen(file_name, "r");
>> +int l;
>> +
>> +if (!f) {
>> +printf("'%s': %s\n", file_name, strerror(errno));
>> +return NULL;
>> +}
>> +
>> +if (qemuio_misalign) {
>> +len += MISALIGN_OFFSET;
>> +}
>> +buf = blk_blockalign(blk, len);
>> +memset(buf, 0, len);
>> +
>> +buf_pos = buf;
>> +
>> +while (len > 0) {
>> +l = fread(buf_pos, sizeof(char), len, f);
>> +
>> +if (feof(f)) {
>> +rewind(f);
>> +}
> 
> Why are we reading the file more than once? Once we've read it once,
> it's more efficient to switch to a loop that memcpy()s the prefix into
> the rest of the buffer, rather than to perform repeated I/O.
> 
Yes, it is. Will change it.

Denis

Re: [Qemu-devel] [PULL 0/8] Python queue, 2019-06-07

[Peter Maydell]

On Fri, 7 Jun 2019 at 22:16, Eduardo Habkost  wrote:
>
> The following changes since commit 185b7ccc11354cbd69b6d53bf8d831dd964f6c88:
>
>   Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20190607-2' into 
> staging (2019-06-07 15:24:13 +0100)
>
> are available in the Git repository at:
>
>   git://github.com/ehabkost/qemu.git tags/python-next-pull-request
>
> for you to fetch changes up to 8297719709f1339506d6da93ec69e6118ace8676:
>
>   travis: Make check-acceptance job more verbose (2019-06-07 18:11:02 -0300)
>
> 
> Python queue, 2019-06-07
>
> * New boot_linux_console test cases (Philippe Mathieu-Daudé)
> * Deprecate Python 2 support (Eduardo Habkost)
> * Require python3 >= 3.5 (Eduardo Habkost)
> * Make check-acceptance Travis job more verbose (Eduardo Habkost)
>
> 

Hi. This fails to build on one of my buildtest machines:

ERROR: Cannot use 'python3', Python 2 >= 2.7 or Python 3 >= 3.5 is required.
   Use --python=/path/to/python to specify a supported Python.

The machine has python 2.7.6 and 3.4.3. (It's an Ubuntu trusty
box; it's one of the gcc compile farm machines so upgrades to its
OS are not really under my control.)

The configure check also spits out deprecation warnings for
the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice
to get those updated.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v18 04/29] !fixup target/rx: CPU definition

[Igor Mammedov]

On Fri, 7 Jun 2019 20:06:12 +0200
Philippe Mathieu-Daudé  wrote:

> On 6/7/19 8:02 PM, Eric Blake wrote:
> > On 6/7/19 10:37 AM, Philippe Mathieu-Daudé wrote:  
> >> Rename macros.  
> > 
> > Why is this marked '!fixup' in the subject instead of just merging the
> > patches directly?  
> 
> Since all the series is reviewed/tested and Igor asked to amend a fixup
> but we are having hard time to figure if we understand Igor request
> correctly, I thought it would be easier for him to review this way, then
> for Richard to squash the patches and send the pull request.
I've explicitly asked for merging if I'm not mistaken.

 
> Is it confuse/bad practice?
it's confusing at least for me, since I have to review broken patch and fix ups
on top doing merge in my head and still won't be sure if I've missed
something in process on not.

(Fix ups are fine for trivial change that affect only one patch,
which isn't case here)

Please respin.

> 
> In the cover I wrote:
> 
>   If Igor aggrees with the fixup patches, Richard, can you squash
>   them and send a pull request?  (without the last patch, which is
>   expected to go via Eduardo's tree, but since it is helpful for
>   testing this series, I included it).
> 
> >>
> >> Signed-off-by: Richard Henderson 
> >> Signed-off-by: Philippe Mathieu-Daudé 
> >> ---
> >>  target/rx/cpu.c | 22 ++
> >>  target/rx/cpu.h | 12 ++--
> >>  2 files changed, 20 insertions(+), 14 deletions(-)
> >>  

[Qemu-devel] [PATCH v5 1/3] machine: show if CLI option '-numa node, mem' is supported in QAPI schema

[Igor Mammedov]

Legacy '-numa node,mem' option has a number of issues and mgmt often
defaults to it. Unfortunately it's no possible to replace it with
an alternative '-numa memdev' without breaking migration compatibility.
What's possible though is to deprecate it, keeping option working with
old machine types only.

In order to help users to find out if being deprecated CLI option
'-numa node,mem' is still supported by particular machine type, add new
"numa-mem-supported" property to output of query-machines.

"numa-mem-supported" is set to 'true' for machines that currently support
NUMA, but it will be flipped to 'false' later on, once deprecation period
expires and kept 'true' only for old machine types that used to support
the legacy option so it won't break existing configuration that are using
it.

Signed-off-by: Igor Mammedov 
---
  v5:
 (Markus Armbruster )
* s/by machine type/by the machine type/
* ammend commit message
   s/to MachineInfo description in QAPI schema/to output of 
query-machines/
  v4:
  * drop idea to use "qom-list-properties" and use MachineInfo instead
which could be inspected with 'query-machines'

 include/hw/boards.h | 3 +++
 hw/arm/virt.c   | 1 +
 hw/i386/pc.c| 1 +
 hw/ppc/spapr.c  | 1 +
 qapi/misc.json  | 5 -
 vl.c| 1 +
 6 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6ff02bf..ab6badc 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -158,6 +158,8 @@ typedef struct {
  * @kvm_type:
  *Return the type of KVM corresponding to the kvm-type string option or
  *computed based on other criteria such as the host kernel capabilities.
+ * @numa_mem_supported:
+ *true if '--numa node.mem' option is supported and false otherwise
  */
 struct MachineClass {
 /*< private >*/
@@ -210,6 +212,7 @@ struct MachineClass {
 bool ignore_boot_device_suffixes;
 bool smbus_no_migration_support;
 bool nvdimm_supported;
+bool numa_mem_supported;
 
 HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
DeviceState *dev);
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index bf54f10..481a603 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1943,6 +1943,7 @@ static void virt_machine_class_init(ObjectClass *oc, void 
*data)
 assert(!mc->get_hotplug_handler);
 mc->get_hotplug_handler = virt_machine_get_hotplug_handler;
 hc->plug = virt_machine_device_plug_cb;
+mc->numa_mem_supported = true;
 }
 
 static void virt_instance_init(Object *obj)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index edc240b..25146d7 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2750,6 +2750,7 @@ static void pc_machine_class_init(ObjectClass *oc, void 
*data)
 nc->nmi_monitor_handler = x86_nmi;
 mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
 mc->nvdimm_supported = true;
+mc->numa_mem_supported = true;
 
 object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int",
 pc_machine_get_device_memory_region_size, NULL,
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index e2b33e5..89d5814 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -4340,6 +4340,7 @@ static void spapr_machine_class_init(ObjectClass *oc, 
void *data)
  * in which LMBs are represented and hot-added
  */
 mc->numa_mem_align_shift = 28;
+mc->numa_mem_supported = true;
 
 smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF;
 smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;
diff --git a/qapi/misc.json b/qapi/misc.json
index 8b3ca4f..2dbfdf0 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -2018,12 +2018,15 @@
 #
 # @hotpluggable-cpus: cpu hotplug via -device is supported (since 2.7.0)
 #
+# @numa-mem-supported: true if '-numa node,mem' option is supported by
+#  the machine type and false otherwise (since 4.1)
+#
 # Since: 1.2.0
 ##
 { 'struct': 'MachineInfo',
   'data': { 'name': 'str', '*alias': 'str',
 '*is-default': 'bool', 'cpu-max': 'int',
-'hotpluggable-cpus': 'bool'} }
+'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool'} }
 
 ##
 # @query-machines:
diff --git a/vl.c b/vl.c
index cd1fbc4..f5b083f 100644
--- a/vl.c
+++ b/vl.c
@@ -1428,6 +1428,7 @@ MachineInfoList *qmp_query_machines(Error **errp)
 info->name = g_strdup(mc->name);
 info->cpu_max = !mc->max_cpus ? 1 : mc->max_cpus;
 info->hotpluggable_cpus = mc->has_hotpluggable_cpus;
+info->numa_mem_supported = mc->numa_mem_supported;
 
 entry = g_malloc0(sizeof(*entry));
 entry->value = info;
-- 
2.7.4

Re: [Qemu-devel] [PULL 0/8] Python queue, 2019-06-07

[Eduardo Habkost]

On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote:
> On Fri, 7 Jun 2019 at 22:16, Eduardo Habkost  wrote:
> >
> > The following changes since commit 185b7ccc11354cbd69b6d53bf8d831dd964f6c88:
> >
> >   Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20190607-2' into 
> > staging (2019-06-07 15:24:13 +0100)
> >
> > are available in the Git repository at:
> >
> >   git://github.com/ehabkost/qemu.git tags/python-next-pull-request
> >
> > for you to fetch changes up to 8297719709f1339506d6da93ec69e6118ace8676:
> >
> >   travis: Make check-acceptance job more verbose (2019-06-07 18:11:02 -0300)
> >
> > 
> > Python queue, 2019-06-07
> >
> > * New boot_linux_console test cases (Philippe Mathieu-Daudé)
> > * Deprecate Python 2 support (Eduardo Habkost)
> > * Require python3 >= 3.5 (Eduardo Habkost)
> > * Make check-acceptance Travis job more verbose (Eduardo Habkost)
> >
> > 
> 
> Hi. This fails to build on one of my buildtest machines:
> 
> ERROR: Cannot use 'python3', Python 2 >= 2.7 or Python 3 >= 3.5 is required.
>Use --python=/path/to/python to specify a supported Python.
> 
> The machine has python 2.7.6 and 3.4.3. (It's an Ubuntu trusty
> box; it's one of the gcc compile farm machines so upgrades to its
> OS are not really under my control.)

Ubuntu 16.04 LTS (Xenial) was released in April 2016.  Doesn't
that mean Trusty is not a supported build platform since April
2018?

> 
> The configure check also spits out deprecation warnings for
> the NetBSD/FreeBSD/OpenBSD tests/vm configurations. It would be nice
> to get those updated.

I'll take a look.

-- 
Eduardo

Re: [Qemu-devel] [PULL 0/8] Python queue, 2019-06-07

[Peter Maydell]

On Mon, 10 Jun 2019 at 14:11, Eduardo Habkost  wrote:
>
> On Mon, Jun 10, 2019 at 01:58:50PM +0100, Peter Maydell wrote:
> > Hi. This fails to build on one of my buildtest machines:
> >
> > ERROR: Cannot use 'python3', Python 2 >= 2.7 or Python 3 >= 3.5 is required.
> >Use --python=/path/to/python to specify a supported Python.
> >
> > The machine has python 2.7.6 and 3.4.3. (It's an Ubuntu trusty
> > box; it's one of the gcc compile farm machines so upgrades to its
> > OS are not really under my control.)
>
> Ubuntu 16.04 LTS (Xenial) was released in April 2016.  Doesn't
> that mean Trusty is not a supported build platform since April
> 2018?

Possibly, but as I say the gcc compile farm is what it is.
Regardless of the strict letter of the deprecation policy,
when we start running into issues with the set of build test
machines I tend to feel we may be being a bit over-hasty in
deprecating things.

PS: re the BSD VMs, you probably want to start with Alex's
recent update which did a fairly major overhaul there (he
sent a pullreq but it didn't pass build tests so isn't merged yet).

thanks
-- PMM

Re: [Qemu-devel] [RFC PATCH 0/7] Proof of concept for Meson integration

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/1560165301-39026-1-git-send-email-pbonz...@redhat.com/



Hi,

This series failed build test on s390x host. Please find the details below.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with
# HEAD pointing to a commit that has the patches applied on top of "base"
# branch
set -e
CC=$HOME/bin/cc
INSTALL=$PWD/install
BUILD=$PWD/build
mkdir -p $BUILD $INSTALL
SRC=$PWD
cd $BUILD
$SRC/configure --cc=$CC --prefix=$INSTALL
make -j4
# XXX: we need reliable clean up
# make check -j4 V=1
make install

echo
echo "=== ENV ==="
env

echo
echo "=== PACKAGES ==="
rpm -qa
=== TEST SCRIPT END ===

=== OUTPUT BEGIN ===

ERROR: Meson not found. Use --meson=/path/to/meson

=== OUTPUT END ===


The full log is available at
http://patchew.org/logs/1560165301-39026-1-git-send-email-pbonz...@redhat.com/testing.s390x/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[Qemu-devel] [PATCH v6] qemu-io: add pattern file for write command

[Denis Plotnikov]

The patch allows to provide a pattern file for write
command. There was no similar ability before.

Signed-off-by: Denis Plotnikov 
---
v6:
  * the pattern file is read once to reduce io

v5:
  * file name initiated with null to make compilers happy

v4:
  * missing signed-off clause added

v3:
  * missing file closing added
  * exclusive flags processing changed
  * buffer void* converted to char* to fix pointer arithmetics
  * file reading error processing added
---
 qemu-io-cmds.c | 88 ++
 1 file changed, 82 insertions(+), 6 deletions(-)

diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index 09750a23ce..e27203f747 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -343,6 +343,69 @@ static void *qemu_io_alloc(BlockBackend *blk, size_t len, 
int pattern)
 return buf;
 }
 
+static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
+ char *file_name)
+{
+char *buf, *buf_origin;
+FILE *f = fopen(file_name, "r");
+int l;
+
+if (!f) {
+printf("'%s': %s\n", file_name, strerror(errno));
+return NULL;
+}
+
+if (qemuio_misalign) {
+len += MISALIGN_OFFSET;
+}
+buf_origin = blk_blockalign(blk, len);
+memset(buf_origin, 0, len);
+
+buf = buf_origin;
+
+l = fread(buf, sizeof(char), len, f);
+
+if (ferror(f)) {
+printf("'%s': %s\n", file_name, strerror(errno));
+goto error;
+}
+
+if (l == 0) {
+printf("'%s' is empty\n", file_name);
+goto error;
+}
+
+if (l < len) {
+char *file_buf = g_malloc(sizeof(char) * l);
+memcpy(file_buf, buf, l);
+len -= l;
+buf += l;
+
+while (len > 0) {
+size_t len_to_copy = len > l ? l : len;
+
+memcpy(buf, file_buf, len_to_copy);
+
+len -= len_to_copy;
+buf += len_to_copy;
+}
+qemu_vfree(file_buf);
+}
+
+if (qemuio_misalign) {
+buf_origin += MISALIGN_OFFSET;
+}
+
+goto out;
+
+error:
+qemu_vfree(buf);
+buf_origin = NULL;
+out:
+fclose(f);
+return buf_origin;
+}
+
 static void qemu_io_free(void *p)
 {
 if (qemuio_misalign) {
@@ -965,7 +1028,7 @@ static const cmdinfo_t write_cmd = {
 .perm   = BLK_PERM_WRITE,
 .argmin = 2,
 .argmax = -1,
-.args   = "[-bcCfnquz] [-P pattern] off len",
+.args   = "[-bcCfnquz] [-P pattern | -s source_file] off len",
 .oneline= "writes a number of bytes at a specified offset",
 .help   = write_help,
 };
@@ -974,7 +1037,7 @@ static int write_f(BlockBackend *blk, int argc, char 
**argv)
 {
 struct timeval t1, t2;
 bool Cflag = false, qflag = false, bflag = false;
-bool Pflag = false, zflag = false, cflag = false;
+bool Pflag = false, zflag = false, cflag = false, sflag = false;
 int flags = 0;
 int c, cnt, ret;
 char *buf = NULL;
@@ -983,8 +1046,9 @@ static int write_f(BlockBackend *blk, int argc, char 
**argv)
 /* Some compilers get confused and warn if this is not initialized.  */
 int64_t total = 0;
 int pattern = 0xcd;
+char *file_name = NULL;
 
-while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) {
+while ((c = getopt(argc, argv, "bcCfnpP:quzs:")) != -1) {
 switch (c) {
 case 'b':
 bflag = true;
@@ -1020,6 +1084,10 @@ static int write_f(BlockBackend *blk, int argc, char 
**argv)
 case 'z':
 zflag = true;
 break;
+case 's':
+sflag = true;
+file_name = g_strdup(optarg);
+break;
 default:
 qemuio_command_usage(&write_cmd);
 return -EINVAL;
@@ -1051,8 +1119,9 @@ static int write_f(BlockBackend *blk, int argc, char 
**argv)
 return -EINVAL;
 }
 
-if (zflag && Pflag) {
-printf("-z and -P cannot be specified at the same time\n");
+if ((int)zflag + (int)Pflag + (int)sflag > 1) {
+printf("Only one of -z, -P, and -s"
+   "can be specified at the same time\n");
 return -EINVAL;
 }
 
@@ -1088,7 +1157,14 @@ static int write_f(BlockBackend *blk, int argc, char 
**argv)
 }
 
 if (!zflag) {
-buf = qemu_io_alloc(blk, count, pattern);
+if (sflag) {
+buf = qemu_io_alloc_from_file(blk, count, file_name);
+if (!buf) {
+return -EINVAL;
+}
+} else {
+buf = qemu_io_alloc(blk, count, pattern);
+}
 }
 
 gettimeofday(&t1, NULL);
-- 
2.17.0

Re: [Qemu-devel] qgraph

[Andreas Färber]

Am 10.06.19 um 14:03 schrieb Paolo Bonzini:
> On 10/06/19 13:57, Andreas Färber wrote:
>> Your question doesn't make sense grammatically or conceptually. As Paolo
>> explained below, QOM is a pure object model, with object types/classes
>> and properties. Buses are just object instances attached as properties
>> and don't necessarily even need their own type of bus object (e.g, CPU).
>> An answer you don't like doesn't change by asking it to other people...
>> The information is all there, you just need to interpret it correctly.
>>
>> There is no technical discussion (no concrete proposal of yours) to
>> comment on here, and kindly refer to last week's change of maintainers.
>>
>> You would be better off just explaining what you really want to achieve.
> 
> Well, that was explained upthread---finding out what device can be
> plugged where.
> 
> Let's see what is in QOM right now:
> 
> $ qemu-kvm -qmp unix:foo.sock,server,nowait -device virtio-scsi-pci,id=vs
> $ ./qmp/qom-list -s ~/foo.sock /machine/peripheral/vs|less
> 
> There is a "virtio-bus" here, and iside it there is a scsi-bus.
> 
> $ ./qmp/qom-list -s ~/foo.sock /machine/peripheral/vs/virtio-bus/child[0]/
> vs.0/
> 
> I guess you could add to virtio-scsi-pci a class property for the bus,
> and then make "vs.0" an alias to that class property.  This would allow
> you find buses by enumerating the class properties.

That kind of goes against the spirit of QOM though and ignores simpler
mechanisms of hot-plugging, as mentioned.

The theoretical generic way would be to discover that any random object
beneath /machine has a null property of a link<> type that inherits from
device and is compatible with the device type to be plugged.

In practice we decided against Anthony's idea of adding tons of empty
slot properties for e.g. PCI buses (the number space is too large). We
did add QOM support for wildcard "foo[*]" array properties though. A
null slot[42] property of a certain link<> type then means a device can
be plugged there, and the property's setter then needs to take care of
hot-plugging and un-plugging.
This was unfortunately obscured by the legacy qdev model where we
magically add new child[0] properties (pointing to
/machine/peripheral(-anon) sub-nodes) to its bus parent during device
instantiation IIRC.

So if we want a new QMP operation, the most sense would probably make
where-can-I-attach-type(foo) returning a list of QOM paths, showing only
the first free slot per bus. That would allow a more efficient lookup
implementation inside QEMU than needing to check each slot[n] property
via qom-get after discovering it with qom-list.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)

Re: [Qemu-devel] [PATCH v6 6/7] block/nbd-client: nbd reconnect

[Vladimir Sementsov-Ogievskiy]

10.06.2019 15:38, Vladimir Sementsov-Ogievskiy wrote:
> 07.06.2019 6:17, Eric Blake wrote:
>>> +typedef struct NBDConnection {
>>> +    BlockDriverState *bs;
>>> +    SocketAddress *saddr;
>>> +    const char *export;
>>> +    QCryptoTLSCreds *tlscreds;
>>> +    const char *hostname;
>>> +    const char *x_dirty_bitmap;
>>> +} NBDConnection;
>> Can we put this type in a header, and use it instead of passing lots of
>> individual parameters to nbd_client_connect()?  Probably as a separate
>> pre-requisite cleanup patch.
>>
> 
> 
> Hmm, and then, include it into BDRVNBDState? I don't remember why didn't do
> it, and now I don't see any reason for it. We store this information anyway
> for the whole life of the driver..
> 
> So, if I'm going to refactor it, I have a question: is there a reason for
> layered BDRVNBDState:
> 
> typedef struct BDRVNBDState {
>      NBDClientSession client;
> 
>      /* For nbd_refresh_filename() */
>      SocketAddress *saddr;
>      char *export, *tlscredsid;
> } BDRVNBDState;
> 
> and use nbd_get_client_session everywhere instead of simple converting void 
> *opaque
> to State pointer like in qcow2 for example?
> 
> The only reason I can imagine is not to share the whole BDRVNBDState, and keep
> things we are using only in nbd.c to be available only in nbd.c.. But I've put
> saddr and export to NBDConnection to be used in nbd-client.c anyway. So, I 
> think
> it's a good moment to flatten and share BDRVNBDState and drop 
> nbd_get_client_session.
> 

And if we are here, what is exact purpose of splitting  nbd-client.c from nbd.c?

I see, it was long ago:

commit 2302c1cafb13df23938b098d9c6595de52ec2577
Author: Marc-André Lureau 
Date:   Sun Dec 1 22:23:41 2013 +0100

 Split nbd block client code


But still, it doesn't explain..

And all this leads to noop wrappers like this

static int nbd_co_flush(BlockDriverState *bs)
{
 return nbd_client_co_flush(bs);
}

static void nbd_detach_aio_context(BlockDriverState *bs)
{
 nbd_client_detach_aio_context(bs);
}

static void nbd_attach_aio_context(BlockDriverState *bs,
AioContext *new_context)
{
 nbd_client_attach_aio_context(bs, new_context);
}


or need of defining driver callbacks in header file, instead of keeping them 
together
with driver struct definition and static (like other block drivers)...


And names of these two files always confused me.. nbd.c is nbd client block 
driver, and
driver is defined here.. But we have nbd-client.c which defines nbd client 
driver too.
And we also have nbd/client.c (OK, this split I understand of course, but it 
increases
confusion anyway).

-- 
Best regards,
Vladimir

[Qemu-devel] [PATCH] file-posix: unlock qemu_global_mutex before pread when attach disk

[Zhengui li]

virtual machine cpu soft lockup when attach a disk to the vm in the case
that backend storage network has a large delay or IO pressure is too large.

The bt of qemu main thread:
#0 0x9d78402c in pread64 () from /lib64/libpthread.so.0
#1 0xce3357d8 in pread64 (__offset=0, __nbytes=4096, 
__buf=0xd47a5200, __fd=202) at /usr/include/bits/unistd.h:99
#2 raw_is_io_aligned (fd=fd@entry=202, buf=buf@entry=0xd47a5200, 
len=len@entry=4096) at block/raw_posix.c:294
#3 0xce33597c in raw_probe_alignment (bs=bs@entry=0xd32ea920, 
fd=202, errp=errp@entry=0xfef7a330) at block/raw_posix.c:349
#4 0xce335a48 in raw_refresh_limits (bs=0xd32ea920, 
errp=0xfef7a330) at block/raw_posix.c:811
#5 0xce3404b0 in bdrv_refresh_limits (bs=0xd32ea920, 
errp=0xfef7a330, errp@entry=0xfef7a360) at block/io.c:122
#6 0xce340504 in bdrv_refresh_limits (bs=bs@entry=0xd09ce800, 
errp=errp@entry=0xfef7a3b0) at block/io.c:97
#7 0xce2eb9f0 in bdrv_open_common (bs=bs@entry=0xd09ce800, 
file=file@entry=0xd0e89800, options=, 
errp=errp@entry=0xfef7a450)
at block.c:1194
#8 0xce2eedec in bdrv_open_inherit (filename=, 
filename@entry=0xd25f92d0 "/dev/mapper/36384c4f100630193359db7a8011d",
reference=reference@entry=0x0, options=, 
options@entry=0xd3d0f4b0, flags=, flags@entry=128, 
parent=parent@entry=0x0,
child_role=child_role@entry=0x0, errp=errp@entry=0xfef7a710) at block.c:1895
#9 0xce2ef510 in bdrv_open (filename=filename@entry=0xd25f92d0 
"/dev/mapper/36384c4f100630193359db7a8011d", reference=reference@entry=0x0,
options=options@entry=0xd3d0f4b0, flags=flags@entry=128, 
errp=errp@entry=0xfef7a710) at block.c:1979
#10 0xce331ef0 in blk_new_open (filename=filename@entry=0xd25f92d0 
"/dev/mapper/36384c4f100630193359db7a8011d", reference=reference@entry=0x0,
options=options@entry=0xd3d0f4b0, flags=128, 
errp=errp@entry=0xfef7a710) at block/block_backend.c:213
#11 0xce0da1f4 in blockdev_init (file=file@entry=0xd25f92d0 
"/dev/mapper/36384c4f100630193359db7a8011d", 
bs_opts=bs_opts@entry=0xd3d0f4b0,
errp=errp@entry=0xfef7a710) at blockdev.c:603
#12 0xce0dc478 in drive_new (all_opts=all_opts@entry=0xd4dc31d0, 
block_default_type=) at blockdev.c:1116
#13 0xce0e3ee0 in add_init_drive (
optstr=optstr@entry=0xd0872ec0 
"file=/dev/mapper/36384c4f100630193359db7a8011d,format=raw,if=none,id=drive-scsi0-0-0-3,cache=none,aio=native")
at device_hotplug.c:46
#14 0xce0e3f78 in hmp_drive_add (mon=0xfef7a810, 
qdict=0xd0c8f000) at device_hotplug.c:67
#15 0xcdf7d688 in handle_hmp_command (mon=0xfef7a810, 
cmdline=) at /usr/src/debug/qemu-kvm-2.8.1/monitor.c:3199
#16 0xcdf7d778 in qmp_human_monitor_command (
command_line=0xcfc8e3c0 "drive_add dummy 
file=/dev/mapper/36384c4f100630193359db7a8011d,format=raw,if=none,id=drive-scsi0-0-0-3,cache=none,aio=native",
has_cpu_index=false, cpu_index=0, errp=errp@entry=0xfef7a968) at 
/usr/src/debug/qemu-kvm-2.8.1/monitor.c:660
#17 0xce0fdb30 in qmp_marshal_human_monitor_command (args=, ret=0xfef7a9e0, errp=0xfef7a9d8) at qmp-marshal.c:2223
#18 0xce3b6ad0 in do_qmp_dispatch (request=, 
errp=0xfef7aa20, errp@entry=0xfef7aa40) at qapi/qmp_dispatch.c:115
#19 0xce3b6d58 in qmp_dispatch (request=) at 
qapi/qmp_dispatch.c:142
#20 0xcdf79398 in handle_qmp_command (parser=, 
tokens=) at /usr/src/debug/qemu-kvm-2.8.1/monitor.c:4010
#21 0xce3bd6c0 in json_message_process_token (lexer=0xcf834c80, 
input=, type=JSON_RCURLY, x=214, y=274) at 
qobject/json_streamer.c:105
#22 0xce3f3d4c in json_lexer_feed_char 
(lexer=lexer@entry=0xcf834c80, ch=, flush=flush@entry=false) 
at qobject/json_lexer.c:319
#23 0xce3f3e6c in json_lexer_feed (lexer=0xcf834c80, 
buffer=, size=) at qobject/json_lexer.c:369
#24 0xcdf77c64 in monitor_qmp_read (opaque=, 
buf=, size=) at 
/usr/src/debug/qemu-kvm-2.8.1/monitor.c:4040
#25 0xce0eab18 in tcp_chr_read (chan=, cond=, opaque=0xcf90b280) at qemu_char.c:3260
#26 0x9dadf200 in g_main_context_dispatch () from 
/lib64/libglib-2.0.so.0
#27 0xce3c4a00 in glib_pollfds_poll () at util/main_loop.c:230
--Type  for more, q to quit, c to continue without paging--
#28 0xce3c4a88 in os_host_main_loop_wait (timeout=) at 
util/main_loop.c:278
#29 0xce3c4bf0 in main_loop_wait (nonblocking=) at 
util/main_loop.c:534
#30 0xce0f5d08 in main_loop () at vl.c:2120
#31 0xcdf3a770 in main (argc=, argv=, 
envp=) at vl.c:5017

when do qmp sush as drive_add,  qemu main thread locks the qemu_global_mutex  
and do pread in raw_probe_alignmen. Pread
is a synchronous operation. If backend storage network has a large delay or IO 
pressure is too large,  the pread operation will not return for a long time, 
which
make vcpu t

Re: [Qemu-devel] [PATCH v4 07/11] hmat acpi: Build Memory Side Cache Information Structure(s) in ACPI HMAT

[Tao Xu]

On 6/7/2019 12:45 AM, Igor Mammedov wrote:

On Thu, 6 Jun 2019 11:00:33 +0800
Tao Xu  wrote:


...


But the kernel HMAT can read othe Memory Side Cache Information except
SMBIOS entries and the host HMAT tables also haven’t SMBIOS Handles it
also shows Number of SMBIOS handles (n) as 0. So I am wondering if it is
better to setting "SMBIOS handles (n)" as 0, remove TODO and comment the
reason why set it 0?


My understanding is that SMBIOS handles are used to associate side cache
descriptions with RAM pointed by SMBIOS handles, so that OS would be
able to figure out what RAM modules are cached by what cache.
Hence I suspect that side cache table is useless in the best case without
valid references to SMBIOS handles.
(I might be totally mistaken but the matter requires clarification before
we commit to it)



I am sorry for not providing a detailed description for Memory Side 
Cache use case. I will add more detailed description in next version of 
patch.


As the commit message and /Documentation/admin-guide/mm/numaperf.rst of 
Kernel HMAT(listed blow), Memory Side Cache Structure is used to provide 
the cache information about System memory for the software to use. Then 
the software can maximize the performance because it can choose the best 
node to use.


Memory Side Cache Information Structure and System Locality Latency and 
Bandwidth Information Structure can both provide more information than 
numa distance for software to see. So back to the SMBIOS, in spec, 
SMBIOS handles point to the memory side cache physical devices, but they 
are also information and not contribute to the performance of the 
described memory. The field "Proximity Domain for the Memory" can show 
the described memory.


I am wondering if this explanation is clear? Thank you.

"System memory may be constructed in a hierarchy of elements with 
various performance characteristics in order to provide large address 
space of slower performing memory cached by a smaller higher performing 
memory."


"An application does not need to know about caching attributes in order
to use the system. Software may optionally query the memory cache
attributes in order to maximize the performance out of such a setup.
If the system provides a way for the kernel to discover this 
information, for example with ACPI HMAT (Heterogeneous Memory Attribute 
Table), the kernel will append these attributes to the NUMA node memory 
target."


"Each cache level's directory provides its attributes. For example, the
following shows a single cache level and the attributes available for
software to query::

# tree sys/devices/system/node/node0/memory_side_cache/
/sys/devices/system/node/node0/memory_side_cache/
|-- index1
|   |-- indexing
|   |-- line_size
|   |-- size
|   `-- write_policy
"

Re: [Qemu-devel] [PATCH v2 1/5] target/i386: Add cpu die-level topology support for X86CPU

[Like Xu]

On 2019/6/6 11:32, Eduardo Habkost wrote:

On Tue, May 21, 2019 at 12:50:52AM +0800, Like Xu wrote:

The die-level as the first PC-specific cpu topology is added to the
leagcy cpu topology model which only covers sockets/cores/threads.

In the new model with die-level support, the total number of logical
processors (including offline) on board will be calculated as:

  #cpus = #sockets * #dies * #cores * #threads

and considering compatibility, the default value for #dies is 1.

A new set of die-related variables are added in smp context and the
CPUX86State.nr_dies is assigned in x86_cpu_initfn() from PCMachineState.

Signed-off-by: Like Xu 
---
  hw/i386/pc.c   | 3 +++
  include/hw/i386/pc.h   | 2 ++
  include/hw/i386/topology.h | 2 ++
  qapi/misc.json | 6 --
  target/i386/cpu.c  | 9 +
  target/i386/cpu.h  | 3 +++
  6 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 896c22e32e..83ab53c814 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2341,6 +2341,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
  
  topo.pkg_id = cpu->socket_id;

  topo.core_id = cpu->core_id;
+topo.die_id = cpu->die_id;
  topo.smt_id = cpu->thread_id;
  cpu->apic_id = apicid_from_topo_ids(smp_cores, smp_threads, &topo);
  }
@@ -2692,6 +2693,8 @@ static const CPUArchIdList 
*pc_possible_cpu_arch_ids(MachineState *ms)
   ms->smp.cores, ms->smp.threads, &topo);
  ms->possible_cpus->cpus[i].props.has_socket_id = true;
  ms->possible_cpus->cpus[i].props.socket_id = topo.pkg_id;
+ms->possible_cpus->cpus[i].props.has_die_id = true;
+ms->possible_cpus->cpus[i].props.die_id = topo.die_id;
  ms->possible_cpus->cpus[i].props.has_core_id = true;
  ms->possible_cpus->cpus[i].props.core_id = topo.core_id;
  ms->possible_cpus->cpus[i].props.has_thread_id = true;
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index ce3c22951e..b5faf2ede9 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -24,6 +24,7 @@
   * PCMachineState:
   * @acpi_dev: link to ACPI PM device that performs ACPI hotplug handling
   * @boot_cpus: number of present VCPUs
+ * @smp_dies: number of dies per one package
   */
  struct PCMachineState {
  /*< private >*/
@@ -59,6 +60,7 @@ struct PCMachineState {
  bool apic_xrupt_override;
  unsigned apic_id_limit;
  uint16_t boot_cpus;
+unsigned smp_dies;
  
  /* NUMA information: */

  uint64_t numa_nodes;
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 1ebaee0f76..7f80498eb3 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -47,6 +47,7 @@ typedef uint32_t apic_id_t;
  
  typedef struct X86CPUTopoInfo {

  unsigned pkg_id;
+unsigned die_id;


Isn't it better to add this field only on patch 4/5?


  unsigned core_id;
  unsigned smt_id;
  } X86CPUTopoInfo;
@@ -130,6 +131,7 @@ static inline void x86_topo_ids_from_apicid(apic_id_t 
apicid,
  topo->core_id = (apicid >> apicid_core_offset(nr_cores, nr_threads)) &
 ~(0xUL << apicid_core_width(nr_cores, nr_threads));
  topo->pkg_id = apicid >> apicid_pkg_offset(nr_cores, nr_threads);
+topo->die_id = -1;


Why are you setting die_id = -1 here?


Hi Eduardo,thanks for your comments and support.

Would it be a better way to introduce all die related variables 
including has_die_id/nr_dies/cpu->die_id/topo.die_id/smp_dies in one 
patch for consistency check and backport convenient?


In this case the default value for topo->die_id would be 0 (for sure, 
one die per package) with has_die_id = false. Is that acceptable to you？




If die_id isn't valid yet, isn't it better to keep has_die_id =
false at pc_possible_cpu_arch_ids() above, and set has_die_id =
true only on patch 4/5?


  }
  
  /* Make APIC ID for the CPU 'cpu_index'

diff --git a/qapi/misc.json b/qapi/misc.json
index 8b3ca4fdd3..cd236c89b3 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -2924,10 +2924,11 @@
  #
  # @node-id: NUMA node ID the CPU belongs to
  # @socket-id: socket number within node/board the CPU belongs to
-# @core-id: core number within socket the CPU belongs to
+# @die-id: die number within node/board the CPU belongs to (Since 4.1)
+# @core-id: core number within die the CPU belongs to
  # @thread-id: thread number within core the CPU belongs to
  #
-# Note: currently there are 4 properties that could be present
+# Note: currently there are 5 properties that could be present
  # but management should be prepared to pass through other
  # properties with device_add command to allow for future
  # interface extension. This also requires the filed names to be kept in
@@ -2938,6 +2939,7 @@
  { 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
  '*socket-id': 'int',
+'*die-id

[Qemu-devel] [PATCH] qemu-ga: Convert invocation documentation to rST

[Peter Maydell]

The qemu-ga documentation is currently in qemu-ga.texi in
Texinfo format, which we present to the user as:
 * a qemu-ga manpage
 * a section of the main qemu-doc HTML documentation

Convert the documentation to rST format, and present it to
the user as:
 * a qemu-ga manpage
 * part of the interop/ Sphinx manual

Signed-off-by: Peter Maydell 
---
This is part of my general proposals about how we should
transition away from texinfo to sphinx (written up at
https://wiki.qemu.org/Features/Documentation). It's the
first part of step 3 (convert standalone manpages), so it's
interesting as a demonstration of Sphinx generating manpages
as well as HTML. The format of the output manpage is broadly
the same, though there are a few minor differences because
rST doesn't support quite the same kinds of output. (It also
fixes a bug where the current manpage renders some text intended
to be in bold as literally "B".)

Having the infrastructure for creating a simple manpage via
Sphinx should be a useful assist for the work in step 1 of the
transition plan that involves conversion of the auto-generated
qemu-ga-ref and qemu-qmp-ref (which need to produce manpages).

The non-manpage HTML version of the doc moves from qemu-doc.html
into docs/interop/ (its final location in the new 5-manual setup).

 Makefile |  13 ++--
 MAINTAINERS  |   2 +-
 docs/conf.py |  18 ++---
 docs/interop/conf.py |   7 ++
 docs/interop/index.rst   |   1 +
 docs/interop/qemu-ga.rst | 133 +
 qemu-doc.texi|   5 --
 qemu-ga.texi | 137 ---
 8 files changed, 161 insertions(+), 155 deletions(-)
 create mode 100644 docs/interop/qemu-ga.rst
 delete mode 100644 qemu-ga.texi

diff --git a/Makefile b/Makefile
index 8e2fc6624c3..cdada210746 100644
--- a/Makefile
+++ b/Makefile
@@ -329,7 +329,7 @@ endif
 endif
 
 ifdef BUILD_DOCS
-DOCS=qemu-doc.html qemu-doc.txt qemu.1 qemu-img.1 qemu-nbd.8 qemu-ga.8
+DOCS=qemu-doc.html qemu-doc.txt qemu.1 qemu-img.1 qemu-nbd.8 
docs/interop/qemu-ga.8
 DOCS+=docs/interop/qemu-qmp-ref.html docs/interop/qemu-qmp-ref.txt 
docs/interop/qemu-qmp-ref.7
 DOCS+=docs/interop/qemu-ga-ref.html docs/interop/qemu-ga-ref.txt 
docs/interop/qemu-ga-ref.7
 DOCS+=docs/qemu-block-drivers.7
@@ -804,7 +804,7 @@ ifdef CONFIG_TRACE_SYSTEMTAP
$(INSTALL_DATA) scripts/qemu-trace-stap.1 "$(DESTDIR)$(mandir)/man1"
 endif
 ifneq (,$(findstring qemu-ga,$(TOOLS)))
-   $(INSTALL_DATA) qemu-ga.8 "$(DESTDIR)$(mandir)/man8"
+   $(INSTALL_DATA) docs/interop/qemu-ga.8 "$(DESTDIR)$(mandir)/man8"
$(INSTALL_DATA) docs/interop/qemu-ga-ref.html "$(DESTDIR)$(qemu_docdir)"
$(INSTALL_DATA) docs/interop/qemu-ga-ref.txt "$(DESTDIR)$(qemu_docdir)"
$(INSTALL_DATA) docs/interop/qemu-ga-ref.7 "$(DESTDIR)$(mandir)/man7"
@@ -969,12 +969,18 @@ build-manual = $(call quiet-command,sphinx-build $(if 
$(V),,-q) -W -n -b html -D
 # We assume all RST files in the manual's directory are used in it
 manual-deps = $(wildcard $(SRC_PATH)/docs/$1/*.rst) 
$(SRC_PATH)/docs/$1/conf.py $(SRC_PATH)/docs/conf.py
 
+# Canned command to build manpages from a single manual
+build-manpages = $(call quiet-command,CONFDIR="$(qemu_confdir)" sphinx-build 
$(if $(V),,-q) -W -n -b man -D version=$(VERSION) -D release="$(FULL_VERSION)" 
-d .doctrees/$1 $(SRC_PATH)/docs/$1 $(MANUAL_BUILDDIR)/$1 
,"SPHINX","$(MANUAL_BUILDDIR)/$1")
+
 $(MANUAL_BUILDDIR)/devel/index.html: $(call manual-deps,devel)
$(call build-manual,devel)
 
 $(MANUAL_BUILDDIR)/interop/index.html: $(call manual-deps,interop)
$(call build-manual,interop)
 
+$(MANUAL_BUILDDIR)/interop/qemu-ga.8: $(call manual-deps,interop)
+   $(call build-manpages,interop)
+
 qemu-options.texi: $(SRC_PATH)/qemu-options.hx $(SRC_PATH)/scripts/hxtool
$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > 
$@,"GEN","$@")
 
@@ -998,7 +1004,6 @@ qemu.1: qemu-option-trace.texi
 qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi
 fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
 qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi
-qemu-ga.8: qemu-ga.texi
 docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi
 docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi
 scripts/qemu-trace-stap.1: scripts/qemu-trace-stap.texi
@@ -1010,7 +1015,7 @@ txt: qemu-doc.txt docs/interop/qemu-qmp-ref.txt 
docs/interop/qemu-ga-ref.txt
 
 qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
qemu-img.texi qemu-nbd.texi qemu-options.texi qemu-option-trace.texi \
-   qemu-deprecated.texi qemu-monitor.texi qemu-img-cmds.texi qemu-ga.texi \
+   qemu-deprecated.texi qemu-monitor.texi qemu-img-cmds.texi \
qemu-monitor-info.texi docs/qemu-block-drivers.texi \
docs/qemu-cpu-models.texi docs/security.texi
 
diff --git a/MAINTAINERS b/MAINTAINERS
index a96829ea83a..27a6ffa7431 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2024,7 +2024

Re: [Qemu-devel] [PULL 00/39] tcg: Move the softmmu tlb to CPUNegativeOffsetState

[Peter Maydell]

On Mon, 10 Jun 2019 at 03:02, Richard Henderson
 wrote:
>
> The following changes since commit 185b7ccc11354cbd69b6d53bf8d831dd964f6c88:
>
>   Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20190607-2' into 
> staging (2019-06-07 15:24:13 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/rth7680/qemu.git tags/pull-tcg-20190609
>
> for you to fetch changes up to e20774aed18cc5e0113e6a6c502ece2fc1c41931:
>
>   tcg/arm: Remove mostly unreachable tlb special case (2019-06-09 18:55:23 
> -0700)
>
> 
> Move softmmu tlb into CPUNegativeOffsetState
>
> 

Hi; this failed to build on OpenBSD:
/tmp/qemu-test.RzUFLe/bsd-user/main.c: In function 'cpu_loop':
/tmp/qemu-test.RzUFLe/bsd-user/main.c:143:28: error: 'cpu' undeclared
(first use in this function)
 CPUState *cs = env_cpu(cpu);
^
/tmp/qemu-test.RzUFLe/bsd-user/main.c:143:28: note: each undeclared
identifier is reported only once for each function it appears in

(freebsd and netbsd were ok)

thanks
-- PMM

[Qemu-devel] [PATCH v5 02/12] qapi/block-core: add option for io_uring

[Aarushi Mehta]

Option only enumerates for hosts that support it.

Signed-off-by: Aarushi Mehta 
Reviewed-by: Stefan Hajnoczi 
---
 qapi/block-core.json | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 1defcde048..db7eedd058 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2792,11 +2792,13 @@
 #
 # @threads: Use qemu's thread pool
 # @native:  Use native AIO backend (only Linux and Windows)
+# @io_uring:Use linux io_uring (since 4.1)
 #
 # Since: 2.9
 ##
 { 'enum': 'BlockdevAioOptions',
-  'data': [ 'threads', 'native' ] }
+  'data': [ 'threads', 'native',
+{ 'name': 'io_uring', 'if': 'defined(CONFIG_LINUX_IO_URING)' } ] }
 
 ##
 # @BlockdevCacheOptions:
-- 
2.17.1

[Qemu-devel] [PATCH v5 00/12] Add support for io_uring

[Aarushi Mehta]

This patch series adds support for the newly developed io_uring Linux AIO
interface. Linux io_uring is faster than Linux's AIO asynchronous I/O code,
offers efficient buffered asynchronous I/O support, the ability to do I/O
without performing a system call via polled I/O, and other efficiency 
enhancements.

Testing it requires a host kernel (5.1+) and the liburing library.
Use the option -drive aio=io_uring to enable it.

v5:
- Adds completion polling
- Extends qemu-io
- Adds qemu-iotest

v4:
- Add error handling
- Add trace events
- Remove aio submission based code

v3:
- Fix major errors in io_uring (sorry)
- Option now enumerates for CONFIG_LINUX_IO_URING
- pkg config support added

Aarushi Mehta (12):
  configure: permit use of io_uring
  qapi/block-core: add option for io_uring Only enumerates option for
devices that support it
  block/block: add BDRV flag for io_uring
  block/io_uring: implements interfaces for io_uring Aborts when sqe
fails to be set as sqes cannot be returned to the ring.
  stubs: add stubs for io_uring interface
  util/async: add aio interfaces for io_uring
  blockdev: accept io_uring as option
  block/file-posix.c: extend to use io_uring
  block: add trace events for io_uring
  block/io_uring: adds userspace completion polling
  qemu-io: adds support for io_uring
  qemu-iotests/087: checks for io_uring

 MAINTAINERS|   8 +
 block/Makefile.objs|   3 +
 block/file-posix.c |  85 --
 block/io_uring.c   | 339 +
 block/trace-events |   8 +
 blockdev.c |   4 +-
 configure  |  27 +++
 include/block/aio.h|  16 +-
 include/block/block.h  |   1 +
 include/block/raw-aio.h|  12 ++
 qapi/block-core.json   |   4 +-
 qemu-io.c  |  13 ++
 stubs/Makefile.objs|   1 +
 stubs/io_uring.c   |  32 
 tests/qemu-iotests/087 |  26 +++
 tests/qemu-iotests/087.out |  10 ++
 util/async.c   |  36 
 17 files changed, 606 insertions(+), 19 deletions(-)
 create mode 100644 block/io_uring.c
 create mode 100644 stubs/io_uring.c

-- 
2.17.1

[Qemu-devel] [PATCH v5 01/12] configure: permit use of io_uring

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
Reviewed-by: Stefan Hajnoczi 
Reviewed-by: Maxim Levitsky 
---
 configure | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/configure b/configure
index b091b82cb3..7aa18d308d 100755
--- a/configure
+++ b/configure
@@ -365,6 +365,7 @@ xen=""
 xen_ctrl_version=""
 xen_pci_passthrough=""
 linux_aio=""
+linux_io_uring=""
 cap_ng=""
 attr=""
 libattr=""
@@ -1266,6 +1267,10 @@ for opt do
   ;;
   --enable-linux-aio) linux_aio="yes"
   ;;
+  --disable-linux-io-uring) linux_io_uring="no"
+  ;;
+  --enable-linux-io-uring) linux_io_uring="yes"
+  ;;
   --disable-attr) attr="no"
   ;;
   --enable-attr) attr="yes"
@@ -1784,6 +1789,7 @@ disabled with --disable-FEATURE, default is enabled if 
available:
   vde support for vde network
   netmap  support for netmap network
   linux-aio   Linux AIO support
+  linux-io-uring  Linux io_uring support
   cap-ng  libcap-ng support
   attrattr and xattr support
   vhost-net   vhost-net kernel acceleration support
@@ -3973,6 +3979,21 @@ EOF
 linux_aio=no
   fi
 fi
+##
+# linux-io-uring probe
+
+if test "$linux_io_uring" != "no" ; then
+  if $pkg_config liburing; then
+linux_io_uring_cflags=$($pkg_config --cflags liburing)
+linux_io_uring_libs=$($pkg_config --libs liburing)
+linux_io_uring=yes
+  else
+if test "$linux_io_uring" = "yes" ; then
+  feature_not_found "linux io_uring" "Install liburing devel"
+fi
+linux_io_uring=no
+  fi
+fi
 
 ##
 # TPM emulation is only on POSIX
@@ -6396,6 +6417,7 @@ echo "PIE   $pie"
 echo "vde support   $vde"
 echo "netmap support$netmap"
 echo "Linux AIO support $linux_aio"
+echo "Linux io_uring support $linux_io_uring"
 echo "ATTR/XATTR support $attr"
 echo "Install blobs $blobs"
 echo "KVM support   $kvm"
@@ -6874,6 +6896,11 @@ fi
 if test "$linux_aio" = "yes" ; then
   echo "CONFIG_LINUX_AIO=y" >> $config_host_mak
 fi
+if test "$linux_io_uring" = "yes" ; then
+  echo "CONFIG_LINUX_IO_URING=y" >> $config_host_mak
+  echo "LINUX_IO_URING_CFLAGS=$linux_io_uring_cflags" >> $config_host_mak
+  echo "LINUX_IO_URING_LIBS=$linux_io_uring_libs" >> $config_host_mak
+fi
 if test "$attr" = "yes" ; then
   echo "CONFIG_ATTR=y" >> $config_host_mak
 fi
-- 
2.17.1

[Qemu-devel] [PATCH] pc: fix possible NULL pointer dereference in pc_machine_get_device_memory_region_size()

[Igor Mammedov]

QEMU will crash when device-memory-region-size property is read if 
ms->device_memory
wasn't initialized yet.

Crash can be reproduced with:
 $QEMU -preconfig -qmp unix:qmp_socket,server,nowait &
 ./scripts/qmp/qom-get -s qmp_socket /machine.device-memory-region-size

Instead of crashing return 0 if ms->device_memory hasn't been initialized.

Signed-off-by: Igor Mammedov 
---
v2:
  add reproducer to commit message
   (Markus Armbruster )

 hw/i386/pc.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index edc240b..1b7ead9 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2459,7 +2459,11 @@ pc_machine_get_device_memory_region_size(Object *obj, 
Visitor *v,
  Error **errp)
 {
 MachineState *ms = MACHINE(obj);
-int64_t value = memory_region_size(&ms->device_memory->mr);
+int64_t value = 0;
+
+if (ms->device_memory) {
+memory_region_size(&ms->device_memory->mr);
+}
 
 visit_type_int(v, name, &value, errp);
 }
-- 
2.7.4

[Qemu-devel] [PATCH v5 05/12] stubs: add stubs for io_uring interface

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
Reviewed-by: Stefan Hajnoczi 
---
 MAINTAINERS |  1 +
 stubs/Makefile.objs |  1 +
 stubs/io_uring.c| 32 
 3 files changed, 34 insertions(+)
 create mode 100644 stubs/io_uring.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 49f896796e..bc38175124 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2522,6 +2522,7 @@ R: Stefan Hajnoczi 
 L: qemu-bl...@nongnu.org
 S: Maintained
 F: block/io_uring.c
+F: stubs/io_uring.c
 
 qcow2
 M: Kevin Wolf 
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index 9c7393b08c..5cf160a9c8 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -13,6 +13,7 @@ stub-obj-y += iothread.o
 stub-obj-y += iothread-lock.o
 stub-obj-y += is-daemonized.o
 stub-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
+stub-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o
 stub-obj-y += machine-init-done.o
 stub-obj-y += migr-blocker.o
 stub-obj-y += change-state-handler.o
diff --git a/stubs/io_uring.c b/stubs/io_uring.c
new file mode 100644
index 00..622d1e4648
--- /dev/null
+++ b/stubs/io_uring.c
@@ -0,0 +1,32 @@
+/*
+ * Linux io_uring support.
+ *
+ * Copyright (C) 2009 IBM, Corp.
+ * Copyright (C) 2009 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "block/aio.h"
+#include "block/raw-aio.h"
+
+void luring_detach_aio_context(LuringState *s, AioContext *old_context)
+{
+abort();
+}
+
+void luring_attach_aio_context(LuringState *s, AioContext *new_context)
+{
+abort();
+}
+
+LuringState *luring_init(Error **errp)
+{
+abort();
+}
+
+void luring_cleanup(LuringState *s)
+{
+abort();
+}
-- 
2.17.1

[Qemu-devel] [PATCH v5 04/12] block/io_uring: implements interfaces for io_uring

[Aarushi Mehta]

Aborts when sqe fails to be set as sqes cannot be returned to the ring.

Signed-off-by: Aarushi Mehta 
---
 MAINTAINERS |   7 +
 block/Makefile.objs |   3 +
 block/io_uring.c| 314 
 include/block/aio.h |  16 +-
 include/block/raw-aio.h |  12 ++
 5 files changed, 351 insertions(+), 1 deletion(-)
 create mode 100644 block/io_uring.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 7be1225415..49f896796e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2516,6 +2516,13 @@ F: block/file-posix.c
 F: block/file-win32.c
 F: block/win32-aio.c
 
+Linux io_uring
+M: Aarushi Mehta 
+R: Stefan Hajnoczi 
+L: qemu-bl...@nongnu.org
+S: Maintained
+F: block/io_uring.c
+
 qcow2
 M: Kevin Wolf 
 M: Max Reitz 
diff --git a/block/Makefile.objs b/block/Makefile.objs
index ae11605c9f..8fde7a23a5 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-y += block-backend.o snapshot.o qapi.o
 block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
 block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
+block-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o
 block-obj-y += null.o mirror.o commit.o io.o create.o
 block-obj-y += throttle-groups.o
 block-obj-$(CONFIG_LINUX) += nvme.o
@@ -61,5 +62,7 @@ block-obj-$(if $(CONFIG_LZFSE),m,n) += dmg-lzfse.o
 dmg-lzfse.o-libs   := $(LZFSE_LIBS)
 qcow.o-libs:= -lz
 linux-aio.o-libs   := -laio
+io_uring.o-cflags  := $(LINUX_IO_URING_CFLAGS)
+io_uring.o-libs:= $(LINUX_IO_URING_LIBS)
 parallels.o-cflags := $(LIBXML2_CFLAGS)
 parallels.o-libs   := $(LIBXML2_LIBS)
diff --git a/block/io_uring.c b/block/io_uring.c
new file mode 100644
index 00..f327c7ef96
--- /dev/null
+++ b/block/io_uring.c
@@ -0,0 +1,314 @@
+/*
+ * Linux io_uring support.
+ *
+ * Copyright (C) 2009 IBM, Corp.
+ * Copyright (C) 2009 Red Hat, Inc.
+ * Copyright (C) 2019 Aarushi Mehta
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include 
+#include "qemu-common.h"
+#include "block/aio.h"
+#include "qemu/queue.h"
+#include "block/block.h"
+#include "block/raw-aio.h"
+#include "qemu/coroutine.h"
+#include "qapi/error.h"
+
+#define MAX_EVENTS 128
+
+typedef struct LuringAIOCB {
+Coroutine *co;
+struct io_uring_sqe sqeq;
+ssize_t ret;
+QEMUIOVector *qiov;
+bool is_read;
+QSIMPLEQ_ENTRY(LuringAIOCB) next;
+} LuringAIOCB;
+
+typedef struct LuringQueue {
+int plugged;
+unsigned int in_queue;
+unsigned int in_flight;
+bool blocked;
+QSIMPLEQ_HEAD(, LuringAIOCB) sq_overflow;
+} LuringQueue;
+
+typedef struct LuringState {
+AioContext *aio_context;
+
+struct io_uring ring;
+
+/* io queue for submit at batch.  Protected by AioContext lock. */
+LuringQueue io_q;
+
+/* I/O completion processing.  Only runs in I/O thread.  */
+QEMUBH *completion_bh;
+} LuringState;
+
+/**
+ * ioq_submit:
+ * @s: AIO state
+ *
+ * Queues pending sqes and submits them
+ *
+ */
+static int ioq_submit(LuringState *s);
+
+/**
+ * qemu_luring_process_completions:
+ * @s: AIO state
+ *
+ * Fetches completed I/O requests, consumes cqes and invokes their callbacks.
+ *
+ */
+static void qemu_luring_process_completions(LuringState *s)
+{
+struct io_uring_cqe *cqes;
+int ret;
+
+/*
+ * Request completion callbacks can run the nested event loop.
+ * Schedule ourselves so the nested event loop will "see" remaining
+ * completed requests and process them.  Without this, completion
+ * callbacks that wait for other requests using a nested event loop
+ * would hang forever.
+ */
+qemu_bh_schedule(s->completion_bh);
+
+while (io_uring_peek_cqe(&s->ring, &cqes) == 0) {
+if (!cqes) {
+break;
+}
+LuringAIOCB *luringcb = io_uring_cqe_get_data(cqes);
+ret = cqes->res;
+
+if (ret == luringcb->qiov->size) {
+ret = 0;
+} else if (ret >= 0) {
+/* Short Read/Write */
+if (luringcb->is_read) {
+/* Read, pad with zeroes */
+qemu_iovec_memset(luringcb->qiov, ret, 0,
+luringcb->qiov->size - ret);
+} else {
+ret = -ENOSPC;;
+}
+}
+luringcb->ret = ret;
+
+io_uring_cqe_seen(&s->ring, cqes);
+cqes = NULL;
+/* Change counters one-by-one because we can be nested. */
+s->io_q.in_flight--;
+
+/*
+ * If the coroutine is already entered it must be in ioq_submit()
+ * and will notice luringcb->ret has been filled in when it
+ * eventually runs later. Coroutines cannot be entered recursively
+ * so avoid doing that!
+ */
+if (!qemu_coroutine_entered(luringcb->co)) {
+aio_co_wake(luringcb->co);
+}
+}
+qemu_bh_cancel(s->completion_bh

[Qemu-devel] [PATCH v5 03/12] block/block: add BDRV flag for io_uring

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
Reviewed-by: Stefan Hajnoczi 
Reviewed-by: Maxim Levitsky 
---
 include/block/block.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/block/block.h b/include/block/block.h
index f9415ed740..5e08df716f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -121,6 +121,7 @@ typedef struct HDGeometry {
   ignoring the format layer */
 #define BDRV_O_NO_IO   0x1 /* don't initialize for I/O */
 #define BDRV_O_AUTO_RDONLY 0x2 /* degrade to read-only if opening 
read-write fails */
+#define BDRV_O_IO_URING0x4 /* use io_uring instead of the thread pool 
*/
 
 #define BDRV_O_CACHE_MASK  (BDRV_O_NOCACHE | BDRV_O_NO_FLUSH)
 
-- 
2.17.1

[Qemu-devel] [PATCH v5 06/12] util/async: add aio interfaces for io_uring

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
Reviewed-by: Stefan Hajnoczi 
---
 util/async.c | 36 
 1 file changed, 36 insertions(+)

diff --git a/util/async.c b/util/async.c
index c10642a385..2709f0edc3 100644
--- a/util/async.c
+++ b/util/async.c
@@ -277,6 +277,14 @@ aio_ctx_finalize(GSource *source)
 }
 #endif
 
+#ifdef CONFIG_LINUX_IO_URING
+if (ctx->linux_io_uring) {
+luring_detach_aio_context(ctx->linux_io_uring, ctx);
+luring_cleanup(ctx->linux_io_uring);
+ctx->linux_io_uring = NULL;
+}
+#endif
+
 assert(QSLIST_EMPTY(&ctx->scheduled_coroutines));
 qemu_bh_delete(ctx->co_schedule_bh);
 
@@ -341,6 +349,29 @@ LinuxAioState *aio_get_linux_aio(AioContext *ctx)
 }
 #endif
 
+#ifdef CONFIG_LINUX_IO_URING
+LuringState *aio_setup_linux_io_uring(AioContext *ctx, Error **errp)
+{
+if (ctx->linux_io_uring) {
+return ctx->linux_io_uring;
+}
+
+ctx->linux_io_uring = luring_init(errp);
+if (!ctx->linux_io_uring) {
+return NULL;
+}
+
+luring_attach_aio_context(ctx->linux_io_uring, ctx);
+return ctx->linux_io_uring;
+}
+
+LuringState *aio_get_linux_io_uring(AioContext *ctx)
+{
+assert(ctx->linux_io_uring);
+return ctx->linux_io_uring;
+}
+#endif
+
 void aio_notify(AioContext *ctx)
 {
 /* Write e.g. bh->scheduled before reading ctx->notify_me.  Pairs
@@ -432,6 +463,11 @@ AioContext *aio_context_new(Error **errp)
 #ifdef CONFIG_LINUX_AIO
 ctx->linux_aio = NULL;
 #endif
+
+#ifdef CONFIG_LINUX_IO_URING
+ctx->linux_io_uring = NULL;
+#endif
+
 ctx->thread_pool = NULL;
 qemu_rec_mutex_init(&ctx->lock);
 timerlistgroup_init(&ctx->tlg, aio_timerlist_notify, ctx);
-- 
2.17.1

[Qemu-devel] [PATCH v5 09/12] block: add trace events for io_uring

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
---
 block/io_uring.c   | 14 --
 block/trace-events |  8 
 2 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/block/io_uring.c b/block/io_uring.c
index f327c7ef96..47e027364a 100644
--- a/block/io_uring.c
+++ b/block/io_uring.c
@@ -17,6 +17,7 @@
 #include "block/raw-aio.h"
 #include "qemu/coroutine.h"
 #include "qapi/error.h"
+#include "trace.h"
 
 #define MAX_EVENTS 128
 
@@ -191,12 +192,15 @@ static int ioq_submit(LuringState *s)
 
 void luring_io_plug(BlockDriverState *bs, LuringState *s)
 {
+trace_luring_io_plug();
 s->io_q.plugged++;
 }
 
 void luring_io_unplug(BlockDriverState *bs, LuringState *s)
 {
 assert(s->io_q.plugged);
+trace_luring_io_unplug(s->io_q.blocked, s->io_q.plugged,
+ s->io_q.in_queue, s->io_q.in_flight);
 if (--s->io_q.plugged == 0 &&
 !s->io_q.blocked && s->io_q.in_queue > 0) {
 ioq_submit(s);
@@ -217,6 +221,7 @@ void luring_io_unplug(BlockDriverState *bs, LuringState *s)
 static int luring_do_submit(int fd, LuringAIOCB *luringcb, LuringState *s,
 uint64_t offset, int type)
 {
+int ret;
 struct io_uring_sqe *sqes = io_uring_get_sqe(&s->ring);
 if (!sqes) {
 sqes = &luringcb->sqeq;
@@ -242,11 +247,14 @@ static int luring_do_submit(int fd, LuringAIOCB 
*luringcb, LuringState *s,
 }
 io_uring_sqe_set_data(sqes, luringcb);
 s->io_q.in_queue++;
-
+trace_luring_do_submit(s->io_q.blocked, s->io_q.plugged,
+   s->io_q.in_queue, s->io_q.in_flight);
 if (!s->io_q.blocked &&
 (!s->io_q.plugged ||
  s->io_q.in_flight + s->io_q.in_queue >= MAX_EVENTS)) {
-return ioq_submit(s);
+ret = ioq_submit(s);
+trace_luring_do_submit_done(ret);
+return ret;
 }
 return 0;
 }
@@ -294,6 +302,7 @@ LuringState *luring_init(Error **errp)
 int rc;
 LuringState *s;
 s = g_malloc0(sizeof(*s));
+trace_luring_init_state((void *)s, sizeof(*s));
 struct io_uring *ring = &s->ring;
 rc =  io_uring_queue_init(MAX_EVENTS, ring, 0);
 if (rc < 0) {
@@ -311,4 +320,5 @@ void luring_cleanup(LuringState *s)
 {
 io_uring_queue_exit(&s->ring);
 g_free(s);
+trace_luring_cleanup_state();
 }
diff --git a/block/trace-events b/block/trace-events
index eab51497fc..c4564dcd96 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -60,6 +60,14 @@ qmp_block_stream(void *bs, void *job) "bs %p job %p"
 file_paio_submit(void *acb, void *opaque, int64_t offset, int count, int type) 
"acb %p opaque %p offset %"PRId64" count %d type %d"
 file_copy_file_range(void *bs, int src, int64_t src_off, int dst, int64_t 
dst_off, int64_t bytes, int flags, int64_t ret) "bs %p src_fd %d offset 
%"PRIu64" dst_fd %d offset %"PRIu64" bytes %"PRIu64" flags %d ret %"PRId64
 
+#io_uring.c
+luring_init_state(void *s, size_t size) "s %p size %zu"
+luring_cleanup_state(void) "s freed"
+disable luring_io_plug(void) "plug"
+disable luring_io_unplug(int blocked, int plugged, int queued, int inflight) 
"blocked %d plugged %d queued %d inflight %d"
+disable luring_do_submit(int blocked, int plugged, int queued, int inflight) 
"blocked %d plugged %d queued %d inflight %d"
+disable luring_do_submit_done(int ret) "submitted to kernel %d"
+
 # qcow2.c
 qcow2_writev_start_req(void *co, int64_t offset, int bytes) "co %p offset 0x%" 
PRIx64 " bytes %d"
 qcow2_writev_done_req(void *co, int ret) "co %p ret %d"
-- 
2.17.1

[Qemu-devel] [PATCH v5 08/12] block/file-posix.c: extend to use io_uring

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
---
 block/file-posix.c | 85 +-
 1 file changed, 69 insertions(+), 16 deletions(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index d018429672..211dfe5337 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -154,6 +154,7 @@ typedef struct BDRVRawState {
 bool has_write_zeroes:1;
 bool discard_zeroes:1;
 bool use_linux_aio:1;
+bool use_linux_io_uring:1;
 bool page_cache_inconsistent:1;
 bool has_fallocate;
 bool needs_alignment;
@@ -423,7 +424,7 @@ static QemuOptsList raw_runtime_opts = {
 {
 .name = "aio",
 .type = QEMU_OPT_STRING,
-.help = "host AIO implementation (threads, native)",
+.help = "host AIO implementation (threads, native, io_uring)",
 },
 {
 .name = "locking",
@@ -482,9 +483,15 @@ static int raw_open_common(BlockDriverState *bs, QDict 
*options,
 goto fail;
 }
 
-aio_default = (bdrv_flags & BDRV_O_NATIVE_AIO)
-  ? BLOCKDEV_AIO_OPTIONS_NATIVE
-  : BLOCKDEV_AIO_OPTIONS_THREADS;
+if (bdrv_flags & BDRV_O_NATIVE_AIO) {
+aio_default = BLOCKDEV_AIO_OPTIONS_NATIVE;
+#ifdef CONFIG_LINUX_IO_URING
+} else if (bdrv_flags & BDRV_O_IO_URING) {
+aio_default = BLOCKDEV_AIO_OPTIONS_IO_URING;
+#endif
+} else {
+aio_default = BLOCKDEV_AIO_OPTIONS_THREADS;
+}
 aio = qapi_enum_parse(&BlockdevAioOptions_lookup,
   qemu_opt_get(opts, "aio"),
   aio_default, &local_err);
@@ -493,7 +500,11 @@ static int raw_open_common(BlockDriverState *bs, QDict 
*options,
 ret = -EINVAL;
 goto fail;
 }
+
 s->use_linux_aio = (aio == BLOCKDEV_AIO_OPTIONS_NATIVE);
+#ifdef CONFIG_LINUX_IO_URING
+s->use_linux_io_uring = (aio == BLOCKDEV_AIO_OPTIONS_IO_URING);
+#endif
 
 locking = qapi_enum_parse(&OnOffAuto_lookup,
   qemu_opt_get(opts, "locking"),
@@ -557,7 +568,7 @@ static int raw_open_common(BlockDriverState *bs, QDict 
*options,
 s->shared_perm = BLK_PERM_ALL;
 
 #ifdef CONFIG_LINUX_AIO
- /* Currently Linux does AIO only for files opened with O_DIRECT */
+/* Currently Linux does AIO only for files opened with O_DIRECT */
 if (s->use_linux_aio) {
 if (!(s->open_flags & O_DIRECT)) {
 error_setg(errp, "aio=native was specified, but it requires "
@@ -579,6 +590,22 @@ static int raw_open_common(BlockDriverState *bs, QDict 
*options,
 }
 #endif /* !defined(CONFIG_LINUX_AIO) */
 
+#ifdef CONFIG_LINUX_IO_URING
+if (s->use_linux_io_uring) {
+if (!aio_setup_linux_io_uring(bdrv_get_aio_context(bs), errp)) {
+error_prepend(errp, "Unable to use io_uring: ");
+goto fail;
+}
+}
+#else
+if (s->use_linux_io_uring) {
+error_setg(errp, "aio=io_uring was specified, but is not supported "
+ "in this build.");
+ret = -EINVAL;
+goto fail;
+}
+#endif /* !defined(CONFIG_LINUX_IO_URING) */
+
 s->has_discard = true;
 s->has_write_zeroes = true;
 if ((bs->open_flags & BDRV_O_NOCACHE) != 0) {
@@ -1875,16 +1902,20 @@ static int coroutine_fn raw_co_prw(BlockDriverState 
*bs, uint64_t offset,
  * If this is the case tell the low-level driver that it needs
  * to copy the buffer.
  */
-if (s->needs_alignment) {
-if (!bdrv_qiov_is_aligned(bs, qiov)) {
-type |= QEMU_AIO_MISALIGNED;
+if (s->needs_alignment && !bdrv_qiov_is_aligned(bs, qiov)) {
+type |= QEMU_AIO_MISALIGNED;
+#ifdef CONFIG_LINUX_IO_URING
+} else if (s->use_linux_io_uring) {
+LuringState *aio = aio_get_linux_io_uring(bdrv_get_aio_context(bs));
+assert(qiov->size == bytes);
+return luring_co_submit(bs, aio, s->fd, offset, qiov, type);
+#endif
 #ifdef CONFIG_LINUX_AIO
-} else if (s->use_linux_aio) {
-LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
-assert(qiov->size == bytes);
-return laio_co_submit(bs, aio, s->fd, offset, qiov, type);
+} else if (s->use_linux_aio && s->needs_alignment) {
+LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
+assert(qiov->size == bytes);
+return laio_co_submit(bs, aio, s->fd, offset, qiov, type);
 #endif
-}
 }
 
 acb = (RawPosixAIOData) {
@@ -1920,24 +1951,36 @@ static int coroutine_fn raw_co_pwritev(BlockDriverState 
*bs, uint64_t offset,
 
 static void raw_aio_plug(BlockDriverState *bs)
 {
+BDRVRawState __attribute__((unused)) *s = bs->opaque;
 #ifdef CONFIG_LINUX_AIO
-BDRVRawState *s = bs->opaque;
 if (s->use_linux_aio) {
 LinuxAioState *aio = aio_get_linux_aio(bdrv_get_aio_context(bs));
 laio_io_plug(bs, aio);
 }
 #endif
+#ifdef CONFIG_LINUX_IO_URING
+if (s->use_linux_io_uring) {
+

[Qemu-devel] [PATCH v5 07/12] blockdev: accept io_uring as option

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
Reviewed-by: Stefan Hajnoczi 
---
 blockdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/blockdev.c b/blockdev.c
index 3f44b891eb..a2a5b32604 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -386,6 +386,8 @@ static void extract_common_blockdev_options(QemuOpts *opts, 
int *bdrv_flags,
 if ((aio = qemu_opt_get(opts, "aio")) != NULL) {
 if (!strcmp(aio, "native")) {
 *bdrv_flags |= BDRV_O_NATIVE_AIO;
+} else if (!strcmp(aio, "io_uring")) {
+*bdrv_flags |= BDRV_O_IO_URING;
 } else if (!strcmp(aio, "threads")) {
 /* this is the default */
 } else {
@@ -4579,7 +4581,7 @@ QemuOptsList qemu_common_drive_opts = {
 },{
 .name = "aio",
 .type = QEMU_OPT_STRING,
-.help = "host AIO implementation (threads, native)",
+.help = "host AIO implementation (threads, native, io_uring)",
 },{
 .name = BDRV_OPT_CACHE_WB,
 .type = QEMU_OPT_BOOL,
-- 
2.17.1

[Qemu-devel] [PATCH v5 12/12] qemu-iotests/087: checks for io_uring

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
---
 tests/qemu-iotests/087 | 26 ++
 tests/qemu-iotests/087.out | 10 ++
 2 files changed, 36 insertions(+)

diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087
index d6c8613419..0cc7283ad8 100755
--- a/tests/qemu-iotests/087
+++ b/tests/qemu-iotests/087
@@ -124,6 +124,32 @@ run_qemu_filter_aio <

[Qemu-devel] [PATCH v5 10/12] block/io_uring: adds userspace completion polling

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
---
 block/io_uring.c | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/block/io_uring.c b/block/io_uring.c
index 47e027364a..acfaa48151 100644
--- a/block/io_uring.c
+++ b/block/io_uring.c
@@ -142,6 +142,21 @@ static void qemu_luring_completion_cb(void *opaque)
 qemu_luring_process_completions_and_submit(s);
 }
 
+static bool qemu_luring_poll_cb(void *opaque)
+{
+LuringState *s = opaque;
+struct io_uring_cqe *cqes;
+
+if (io_uring_peek_cqe(&s->ring, &cqes) == 0) {
+if (!cqes) {
+qemu_luring_process_completions_and_submit(s);
+return true;
+}
+}
+
+return false;
+}
+
 static void ioq_init(LuringQueue *io_q)
 {
 QSIMPLEQ_INIT(&io_q->sq_overflow);
@@ -294,7 +309,7 @@ void luring_attach_aio_context(LuringState *s, AioContext 
*new_context)
 s->aio_context = new_context;
 s->completion_bh = aio_bh_new(new_context, qemu_luring_completion_bh, s);
 aio_set_fd_handler(s->aio_context, s->ring.ring_fd, false,
-   qemu_luring_completion_cb, NULL, NULL, s);
+   qemu_luring_completion_cb, NULL, qemu_luring_poll_cb, 
s);
 }
 
 LuringState *luring_init(Error **errp)
-- 
2.17.1

[Qemu-devel] [PATCH v5 11/12] qemu-io: adds support for io_uring

[Aarushi Mehta]

Signed-off-by: Aarushi Mehta 
---
 qemu-io.c | 13 +
 1 file changed, 13 insertions(+)

diff --git a/qemu-io.c b/qemu-io.c
index 8d5d5911cb..54b82151c4 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -129,6 +129,7 @@ static void open_help(void)
 " -n, -- disable host cache, short for -t none\n"
 " -U, -- force shared permissions\n"
 " -k, -- use kernel AIO implementation (on Linux only)\n"
+" -i  -- use kernel io_uring (Linux 5.1+)\n"
 " -t, -- use the given cache mode for the image\n"
 " -d, -- use the given discard mode for the image\n"
 " -o, -- options to be given to the block driver"
@@ -188,6 +189,11 @@ static int open_f(BlockBackend *blk, int argc, char **argv)
 case 'k':
 flags |= BDRV_O_NATIVE_AIO;
 break;
+#ifdef CONFIG_LINUX_IO_URING
+case 'i':
+flags |= BDRV_O_IO_URING;
+break;
+#endif
 case 't':
 if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) < 0) {
 error_report("Invalid cache option: %s", optarg);
@@ -290,6 +296,7 @@ static void usage(const char *name)
 "  -C, --copy-on-read   enable copy-on-read\n"
 "  -m, --misalign   misalign allocations for O_DIRECT\n"
 "  -k, --native-aio use kernel AIO implementation (on Linux only)\n"
+"  -i  --io_uring   use kernel io_uring (Linux 5.1+)\n"
 "  -t, --cache=MODE use the given cache mode for the image\n"
 "  -d, --discard=MODE   use the given discard mode for the image\n"
 "  -T, --trace [[enable=]][,events=][,file=]\n"
@@ -499,6 +506,7 @@ int main(int argc, char **argv)
 { "copy-on-read", no_argument, NULL, 'C' },
 { "misalign", no_argument, NULL, 'm' },
 { "native-aio", no_argument, NULL, 'k' },
+{ "io_uring", no_argument, NULL, 'i' },
 { "discard", required_argument, NULL, 'd' },
 { "cache", required_argument, NULL, 't' },
 { "trace", required_argument, NULL, 'T' },
@@ -566,6 +574,11 @@ int main(int argc, char **argv)
 case 'k':
 flags |= BDRV_O_NATIVE_AIO;
 break;
+#ifdef CONFIG_LINUX_IO_URING
+case 'i':
+flags |= BDRV_O_IO_URING;
+break;
+#endif
 case 't':
 if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) < 0) {
 error_report("Invalid cache option: %s", optarg);
-- 
2.17.1

Re: [Qemu-devel] [PATCH] pc: fix possible NULL pointer dereference in pc_machine_get_device_memory_region_size()

[Paolo Bonzini]

On 10/06/19 15:50, Igor Mammedov wrote:
> QEMU will crash when device-memory-region-size property is read if 
> ms->device_memory
> wasn't initialized yet.
> 
> Crash can be reproduced with:
>  $QEMU -preconfig -qmp unix:qmp_socket,server,nowait &
>  ./scripts/qmp/qom-get -s qmp_socket /machine.device-memory-region-size
> 
> Instead of crashing return 0 if ms->device_memory hasn't been initialized.
> 
> Signed-off-by: Igor Mammedov 
> ---
> v2:
>   add reproducer to commit message
>(Markus Armbruster )
> 
>  hw/i386/pc.c | 6 +-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index edc240b..1b7ead9 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -2459,7 +2459,11 @@ pc_machine_get_device_memory_region_size(Object *obj, 
> Visitor *v,
>   Error **errp)
>  {
>  MachineState *ms = MACHINE(obj);
> -int64_t value = memory_region_size(&ms->device_memory->mr);
> +int64_t value = 0;
> +
> +if (ms->device_memory) {
> +memory_region_size(&ms->device_memory->mr);
> +}
>  
>  visit_type_int(v, name, &value, errp);
>  }
> 

Queued, thanks.

Paolo

Re: [Qemu-devel] [Qemu-block] [PATCH] file-posix: unlock qemu_global_mutex before pread when attach disk

[Paolo Bonzini]

On 10/06/19 15:34, Zhengui li wrote:
> 
> when do qmp sush as drive_add,  qemu main thread locks the
> qemu_global_mutex  and do pread in raw_probe_alignmen. Pread is a
> synchronous operation. If backend storage network has a large delay
> or IO pressure is too large,  the pread operation will not return for
> a long time, which make vcpu thread can't acquire qemu_global_mutex
> for a long time and make the vcpu thread unable to be scheduled for a
> long time.  So virtual machine cpu soft lockup happened.
> 
> qemu main thread should not hold qemu_global_mutex for a long time
> when do qmp that involving IO synchronous operation sush pread ,
> ioctl, etc. So this patch unlock qemu_global_mutex before IO
> synchronous operation sush pread.

These preads are for 512-4096 bytes, can they really last much longer
than the "open" that precedes them?  If pread of 4K can trigger a soft
lockup, things are really screwed up---and it's hard to be sure that all
callers of raw_probe_alignment are okay with releasing the global mutex.

Paolo

Re: [Qemu-devel] [PATCH] qemu-ga: Convert invocation documentation to rST

[Paolo Bonzini]

On 10/06/19 15:45, Peter Maydell wrote:
> +# Canned command to build manpages from a single manual
> +build-manpages = $(call quiet-command,CONFDIR="$(qemu_confdir)" sphinx-build 
> $(if $(V),,-q) -W -n -b man -D version=$(VERSION) -D 
> release="$(FULL_VERSION)" -d .doctrees/$1 $(SRC_PATH)/docs/$1 
> $(MANUAL_BUILDDIR)/$1 ,"SPHINX","$(MANUAL_BUILDDIR)/$1")


...

> +# The rst_epilog fragment is effectively included in every rST file.
> +# We use it to define substitutions based on build config that
> +# can then be used in the documentation. The fallback if the
> +# environment variable is not set is for the benefit of readthedocs
> +# style document building; our Makefile always sets the variable.
> +confdir = os.getenv('CONFDIR', "/usr/local/etc")

This should be /usr/local/etc/qemu is you want $(qemu_confdir) above and
not $(sysconfdir).

But since we always set the variable when building from the QEMU build
system, perhaps "/etc" or "/etc/qemu" is a more useful default when
building the manual outside QEMU?

Paolo

> +rst_epilog = ".. |CONFDIR| replace:: ``" + confdir + "``\n"
> +

Re: [Qemu-devel] qgraph

[Paolo Bonzini]

On 10/06/19 15:28, Andreas Färber wrote:
> So if we want a new QMP operation, the most sense would probably make
> where-can-I-attach-type(foo) returning a list of QOM paths, showing only
> the first free slot per bus. That would allow a more efficient lookup
> implementation inside QEMU than needing to check each slot[n] property
> via qom-get after discovering it with qom-list.

Note that what Natalia is seeking is an introspection mechanism to be
used _before_ creating a virtual machine though.

Paolo

Re: [Qemu-devel] [PATCH] RFC: qio: Improve corking of TLS sessions

[Eric Blake]

On 6/10/19 4:08 AM, Daniel P. Berrangé wrote:
> On Fri, Jun 07, 2019 at 05:14:14PM -0500, Eric Blake wrote:
>> Our current implementation of qio_channel_set_cork() is pointless for
>> TLS sessions: we block the underlying channel, but still hand things
>> piecemeal to gnutls which then produces multiple encryption packets.
>> Better is to directly use gnutls corking, which collects multiple
>> inputs into a single encryption packet.
>>
>> Signed-off-by: Eric Blake 
>>
>> ---
>>
>> RFC because unfortunately, I'm not sure I like the results.  My test
>> case (using latest nbdkit.git) involves sending 10G of random data
>> over NBD using parallel writes (and doing nothing on the server end;
>> this is all about timing the data transfer):
>>
>> $ dd if=/dev/urandom of=rand bs=1M count=10k
>> $ time nbdkit -p 10810 --tls=require --tls-verify-peer \
>>--tls-psk=/tmp/keys.psk --filter=stats null 10g statsfile=/dev/stderr \
>>--run '~/qemu/qemu-img convert -f raw -W -n --target-image-opts \
>>  --object tls-creds-psk,id=tls0,endpoint=client,dir=/tmp,username=eblake 
>> \
>>  rand 
>> driver=nbd,server.type=inet,server.host=localhost,server.port=10810,tls-creds=tls0'
>>
>> Pre-patch, I measured:
>> real 0m34.536s
>> user 0m29.264s
>> sys  0m4.014s
>>
>> while post-patch, it changed to:
>> real 0m36.055s
>> user 0m27.685s
>> sys  0m10.138s
>>
>> Less time spent in user space, but for this particular qemu-img
>> behavior (writing 2M chunks at a time), gnutls is now uncorking huge
>> packets and the kernel is doing enough extra work that the overall
>> program actually takes longer. :(
> 
> I wonder if the problem is that we're hitting a tradeoff between
> time spent in encryption vs time spent in network I/O.
> 
> When we don't cork, the kernel has already sent the first packet
> during the time gnutls is burning CPU encrypting the second packet.
> 
> When we do cork gnutls has to encrypt both packets before the kernel
> can send anything.

Exactly. That's why my gut feel is that having only a binary cork is too
course, and we instead want something more like MSG_MORE. If the flag is
set, AND the current size is small, then cork; then on the next message,
we see that we are still corked, and decide to either uncork immediately
(current message is large, corking buys us nothing useful as we're going
to split the current message anyway) or uncork after appending the
current message (current message is small enough that keeping things
corked may still be a win).  Visually,

cork
send()
send()
uncork

is too late - you can't tell whether the second send would have
benefitted from staying corked after the first.  But:

send(MSG_MORE)
send()

is ideal; under the hood, we can translate it to:

send(MSG_MORE)
  gnutls_record_cork()
  gnutls_record_send()
send()
  if (size > threshold) {
gnutls_record_uncork()
gnutls_record_send()
  } else {
gnutls_record_send()
gnutls_record_uncork()
  }

So we really need a way to plumb a MSG_MORE flag for senders to use,
when they KNOW they will be sending back-to-back pieces and where the
first piece is short, but it is not yet obvious whether the second piece
is short or long.

MSG_MORE was lon the next message to go through the stack, if the
previous message next paccork for

> 
>> For smaller I/O patterns, the effects of corking are more likely to be
>> beneficial, but I don't have a ready test case to produce that pattern
>> (short of creating a guest running fio on a device backed by nbd).
> 
> I think it would probably be useful to see guest kernels with fio
> and definitely see results for something closer to sector sized
> I/O.
> 
> 2 MB chunks is pretty unrealistic for a guest workload where there
> will be lots of sector sized I/O. Sure QEMU / guest OS can merge
> sectors, but it still feels like most I/O is going to be much smaller
> than 2 MB.

I'll have to play with other benchmarking setups, and see if I can come
up with a reliable fio test.

> 
> 
>> diff --git a/io/channel.c b/io/channel.c
>> index 2a26c2a2c0b9..3510912465ac 100644
>> --- a/io/channel.c
>> +++ b/io/channel.c
>> @@ -379,7 +379,16 @@ void qio_channel_set_cork(QIOChannel *ioc,
>>  QIOChannelClass *klass = QIO_CHANNEL_GET_CLASS(ioc);
>>
>>  if (klass->io_set_cork) {
>> -klass->io_set_cork(ioc, enabled);
>> +int r = klass->io_set_cork(ioc, enabled);
>> +
>> +while (r == QIO_CHANNEL_ERR_BLOCK) {
>> +if (qemu_in_coroutine()) {
>> +qio_channel_yield(ioc, G_IO_OUT);
>> +} else {
>> +qio_channel_wait(ioc, G_IO_OUT);
>> +}
>> +r = klass->io_set_cork(ioc, enabled);
>> +}
>>  }
> 
> Interesting - did you actually hit this EAGAIN behaviour ? I wouldn't
> have expected anything to be pending in gnutls that needs retry. 

Yes. When gnutls_record_cork() is called, NOTHING goes over the wire,
but instead everything gets appended to a realloc'd buffer; when you
finally gn

Re: [Qemu-devel] [PATCH] qemu-ga: Convert invocation documentation to rST

[Peter Maydell]

On Mon, 10 Jun 2019 at 14:56, Paolo Bonzini  wrote:
>
> On 10/06/19 15:45, Peter Maydell wrote:
> > +# Canned command to build manpages from a single manual
> > +build-manpages = $(call quiet-command,CONFDIR="$(qemu_confdir)" 
> > sphinx-build $(if $(V),,-q) -W -n -b man -D version=$(VERSION) -D 
> > release="$(FULL_VERSION)" -d .doctrees/$1 $(SRC_PATH)/docs/$1 
> > $(MANUAL_BUILDDIR)/$1 ,"SPHINX","$(MANUAL_BUILDDIR)/$1")
>
>
> ...
>
> > +# The rst_epilog fragment is effectively included in every rST file.
> > +# We use it to define substitutions based on build config that
> > +# can then be used in the documentation. The fallback if the
> > +# environment variable is not set is for the benefit of readthedocs
> > +# style document building; our Makefile always sets the variable.
> > +confdir = os.getenv('CONFDIR', "/usr/local/etc")
>
> This should be /usr/local/etc/qemu is you want $(qemu_confdir) above and
> not $(sysconfdir).
>
> But since we always set the variable when building from the QEMU build
> system, perhaps "/etc" or "/etc/qemu" is a more useful default when
> building the manual outside QEMU?

Yeah, I wasn't sure what to set this to either...

thanks
-- PMM

Re: [Qemu-devel] [PATCH] file-posix: unlock qemu_global_mutex before pread when attach disk

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/1560173684-6264-1-git-send-email-lizhen...@huawei.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH] file-posix: unlock qemu_global_mutex before pread 
when attach disk
Type: series
Message-id: 1560173684-6264-1-git-send-email-lizhen...@huawei.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
 * [new tag]   
patchew/1560173684-6264-1-git-send-email-lizhen...@huawei.com -> 
patchew/1560173684-6264-1-git-send-email-lizhen...@huawei.com
Switched to a new branch 'test'
a5549055a3 file-posix: unlock qemu_global_mutex before pread when attach disk

=== OUTPUT BEGIN ===
ERROR: Missing Signed-off-by: line(s)

total: 1 errors, 0 warnings, 9 lines checked

Commit a5549055a30d (file-posix: unlock qemu_global_mutex before pread when 
attach disk) has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/1560173684-6264-1-git-send-email-lizhen...@huawei.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[Qemu-devel] Need help generating instruction traces from the emulator.

[Nisarg Ujjainkar]

Hello,

I am using qemu based aosp (emu-master-dev branch of the aosp) to generate
the instruction traces for android os running on the ARM architecture.

I am able to generate the CPU instruction using the qemu invocation
 flags. For the
purpose of my study, I need all the memory requests from all the IPs and so
far I only have the memory requests from the CPU.

Can tell me about how to generate the traces from all other IPs (GPU, GSM
chip etc.). Since memory requests from other IPs account for more than 30%
of all the memory requests from the SoC (Source unconfirmed). Getting
memory requests from these IPs is very crucial for my research.

Thanks and regards
Nisarg Ujjainkar

-- 
Nisarg Ujjainkar
Junior Undergraduate Student
Department of Computer Science and Engineering
IIT Gandhinagar
+91 926488 \ 9425665211
ᐧ

Re: [Qemu-devel] [PATCH v6 6/7] block/nbd-client: nbd reconnect

[Eric Blake]

On 6/10/19 8:29 AM, Vladimir Sementsov-Ogievskiy wrote:

>> Hmm, and then, include it into BDRVNBDState? I don't remember why didn't do
>> it, and now I don't see any reason for it. We store this information anyway
>> for the whole life of the driver..
>>
>> So, if I'm going to refactor it, I have a question: is there a reason for
>> layered BDRVNBDState:
>>
>> typedef struct BDRVNBDState {
>>      NBDClientSession client;
>>
>>      /* For nbd_refresh_filename() */
>>      SocketAddress *saddr;
>>      char *export, *tlscredsid;
>> } BDRVNBDState;
>>
>> and use nbd_get_client_session everywhere instead of simple converting void 
>> *opaque
>> to State pointer like in qcow2 for example?
>>
>> The only reason I can imagine is not to share the whole BDRVNBDState, and 
>> keep
>> things we are using only in nbd.c to be available only in nbd.c.. But I've 
>> put
>> saddr and export to NBDConnection to be used in nbd-client.c anyway. So, I 
>> think
>> it's a good moment to flatten and share BDRVNBDState and drop 
>> nbd_get_client_session.
>>
> 
> And if we are here, what is exact purpose of splitting  nbd-client.c from 
> nbd.c?

I see no strong reasons to keep the separation. If a single large file
is easier to maintain than an arbitrary split between two files, I'm
open to the idea of a patch that undoes the split.


> or need of defining driver callbacks in header file, instead of keeping them 
> together
> with driver struct definition and static (like other block drivers)...
> 
> 
> And names of these two files always confused me.. nbd.c is nbd client block 
> driver, and
> driver is defined here.. But we have nbd-client.c which defines nbd client 
> driver too.
> And we also have nbd/client.c (OK, this split I understand of course, but it 
> increases
> confusion anyway).

I'm also toying with the idea of writing block/nbd.c to utilize the
relatively-new libnbd library, to see if there are any differences in
behavior by offloading NBD access to a 3rd-party library.  But that's
further down the road.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 0/8] WIP: Multifd compression support

[Eric Blake]

On 5/20/19 1:35 AM, Wei Yang wrote:
> On Wed, May 15, 2019 at 02:15:36PM +0200, Juan Quintela wrote:
>> v3:
>> - improve the code
>> - address David and Markus comments
>> - make compression code into methods
>>  so we can add any other method ading just three functions
>>
>> Please review, as far as I know everything is ok now.
>>
>> Todo: Add zstd support
> 
> Confusion here. It is zstd or sztd?
> 
> BTW, I am not sure what it is :-)
> 
>>
>> v2:
>> - improve the code left and right
>> - Split better the zlib code
>> - rename everything to v4.1
>> - Add tests for multifd-compress zlib
>> - Parameter is now an enum (soon will see sztd)
>^^^

zstd is the name of the new compression algorithm.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 1/7] iotests: allow Valgrind checking all QEMU processes

[Eric Blake]

On 6/9/19 1:35 PM, Andrey Shinkevich wrote:
> With the '-valgrind' option, let all the QEMU processes be run under
> the Valgrind tool. The Valgrind own parameters may be set with its
> environment variable VALGRIND_OPTS, e.g.
> VALGRIND_OPTS="--leak-check=yes" ./check -qcow2 -valgrind 

Let's spell this --valgrind; long options should prefer the use of --
(as in getopt_long), whether or not we also have a reason to support
-valgrind (as in getopt_long_only). Yes, qemu is an oddball in this
regards, but no need to make it worse.

> 
> Signed-off-by: Andrey Shinkevich 
> ---
>  tests/qemu-iotests/common.rc | 65 
> 
>  1 file changed, 48 insertions(+), 17 deletions(-)
> 
> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
> index 93f8738..3caaca4 100644


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] RFC: qio: Improve corking of TLS sessions

[Eric Blake]

On 6/10/19 9:02 AM, Eric Blake wrote:

> 
> send(MSG_MORE)
> send()
> 
> is ideal; under the hood, we can translate it to:
> 
> send(MSG_MORE)
>   gnutls_record_cork()
>   gnutls_record_send()
> send()
>   if (size > threshold) {
> gnutls_record_uncork()
> gnutls_record_send()
>   } else {
> gnutls_record_send()
> gnutls_record_uncork()
>   }
> 
> So we really need a way to plumb a MSG_MORE flag for senders to use,
> when they KNOW they will be sending back-to-back pieces and where the
> first piece is short, but it is not yet obvious whether the second piece
> is short or long.

This is what I meant to say,

> 
> MSG_MORE was lon the next message to go through the stack, if the
> previous message next paccork for

this was an editing accident on incomplete thoughts.  But I wanted to add:

Setting up the ability to pass MGS_MORE through the qio stack will
require either an update to ALL callers of qio_write to pass a flags
argument (usually 0), or to add a set of new entry points to qio for the
few callers that want to pass a non-zero flags argument (for now, nbd
and sheepdog).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] Need help generating instruction traces from the emulator.

[Peter Maydell]

On Mon, 10 Jun 2019 at 15:24, Nisarg Ujjainkar
 wrote:
>
> Hello,
>
> I am using qemu based aosp (emu-master-dev branch of the aosp) to generate
> the instruction traces for android os running on the ARM architecture.
>
> I am able to generate the CPU instruction using the qemu invocation
>  flags. For the
> purpose of my study, I need all the memory requests from all the IPs and so
> far I only have the memory requests from the CPU.

This isn't supported by QEMU's logging infrastructure.
You might be able to find places to add suitable logging,
but you'd be looking at modifying the source code of the
relevant devices to do that.

thanks
-- PMM

Re: [Qemu-devel] [Qemu-block] [PATCH] file-posix: unlock qemu_global_mutex before pread when attach disk

[l00284672]

The pread will hang in attaching disk just when backend storage network 
disconnection .


I think the locking range of qemu_global_mutex is too large when do qmp 
operation. what


does the qemu_global_mutex  really protect?  what is the risk of 
unlocking qemu_global_mutex


in qmp?


On 2019/6/10 21:51, Paolo Bonzini wrote:

On 10/06/19 15:34, Zhengui li wrote:

when do qmp sush as drive_add,  qemu main thread locks the
qemu_global_mutex  and do pread in raw_probe_alignmen. Pread is a
synchronous operation. If backend storage network has a large delay
or IO pressure is too large,  the pread operation will not return for
a long time, which make vcpu thread can't acquire qemu_global_mutex
for a long time and make the vcpu thread unable to be scheduled for a
long time.  So virtual machine cpu soft lockup happened.

qemu main thread should not hold qemu_global_mutex for a long time
when do qmp that involving IO synchronous operation sush pread ,
ioctl, etc. So this patch unlock qemu_global_mutex before IO
synchronous operation sush pread.

These preads are for 512-4096 bytes, can they really last much longer
than the "open" that precedes them?  If pread of 4K can trigger a soft
lockup, things are really screwed up---and it's hard to be sure that all
callers of raw_probe_alignment are okay with releasing the global mutex.

Paolo

.



<>

Re: [Qemu-devel] [PATCH] qemu-ga: Convert invocation documentation to rST

[Peter Maydell]

On Mon, 10 Jun 2019 at 14:45, Peter Maydell  wrote:
>
> The qemu-ga documentation is currently in qemu-ga.texi in
> Texinfo format, which we present to the user as:
>  * a qemu-ga manpage
>  * a section of the main qemu-doc HTML documentation
>
> Convert the documentation to rST format, and present it to
> the user as:
>  * a qemu-ga manpage
>  * part of the interop/ Sphinx manual
>
> Signed-off-by: Peter Maydell 
> ---
> This is part of my general proposals about how we should
> transition away from texinfo to sphinx (written up at
> https://wiki.qemu.org/Features/Documentation). It's the
> first part of step 3 (convert standalone manpages), so it's
> interesting as a demonstration of Sphinx generating manpages
> as well as HTML. The format of the output manpage is broadly
> the same, though there are a few minor differences because
> rST doesn't support quite the same kinds of output. (It also
> fixes a bug where the current manpage renders some text intended
> to be in bold as literally "B".)
>
> Having the infrastructure for creating a simple manpage via
> Sphinx should be a useful assist for the work in step 1 of the
> transition plan that involves conversion of the auto-generated
> qemu-ga-ref and qemu-qmp-ref (which need to produce manpages).
>
> The non-manpage HTML version of the doc moves from qemu-doc.html
> into docs/interop/ (its final location in the new 5-manual setup).
>
>  Makefile |  13 ++--
>  MAINTAINERS  |   2 +-
>  docs/conf.py |  18 ++---
>  docs/interop/conf.py |   7 ++
>  docs/interop/index.rst   |   1 +
>  docs/interop/qemu-ga.rst | 133 +
>  qemu-doc.texi|   5 --
>  qemu-ga.texi | 137 ---
>  8 files changed, 161 insertions(+), 155 deletions(-)
>  create mode 100644 docs/interop/qemu-ga.rst
>  delete mode 100644 qemu-ga.texi

I just realised that I forgot to update the Makefile 'install-manual'
macro to filter out the generated docs/interop/qemu-ga.8 file,
so at the moment 'make install' will put a copy into
/usr/local/share/doc/qemu/interop/ as well as into
/usr/local/share/man/man8/. I'll fix that in v2.

thanks
-- PMM

[Qemu-devel] [PATCH 14/39] target/i386: Use env_cpu, env_archcpu

[Richard Henderson]

Cleanup in the boilerplate that each target must define.
Replace x86_env_get_cpu with env_archcpu.  The combination
CPU(x86_env_get_cpu) should have used ENV_GET_CPU to begin;
use env_cpu now.

Reviewed-by: Alistair Francis 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/i386/cpu.h  |  5 -
 bsd-user/main.c|  3 +--
 hw/i386/kvmvapic.c |  4 ++--
 hw/i386/pc.c   |  2 +-
 linux-user/i386/cpu_loop.c |  2 +-
 linux-user/i386/signal.c   |  2 +-
 linux-user/vm86.c  | 18 +-
 target/i386/bpt_helper.c   |  4 ++--
 target/i386/cpu.c  |  4 ++--
 target/i386/excp_helper.c  |  2 +-
 target/i386/fpu_helper.c   |  2 +-
 target/i386/helper.c   | 16 ++--
 target/i386/misc_helper.c  | 24 +++-
 target/i386/seg_helper.c   | 14 +++---
 target/i386/smm_helper.c   |  4 ++--
 target/i386/svm_helper.c   | 22 +++---
 16 files changed, 58 insertions(+), 70 deletions(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 103fd709b0..709d88cfcf 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1480,11 +1480,6 @@ struct X86CPU {
 int32_t hv_max_vps;
 };
 
-static inline X86CPU *x86_env_get_cpu(CPUX86State *env)
-{
-return container_of(env, X86CPU, env);
-}
-
 #define ENV_OFFSET offsetof(X86CPU, env)
 
 #ifndef CONFIG_USER_ONLY
diff --git a/bsd-user/main.c b/bsd-user/main.c
index 6192e9d91e..53e1f42408 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -140,8 +140,7 @@ static void set_idt(int n, unsigned int dpl)
 
 void cpu_loop(CPUX86State *env)
 {
-X86CPU *cpu = x86_env_get_cpu(env);
-CPUState *cs = CPU(cpu);
+CPUState *cs = env_cpu(env);
 int trapnr;
 abi_ulong pc;
 //target_siginfo_t info;
diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index 70f6f26a94..fe5b12ef6e 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -152,7 +152,7 @@ static void update_guest_rom_state(VAPICROMState *s)
 
 static int find_real_tpr_addr(VAPICROMState *s, CPUX86State *env)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 hwaddr paddr;
 target_ulong addr;
 
@@ -279,7 +279,7 @@ instruction_ok:
 
 static int update_rom_mapping(VAPICROMState *s, CPUX86State *env, target_ulong 
ip)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 hwaddr paddr;
 uint32_t rom_state_vaddr;
 uint32_t pos, patch, offset;
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index edc240bcbf..1b08b56362 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -406,7 +406,7 @@ uint64_t cpu_get_tsc(CPUX86State *env)
 /* IRQ handling */
 int cpu_get_pic_interrupt(CPUX86State *env)
 {
-X86CPU *cpu = x86_env_get_cpu(env);
+X86CPU *cpu = env_archcpu(env);
 int intno;
 
 if (!kvm_irqchip_in_kernel()) {
diff --git a/linux-user/i386/cpu_loop.c b/linux-user/i386/cpu_loop.c
index 51cfa006c9..71da24384f 100644
--- a/linux-user/i386/cpu_loop.c
+++ b/linux-user/i386/cpu_loop.c
@@ -82,7 +82,7 @@ static void set_idt(int n, unsigned int dpl)
 
 void cpu_loop(CPUX86State *env)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 int trapnr;
 abi_ulong pc;
 abi_ulong ret;
diff --git a/linux-user/i386/signal.c b/linux-user/i386/signal.c
index fecb4c99c3..97a39204cc 100644
--- a/linux-user/i386/signal.c
+++ b/linux-user/i386/signal.c
@@ -198,7 +198,7 @@ static void setup_sigcontext(struct target_sigcontext *sc,
 struct target_fpstate *fpstate, CPUX86State *env, abi_ulong mask,
 abi_ulong fpstate_addr)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 #ifndef TARGET_X86_64
 uint16_t magic;
 
diff --git a/linux-user/vm86.c b/linux-user/vm86.c
index 9c393df424..2fa7a89edc 100644
--- a/linux-user/vm86.c
+++ b/linux-user/vm86.c
@@ -72,7 +72,7 @@ static inline unsigned int vm_getl(CPUX86State *env,
 
 void save_v86_state(CPUX86State *env)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 TaskState *ts = cs->opaque;
 struct target_vm86plus_struct * target_v86;
 
@@ -132,7 +132,7 @@ static inline void return_to_32bit(CPUX86State *env, int 
retval)
 
 static inline int set_IF(CPUX86State *env)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 TaskState *ts = cs->opaque;
 
 ts->v86flags |= VIF_MASK;
@@ -145,7 +145,7 @@ static inline int set_IF(CPUX86State *env)
 
 static inline void clear_IF(CPUX86State *env)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 TaskState *ts = cs->opaque;
 
 ts->v86flags &= ~VIF_MASK;
@@ -163,7 +163,7 @@ static inline void clear_AC(CPUX86State *env)
 
 static inline int set_vflags_long(unsigned long eflags, CPUX86State *env)
 {
-CPUState *cs = CPU(x86_env_get_cpu(env));
+CPUState *cs = env_cpu(env);
 TaskState *ts = cs->opaque;
 
 set_f

[Qemu-devel] [PULL v2 00/39] tcg: Move softmmu tlb into CPUNegativeOffsetState

[Richard Henderson]

V2 should fix a typo affecting OpenBSD.


r~

The following changes since commit 19735c837ae2056b4651720290eda59498eca65a:

  Merge remote-tracking branch 'remotes/kraxel/tags/usb-20190607-pull-request' 
into staging (2019-06-10 11:53:19 +0100)

are available in the Git repository at:

  https://github.com/rth7680/qemu.git tags/pull-tcg-20190610

for you to fetch changes up to 43b3952dea0f763ceeaa2f119c473b5cc6d29c90:

  tcg/arm: Remove mostly unreachable tlb special case (2019-06-10 07:03:42 
-0700)


Move softmmu tlb into CPUNegativeOffsetState


Richard Henderson (39):
  tcg: Fold CPUTLBWindow into CPUTLBDesc
  tcg: Split out target/arch/cpu-param.h
  tcg: Create struct CPUTLB
  cpu: Define CPUArchState with typedef
  cpu: Define ArchCPU
  cpu: Replace ENV_GET_CPU with env_cpu
  cpu: Introduce env_archcpu
  target/alpha: Use env_cpu, env_archcpu
  target/arm: Use env_cpu, env_archcpu
  target/cris: Reindent mmu.c
  target/cris: Reindent op_helper.c
  target/cris: Use env_cpu, env_archcpu
  target/hppa: Use env_cpu, env_archcpu
  target/i386: Use env_cpu, env_archcpu
  target/lm32: Use env_cpu, env_archcpu
  target/m68k: Use env_cpu
  target/microblaze: Use env_cpu, env_archcpu
  target/mips: Use env_cpu, env_archcpu
  target/moxie: Use env_cpu, env_archcpu
  target/nios2: Use env_cpu, env_archcpu
  target/openrisc: Use env_cpu, env_archcpu
  target/ppc: Use env_cpu, env_archcpu
  target/riscv: Use env_cpu, env_archcpu
  target/s390x: Use env_cpu, env_archcpu
  target/sh4: Use env_cpu, env_archcpu
  target/sparc: Use env_cpu, env_archcpu
  target/tilegx: Use env_cpu
  target/tricore: Use env_cpu
  target/unicore32: Use env_cpu, env_archcpu
  target/xtensa: Use env_cpu, env_archcpu
  cpu: Move ENV_OFFSET to exec/gen-icount.h
  cpu: Introduce cpu_set_cpustate_pointers
  cpu: Introduce CPUNegativeOffsetState
  cpu: Move icount_decr to CPUNegativeOffsetState
  cpu: Move the softmmu tlb to CPUNegativeOffsetState
  cpu: Remove CPU_COMMON
  tcg/aarch64: Use LDP to load tlb mask+table
  tcg/arm: Use LDRD to load tlb mask+table
  tcg/arm: Remove mostly unreachable tlb special case

 accel/tcg/atomic_template.h   |   8 +-
 include/exec/cpu-all.h|  69 +++
 include/exec/cpu-defs.h   | 111 ++--
 include/exec/cpu_ldst.h   |   6 +-
 include/exec/cpu_ldst_template.h  |   6 +-
 include/exec/cpu_ldst_useronly_template.h |   6 +-
 include/exec/gen-icount.h |  14 +-
 include/exec/softmmu-semi.h   |  16 +-
 include/qom/cpu.h |  40 +-
 linux-user/cpu_loop-common.h  |   2 +-
 linux-user/m68k/target_cpu.h  |   2 +-
 target/alpha/cpu-param.h  |  31 ++
 target/alpha/cpu.h|  40 +-
 target/arm/cpu-param.h|  34 ++
 target/arm/cpu.h  |  52 +-
 target/cris/cpu-param.h   |  17 +
 target/cris/cpu.h |  25 +-
 target/hppa/cpu-param.h   |  34 ++
 target/hppa/cpu.h |  38 +-
 target/i386/cpu-param.h   |  28 +
 target/i386/cpu.h |  40 +-
 target/lm32/cpu-param.h   |  17 +
 target/lm32/cpu.h |  25 +-
 target/m68k/cpu-param.h   |  22 +
 target/m68k/cpu.h |  28 +-
 target/microblaze/cpu-param.h |  18 +
 target/microblaze/cpu.h   |  63 +--
 target/mips/cpu-param.h   |  29 ++
 target/mips/cpu.h |  21 +-
 target/mips/mips-defs.h   |  15 -
 target/moxie/cpu-param.h  |  17 +
 target/moxie/cpu.h|  29 +-
 target/nios2/cpu-param.h  |  21 +
 target/nios2/cpu.h|  33 +-
 target/openrisc/cpu-param.h   |  17 +
 target/openrisc/cpu.h |  31 +-
 target/ppc/cpu-param.h|  37 ++
 target/ppc/cpu.h  |  61 +--
 target/ppc/helper_regs.h  |   4 +-
 target/riscv/cpu-param.h  |  23 +
 target/riscv/cpu.h|  34 +-
 target/s390x/cpu-param.h  |  17 +
 target/s390x/cpu.h|  31 +-
 target/sh4/cpu-param.h|  21 +
 target/sh4/cpu.h  |  30 +-
 target/sparc/cpu-param.h  |  28 +
 target/sparc/cpu.h|  36 +-
 target/tilegx/cpu-param.h |  17 +
 target/tilegx/cpu.h   |  23 +-
 target/tricore/cpu-param.h|  17

Re: [Qemu-devel] [PATCH 1/7] iotests: allow Valgrind checking all QEMU processes

[Andrey Shinkevich]

On 10/06/2019 17:24, Eric Blake wrote:
> On 6/9/19 1:35 PM, Andrey Shinkevich wrote:
>> With the '-valgrind' option, let all the QEMU processes be run under
>> the Valgrind tool. The Valgrind own parameters may be set with its
>> environment variable VALGRIND_OPTS, e.g.
>> VALGRIND_OPTS="--leak-check=yes" ./check -qcow2 -valgrind 
> 
> Let's spell this --valgrind; long options should prefer the use of --
> (as in getopt_long), whether or not we also have a reason to support
> -valgrind (as in getopt_long_only). Yes, qemu is an oddball in this
> regards, but no need to make it worse.
> 

Thank you, Eric. That sounds good but the short option'-valgrind' is
preexisting in QEMU. Should I create a new patch for the long option?
If so, will we have both options supported by QEMU?
Andrey

>>
>> Signed-off-by: Andrey Shinkevich 
>> ---
>>   tests/qemu-iotests/common.rc | 65 
>> 
>>   1 file changed, 48 insertions(+), 17 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
>> index 93f8738..3caaca4 100644
> 
> 

-- 
With the best regards,
Andrey Shinkevich

Re: [Qemu-devel] [PATCH v21 3/7] target/avr: Add instruction decoding

[Richard Henderson]

On 6/6/19 12:30 PM, Michael Rolnik wrote:
> +STS 1001 001 .  @ldst_s
> +
> -- 

checkpatch error for extra blank line at end of file.


r~

Re: [Qemu-devel] [PATCH v21 4/7] target/avr: Add instruction translation

[Richard Henderson]

On 6/6/19 12:30 PM, Michael Rolnik wrote:
> +void avr_cpu_tcg_init(void)
> +{
> +int i;
> +
> +#define AVR_REG_OFFS(x) offsetof(CPUAVRState, x)
> +cpu_pc = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(pc_w), "pc");
> +cpu_Cf = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregC), "Cf");
> +cpu_Zf = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregZ), "Zf");
> +cpu_Nf = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregN), "Nf");
> +cpu_Vf = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregV), "Vf");
> +cpu_Sf = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregS), "Sf");
> +cpu_Hf = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregH), "Hf");
> +cpu_Tf = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregT), "Tf");
> +cpu_If = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sregI), "If");
> +cpu_rampD = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(rampD), 
> "rampD");
> +cpu_rampX = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(rampX), 
> "rampX");
> +cpu_rampY = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(rampY), 
> "rampY");
> +cpu_rampZ = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(rampZ), 
> "rampZ");
> +cpu_eind = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(eind), "eind");
> +cpu_sp = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(sp), "sp");
> +cpu_skip = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(skip), "skip");
> +
> +for (i = 0; i < 32; i++) {
> +char name[16];
> +
> +sprintf(name, "r[%d]", i);
> +
> +cpu_r[i] = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(r[i]), name);
> +}
> +}

These register names need to be permanently allocated.
I suggest

static const char reg_names[32][8] = {
"r[0]", "r[1]" ...
};
cpu_r[i] = tcg_global_mem_new_i32(cpu_env, AVR_REG_OFFS(r[i]),
  reg_names[i]);


r~