[PATCH] .gitignore: add virtiofsd binary

2020-08-21 Thread Li Zhijian
Signed-off-by: Li Zhijian 
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 2992d15931..28729241f1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -57,6 +57,7 @@
 /qemu-ga
 /qemu-bridge-helper
 /qemu-keymap
+/virtiofsd
 /qemu-monitor.texi
 /qemu-monitor-info.texi
 /qemu-storage-daemon
-- 
2.17.1






Re: [PATCH v4 4/4] iotests: Test node/bitmap aliases during migration

2020-08-21 Thread Max Reitz
On 20.08.20 17:49, Vladimir Sementsov-Ogievskiy wrote:
> # MYPYPATH=../../python/ mypy 300
> 300:154: error: Item "None" of "Optional[Match[Any]]" has no attribute
> "group"
> Found 1 error in 1 file (checked 1 source file)

:(

> - the only complain. Suggest a fix:
> 
> diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300
> index c6d86b1dbc..0241903743 100755
> --- a/tests/qemu-iotests/300
> +++ b/tests/qemu-iotests/300
> @@ -148,11 +148,11 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
>  result = vm.qmp('human-monitor-command',
>  command_line='info migrate_parameters')
>  
> -    hmp_mapping = re.search(r'^block-bitmap-mapping:\r?(\n 
> .*)*\n',
> -    result['return'], flags=re.MULTILINE)
> +    m = re.search(r'^block-bitmap-mapping:\r?(\n  .*)*\n',
> +  result['return'], flags=re.MULTILINE)
> +    hmp_mapping = m.group(0).replace('\r', '') if m else None
>  
> -    self.assertEqual(hmp_mapping.group(0).replace('\r', ''),
> - self.to_hmp_mapping(mapping))
> +    self.assertEqual(hmp_mapping, self.to_hmp_mapping(mapping))

Looks good, thanks.

>  else:
>  self.assert_qmp(result, 'error/desc', error)
>  
> ===
> 
> # flake8 300
[...]
> - I post it just because ALE plugin in vim highlights all these things
> for me. Up to you, I don't ask you to fix it.

Thanks, I’ll run it from now on (unless I forget, like mypy above...).

> 18.08.2020 16:32, Max Reitz wrote:
>> Signed-off-by: Max Reitz 
>> ---
>>   tests/qemu-iotests/300 | 595 +
>>   tests/qemu-iotests/300.out |   5 +
>>   tests/qemu-iotests/group   |   1 +
>>   3 files changed, 601 insertions(+)
>>   create mode 100755 tests/qemu-iotests/300
>>   create mode 100644 tests/qemu-iotests/300.out
>>
> 
> [..]
> 
>> +    def test_alias_on_both_migration(self) -> None:
>> +    src_map = self.mapping(self.src_node_name, 'node-alias',
>> +   self.src_bmap_name, 'bmap-alias')
>> +
>> +    dst_map = self.mapping(self.dst_node_name, 'node-alias',
>> +   self.dst_bmap_name, 'bmap-alias')
>> +
>> +    self.set_mapping(self.vm_a, src_map)
>> +    self.set_mapping(self.vm_b, dst_map)
>> +    self.migrate()
>> +    self.verify_dest_error(None)
> 
> Hmm, probably verify_dest_error() should be called from migrate(), as
> you call it (almost) always after migrate()

I don’t know, it shuts down the destination VM, so it would seem a bit
strange to do that as part of migrate().

>> +    def test_unused_mapping_on_dst(self) -> None:
>> +    # Let the source not send any bitmaps
>> +    self.set_mapping(self.vm_a, [])
>> +
>> +    # Establish some mapping on the destination
>> +    self.set_mapping(self.vm_b, [])
> 
> Seems, you wanted to specify non-empty mapping for vm_b, yes?

Oops.  I don’t know how it could have happened that I forgot that.

> With any non-empty mapping here, just to better correspond to the comments:
> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> 
> (or keep this case with both mappings empty, and add one similar with
> empty mapping only on src)

I’ll see in what shape this patch lands in master, and then I’ll
probably add that other case, yes.

Max



signature.asc
Description: OpenPGP digital signature


Re: [PATCH] hw: cirrus_vga: mask 'off_begin' in cirrus_invalidate_region()

2020-08-21 Thread Gerd Hoffmann
> This patch fixes this.

> --- a/hw/display/cirrus_vga.c
> +++ b/hw/display/cirrus_vga.c
> @@ -644,7 +644,7 @@ static void cirrus_invalidate_region(CirrusVGAState * s, 
> int off_begin,
>  off_cur_end = ((off_cur + bytesperline - 1) & s->cirrus_addr_mask) + 
> 1;
>  assert(off_cur_end >= off_cur);
>  memory_region_set_dirty(&s->vga.vram, off_cur, off_cur_end - 
> off_cur);
> -off_begin += off_pitch;
> +off_begin = (off_begin + off_pitch) & s->cirrus_addr_mask;
>  }

Well.  Only in case the wraparound doesn't split a scanline ...

take care,
  Gerd




[PATCH] cirrus: handle wraparound in cirrus_invalidate_region

2020-08-21 Thread Gerd Hoffmann
Code simply asserts that there is no wraparound instead of handling
it properly.  The assert() can be triggered by the guest (must be
privilidged inside the guest though).  Fix it.

Buglink: https://bugs.launchpad.net/qemu/+bug/1880189
Cc: Li Qiang 
Reported-by: Alexander Bulekov 
Signed-off-by: Gerd Hoffmann 
---
 hw/display/cirrus_vga.c | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/hw/display/cirrus_vga.c b/hw/display/cirrus_vga.c
index 212d6f5e6145..b91b64347473 100644
--- a/hw/display/cirrus_vga.c
+++ b/hw/display/cirrus_vga.c
@@ -640,10 +640,15 @@ static void cirrus_invalidate_region(CirrusVGAState * s, 
int off_begin,
 }
 
 for (y = 0; y < lines; y++) {
-off_cur = off_begin;
+off_cur = off_begin & s->cirrus_addr_mask;
 off_cur_end = ((off_cur + bytesperline - 1) & s->cirrus_addr_mask) + 1;
-assert(off_cur_end >= off_cur);
-memory_region_set_dirty(&s->vga.vram, off_cur, off_cur_end - off_cur);
+if (off_cur_end >= off_cur) {
+memory_region_set_dirty(&s->vga.vram, off_cur, off_cur_end - 
off_cur);
+} else {
+/* wraparound */
+memory_region_set_dirty(&s->vga.vram, off_cur, s->cirrus_addr_mask 
- off_cur);
+memory_region_set_dirty(&s->vga.vram, 0, off_cur_end);
+}
 off_begin += off_pitch;
 }
 }
-- 
2.27.0




Re: [PATCH v2] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Daniel P . Berrangé
On Thu, Aug 20, 2020 at 04:15:59PM -0500, Eric Blake wrote:
> On 8/20/20 12:31 PM, Daniel P. Berrangé wrote:
> > Meson requires the build dir to be separate from the source tree. Many
> > people are used to just running "./configure && make" though and the
> > meson conversion breaks that.
> > 
> > This introduces some backcompat support to make it appear as if an
> > "in source tree" build is being done, but with the the results in the
> > "build/" directory. This allows "./configure && make" to work as it
> > did historically, albeit with the output binaries staying under build/.
> 
> An observation:
> 
> If you already had out-of-tree builds, this does not change anything. You
> can do an incremental build, where a tree that builds pre-merge should
> continue to build incrementally with 'make -C dir' post-merge.
> 
> If you are used to in-tree builds, and do a fresh checkout, this lets you
> maintain the illusion of an in-tree build although binaries may be located
> differently than you are used to.
> 
> But if you do an incremental update to an in-tree build, this will cause
> some odd behaviors, starting with the git update:

snip.

> So I'm not sure why the first build gets as far as it does, but does NOT
> complete things and yet does not fail make, but my advice is that you should
> NOT try to an incremental build on in-tree build when crossing the meson
> epoch.  If you are a fan of in-tree convenience, you need a ONE-TIME
> distclean when pulling in these changes (the fact that you HAVE to clean up
> trace.h files to merge in the meson stuff should be a hint for that).  After
> that has been done, you can go back to pretending meson supports in-tree.

Yeah, I think that's something we'll have to live with. The amount of
extra complexity that would be required to try to make the incremental
build work across the meson introduction would be quite large. A more
efficient use of time is to just clean the dir manually, rather than
debugging some makefile rules that will only be needed once.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|




[PATCH] hw/arm/sbsa-ref: fix typo breaking PCIe IRQs

2020-08-21 Thread Graeme Gregory
Fixing a typo in a previous patch that translated an "i" to a 1
and therefore breaking the allocation of PCIe interrupts. This was
discovered when virtio-net-pci devices ceased to function correctly.

Fixes: 48ba18e6d3f3 ("hw/arm/sbsa-ref: Simplify by moving the gic in the 
machine state")
Signed-off-by: Graeme Gregory 
---
 hw/arm/sbsa-ref.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
index 570faac6e2..48c7cea291 100644
--- a/hw/arm/sbsa-ref.c
+++ b/hw/arm/sbsa-ref.c
@@ -565,7 +565,7 @@ static void create_pcie(SBSAMachineState *sms)
 
 for (i = 0; i < GPEX_NUM_IRQS; i++) {
 sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
-   qdev_get_gpio_in(sms->gic, irq + 1));
+   qdev_get_gpio_in(sms->gic, irq + i));
 gpex_set_irq_num(GPEX_HOST(dev), i, irq + i);
 }
 
-- 
2.25.1




[PATCH] virtio-gpu: fix unmap the already mapped items

2020-08-21 Thread Li Zhijian
we go here either (!(*iov)[i].iov_base) or (len != l), so we need to consider
to unmap the 'i'th item as well when the 'i'th item is not nil

Signed-off-by: Li Zhijian 
---
 hw/display/virtio-gpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 5f0dd7c150..1f777e43ff 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -656,7 +656,7 @@ int virtio_gpu_create_mapping_iov(VirtIOGPU *g,
 qemu_log_mask(LOG_GUEST_ERROR, "%s: failed to map MMIO memory for"
   " resource %d element %d\n",
   __func__, ab->resource_id, i);
-virtio_gpu_cleanup_mapping_iov(g, *iov, i);
+virtio_gpu_cleanup_mapping_iov(g, *iov, i + !!(*iov)[i].iov_base);
 g_free(ents);
 *iov = NULL;
 if (addr) {
-- 
2.17.1






Re: [PATCH 2/2] libvhost-user: handle endianness as mandated by the spec

2020-08-21 Thread Marc Hartmayer
On Mon, Aug 03, 2020 at 11:26 AM +0200, Cornelia Huck  wrote:
> On Thu, 30 Jul 2020 16:07:31 +0200
> Marc Hartmayer  wrote:
>
>> Since virtio existed even before it got standardized, the virtio
>> standard defines the following types of virtio devices:
>> 
>>  + legacy device (pre-virtio 1.0)
>>  + non-legacy or VIRTIO 1.0 device
>>  + transitional device (which can act both as legacy and non-legacy)
>> 
>> Virtio 1.0 defines the fields of the virtqueues as little endian,
>> while legacy uses guest's native endian [1]. Currently libvhost-user
>> does not handle virtio endianness at all, i.e. it works only if the
>> native endianness matches with whatever is actually needed. That means
>> things break spectacularly on big-endian targets. Let us handle virtio
>> endianness for non-legacy as required by the virtio specification
>> [1]. 
>
> Maybe add
>
> "and fence legacy virtio, as there is no safe way to figure out the
> needed endianness conversions for all cases."

Okay.

>
>> The fencing of legacy virtio devices is done in
>> `vu_set_features_exec`.
>
> Not that I disagree with fencing legacy virtio, but looking at some
> vhost-user* drivers, I'm not sure everything will work as desired for
> those (I might be missing something, though.)
>
> - vhost-user-blk lists VERSION_1 in the supported features, but
>   vhost-user-scsi doesn't... is there some inheritance going on that
>   I'm missing?
> - vhost-user-gpu-pci inherits from virtio-gpu-pci, so I guess it's fine
> - vhost-user-input should also always have been virtio-1
>
> So, has anybody been using vhost-user-scsi and can confirm that it
> still works, or at least can be made to work?

Unfortunately, I don’t have the required hardware :/ Can please anybody
verify this?

>
>> 
>> [1] 
>> https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/virtio-v1.1-cs01.html#x1-210003
>> 
>> Signed-off-by: Marc Hartmayer 
>> ---
>>  contrib/libvhost-user/libvhost-user.c | 77 +++
>>  1 file changed, 43 insertions(+), 34 deletions(-)
>
> The code change per se LGTM.

Thanks for the feedback!

>
-- 
Kind regards / Beste Grüße
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen 
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294



Re: [PATCH 02/10] numa: introduce MachineClass::forbid_asymmetrical_numa

2020-08-21 Thread Igor Mammedov
On Thu, 20 Aug 2020 12:51:03 -0400
Eduardo Habkost  wrote:

> On Thu, Aug 20, 2020 at 02:15:04PM +1000, David Gibson wrote:
> > On Wed, Aug 19, 2020 at 10:11:28PM -0400, Eduardo Habkost wrote:  
> > > On Thu, Aug 20, 2020 at 11:17:26AM +1000, David Gibson wrote:  
> > > > On Fri, Aug 14, 2020 at 05:54:16PM -0300, Daniel Henrique Barboza 
> > > > wrote:  
> > > > > The pSeries machine does not support asymmetrical NUMA
> > > > > configurations.  
> > > > 
> > > > This seems a bit oddly specific to have as a global machine class
> > > > property.
> > > > 
> > > > Would it make more sense for machines with specific NUMA constraints
> > > > to just verify those during their initialization?  
> > > 
> > > This would be much simpler.  However, I like the idea of
> > > representing machine-specific configuration validation rules as
> > > data that can eventually be exported to management software.  
> > 
> > Ah, ok, so basically the usual tradeoff between flexibility and
> > advertisability.
> > 
> > So, in that case, I guess the question is whether we envisage "no
> > assymmetry" as a constraint common enough that it's worth creating an
> > advertisable rule or not.  If we only ever have one user, then we
> > haven't really done any better than hard coding the constraint in the
> > manageent software.
> > 
> > Of course to complicate matters, in the longer term we're looking at
> > removing that constraint from pseries - but doing so will be dependent
> > on the guest kernel understanding a new format for the NUMA
> > information in the device tree.  So qemu alone won't have enough
> > information to tell if such a configuration is possible or not.  
> 
> Requiring both QEMU (and possibly management software) to be
> patched again after the guest kernel is fixed sounds undesirable.
If we drop this restriction, then we don't need to touch QEMU when
guest kernel is ready.

Btw, what spapr spec says about the matter?

> Perhaps a warning would be better in this case?
> 
> In either case, it sounds like this won't be a common constraint
> and I now agree with your original suggestion of doing this in
> machine initialization code.
Agreed, if it goes to spapr specific machine code I will not object much.
(it will burden just spapr maintainers, so it's about convincing
David in the end)




Re: [PATCH] hw/arm/sbsa-ref: fix typo breaking PCIe IRQs

2020-08-21 Thread no-reply
Patchew URL: 
https://patchew.org/QEMU/20200821083853.356490-1-gra...@nuviainc.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing 
commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  TESTcheck-unit: tests/test-char
Unexpected error in object_property_try_add() at 
/tmp/qemu-test/src/qom/object.c:1181:
attempt to add duplicate property 'serial-id' to object (type 'container')
ERROR test-char - too few tests run (expected 38, got 9)
make: *** [check-unit] Error 1
make: *** Waiting for unfinished jobs
  TESTiotest-qcow2: 029
  TESTcheck-qtest-x86_64: tests/qtest/hd-geo-test
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', 
'--label', 'com.qemu.instance.uuid=d020541d82784cb2ac804994220ba4ec', '-u', 
'1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', 
'-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 
'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', 
'/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', 
'/var/tmp/patchew-tester-tmp-q4fd6ckp/src/docker-src.2020-08-21-04.52.29.29067:/var/tmp/qemu:z,ro',
 'qemu/centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit 
status 2.
filter=--filter=label=com.qemu.instance.uuid=d020541d82784cb2ac804994220ba4ec
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-q4fd6ckp/src'
make: *** [docker-run-test-quick@centos7] Error 2

real15m58.687s
user0m9.025s


The full log is available at
http://patchew.org/logs/20200821083853.356490-1-gra...@nuviainc.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [PATCH V6] Introduce a new flag for i440fx to disable PCI hotplug on the root bus

2020-08-21 Thread Igor Mammedov
On Thu, 20 Aug 2020 22:11:41 +0530
Ani Sinha  wrote:

> > On Aug 20, 2020, at 9:11 PM, Ani Sinha  wrote:
> > 
> > On Thu, Aug 20, 2020 at 7:37 PM Igor Mammedov  wrote: 
> >  
> >>   
> >>> On Thu, 20 Aug 2020 14:51:56 +0530
> >>> Ani Sinha  wrote:
> >>> 
> >>> We introduce a new global flag 'acpi-root-pci-hotplug' for i440fx with 
> >>> which
> >>> we can turn on or off PCI device hotplug on the root bus. This flag can be
> >>> used to prevent all PCI devices from getting hotplugged or unplugged from 
> >>> the
> >>> root PCI bus.
> >>> This feature is targetted mostly towards Windows VMs. It is useful in 
> >>> cases
> >>> where some hypervisor admins want to deploy guest VMs in a way so that the
> >>> users of the guest OSes are not able to hot-eject certain PCI devices from
> >>> the Windows system tray. Laine has explained the use case here in detail:
> >>> https://www.redhat.com/archives/libvir-list/2020-February/msg00110.html
> >>> 
> >>> Julia has resolved this issue for PCIE buses with the following commit:
> >>> 530a0963184e57e71a5b538 ("pcie_root_port: Add hotplug disabling option")
> >>> 
> >>> This commit attempts to introduce similar behavior for PCI root buses 
> >>> used in
> >>> i440fx machine types (although in this case, we do not have a per-slot
> >>> capability to turn hotplug on or off).
> >>> 
> >>> Usage:
> >>>   -global PIIX4_PM.acpi-root-pci-hotplug=off
> >>> 
> >>> By default, this option is enabled which means that hotplug is turned on 
> >>> for
> >>> the PCI root bus.
> >>> 
> >>> The previously existing flag 'acpi-pci-hotplug-with-bridge-support' for 
> >>> PCI-PCI
> >>> bridges remain as is and can be used along with this new flag to control 
> >>> PCI
> >>> hotplug on PCI bridges.
> >>> 
> >>> This change has been tested using a Windows 2012R2 server guest image and 
> >>> also
> >>> with a Windows 2019 server guest image on a Ubuntu 18.04 host using the 
> >>> latest
> >>> master qemu from upstream (v5.1.0 tag).
> >>> 
> >>> Signed-off-by: Ani Sinha 
> >>> ---
> >>> hw/acpi/piix4.c  |  8 ++--
> >>> hw/i386/acpi-build.c | 26 +++---
> >>> 2 files changed, 25 insertions(+), 9 deletions(-)
> >>> 
> >>> Change Log:
> >>> V5..V6: specified upstream master tag information off which this patch is
> >>> based off of.
> >>> 
> >>> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> >>> index 26bac4f16c..4f436e5bf3 100644
> >>> --- a/hw/acpi/piix4.c
> >>> +++ b/hw/acpi/piix4.c
> >>> @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> >>> 
> >>> AcpiPciHpState acpi_pci_hotplug;
> >>> bool use_acpi_hotplug_bridge;
> >>> +bool use_acpi_root_pci_hotplug;
> >>> 
> >>> uint8_t disable_s3;
> >>> uint8_t disable_s4;  
> >>   
> >>> @@ -595,8 +596,9 @@ static void 
> >>> piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> >>>   "acpi-gpe0", GPE_LEN);
> >>> memory_region_add_subregion(parent, GPE_BASE, &s->io_gpe);
> >>> 
> >>> -acpi_pcihp_init(OBJECT(s), &s->acpi_pci_hotplug, bus, parent,
> >>> -s->use_acpi_hotplug_bridge);
> >>> +if (s->use_acpi_hotplug_bridge || s->use_acpi_root_pci_hotplug)
> >>> +acpi_pcihp_init(OBJECT(s), &s->acpi_pci_hotplug, bus, parent,
> >>> +s->use_acpi_hotplug_bridge);  
> >> If intent was to disable hardware part of ACPI hotplug,
> >> then this hunk is not enough. I'd say it introduces bug since you are 
> >> leaving
> >> device_add/del route open and "_E01" AML code around trying to access no 
> >> longer
> >> described/present io ports.
> >> 
> >> Without this hunk patch is fine, as a means to hide hotplug from Windows.
> >> 
> >> If you'd like to disable hw part, you will need to consider case where 
> >> hotplug is
> >> disabled completly and block all related AML and block device_add|del.
> >> So it would be a bit more than above hunk.  
> > 
> > Ok maybe I will just remove it.
> >   
> >> 
> >>   
> >>> s->cpu_hotplug_legacy = true;
> >>> object_property_add_bool(OBJECT(s), "cpu-hotplug-legacy",
> >>> @@ -635,6 +637,8 @@ static Property piix4_pm_properties[] = {
> >>> DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> >>> DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> >>>  use_acpi_hotplug_bridge, true),
> >>> +DEFINE_PROP_BOOL("acpi-root-pci-hotplug", PIIX4PMState,
> >>> + use_acpi_root_pci_hotplug, true),
> >>> DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> >>>  acpi_memory_hotplug.is_enabled, true),
> >>> DEFINE_PROP_END_OF_LIST(),
> >>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> >>> index b7bc2a..19a1702ad1 100644
> >>> --- a/hw/i386/acpi-build.c
> >>> +++ b/hw/i386/acpi-build.c
> >>> @@ -95,6 +95,7 @@ typedef struct AcpiPmInfo {
> >>> bool s3_disabled;
> >>> bool s4_disabled;
> >>> bool pcihp_bridge_en;
> >>> +bool pcihp_root_en;
> >>> uint8_t

Re: [PATCH] configure: add support for psuedo-"in source tree" builds

2020-08-21 Thread Daniel P . Berrangé
On Thu, Aug 20, 2020 at 12:42:20PM -0500, Eric Blake wrote:
> On 8/20/20 11:55 AM, Daniel P. Berrangé wrote:
> > Meson requires the build dir to be separate from the source tree. Many
> > people are used to just running "./configure && make" though and the
> > meson conversion breaks that.
> > 
> > This introduces some backcompat support to make it appear as if an
> > "in source tree" build is being done, but with the the results in the
> > "build/" directory. This allows "./configure && make" to work as it
> > did historically, albeit with the output binaries staying under build/.
> > 
> > Signed-off-by: Daniel P. Berrangé 
> > ---
> 
> In addition to reviews you already have,
> 
> 
> > I've not tested it beyond that. Note it blows away the "build/"
> > dir each time ./configure is run so it is pristine each time.
> 
> I definitely like the idea of only blowing away what we created - but if we
> created build, then recreating it for each new configure run is nice.
> 
> > 
> > We could optionally symlink binaries from build/ into $PWD
> > if poeople think that is important, eg by changing GNUmakefile
> > to have:
> > 
> > recurse: all
> > for bin in `find build -maxdepth 1 -type f -executable | grep -v -E 
> > '(ninjatool|config.status)'`; \
> 
> Using -maxdepth gets rid of the need to pre-create empty directories for
> nested binaries, but also loses out on binaries such as
> x86_64-softmmu/qemu-system-x86_64.  Oh, it looks like meson creates
> qemu-system-x86_64 as a binary in the top level, then a symlink in its old
> location.  Populating symlinks to ALL old locations is thus trickier than
> what you are proposing here, so it is fine to save that for a followup patch
> (let's get the bare minimum in first, so that at least ./configure && make
> works, before we worry about back-compat symlinks).
> 
> > 
> > This goes on top of Paolo's most recent meson port v175 posting,
> > or whatever number it is upto now :-)
> 
> Nice comment for reviewers, but doesn't quite need to be preserved in git.
> 
> > 
> >   .gitignore |  2 ++
> >   configure  | 40 
> >   2 files changed, 34 insertions(+), 8 deletions(-)
> > 
> > diff --git a/.gitignore b/.gitignore
> > index 92311284ef..4ccb9ed975 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -1,3 +1,5 @@
> > +/GNUmakefile
> > +/build/
> >   /.doctrees
> >   /config-devices.*
> >   /config-all-devices.*
> > diff --git a/configure b/configure
> > index cc5f58f31a..a5c88ad1ac 100755
> > --- a/configure
> > +++ b/configure
> > @@ -11,6 +11,38 @@ unset CLICOLOR_FORCE GREP_OPTIONS
> >   # Don't allow CCACHE, if present, to use cached results of compile tests!
> >   export CCACHE_RECACHE=yes
> > +source_path=$(cd "$(dirname -- "$0")"; pwd)
> 
> This behaves wrong if CDPATH is set in the environment.  We should really
> include CDPATH in our environment sanitization at the top of the file.

This code is pre-existing, so any fixes wrt CDPATH can be done by
a different patch.

> 
> > +
> > +if printf %s\\n "$source_path" "$PWD" | grep -q "[[:space:]:]";
> > +then
> > +  error_exit "main directory cannot contain spaces nor colons"
> > +fi
> > +
> > +if test "$PWD" == "$source_path"
> 
> bashism; s/==/=/ or you will break configure on dash systems
> 
> > +then
> > +echo "Using './build' as the directory for build output"
> 
> Do we want a way for a user to type './configure builddir=build/' and 'make
> builddir=build/' so they can specify different builddir overrides per
> invocation (of course, where builddir defaults to 'build/' if not
> specified)?  But hardcoding to _just_ ./build/ for getting this patch in
> quickly is fine.

Adding "builddir=build" as an option would be introducing new
functionality that doesn't already exist. The goal is just to
do some minimal work to preserve existing functionality. If
people want control over the name of their build dir, then
they can do that by using a out-of-source tree build already.

> 
> > +rm -rf build
> > +mkdir -p build
> > +cat > GNUmakefile < 
> If you use 'EOF' or \EOF here, then...
> 
> > +
> > +ifeq (\$(MAKECMDGOALS),)
> 
> you wouldn't have to escape all these $.  Looking through the file...

Ah, subtle, but nice.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|




Re: [PATCH] configure: add support for psuedo-"in source tree" builds

2020-08-21 Thread Daniel P . Berrangé
On Thu, Aug 20, 2020 at 07:15:03PM +0100, Peter Maydell wrote:
> On Thu, 20 Aug 2020 at 17:56, Daniel P. Berrangé  wrote:
> >
> > Meson requires the build dir to be separate from the source tree. Many
> > people are used to just running "./configure && make" though and the
> > meson conversion breaks that.
> >
> > This introduces some backcompat support to make it appear as if an
> > "in source tree" build is being done, but with the the results in the
> > "build/" directory. This allows "./configure && make" to work as it
> > did historically, albeit with the output binaries staying under build/.
> >
> > Signed-off-by: Daniel P. Berrangé 
> > ---
> 
> Oh, one more minor thing: 'make distclean' should remove
> the created GNUMakefile (and also build/ if configure
> created it? dunno).

Yes, I mistakenly put that logic for "make clean" in this v1. My v2
has fixed this, and in v3 i'll remove the "GNUMakefile" too.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|




Re: Suspicious QOM types without instance/class size

2020-08-21 Thread Peter Maydell
On Thu, 20 Aug 2020 at 22:55, Eduardo Habkost  wrote:
>
> While trying to convert TypeInfo declarations to the new
> OBJECT_DECLARE* macros, I've stumbled on a few suspicious cases
> where instance_size or class_size is not set, despite having type
> checker macros that use a specific type.
>
> The ones with "WARNING" are abstract types (maybe not serious if
> subclasses set the appropriate sizes).  The ones with "ERROR"
> don't seem to be abstract types.
>
> WARNING: hw/arm/armsse.c:1159:1: class_size should be set to 
> sizeof(ARMSSEClass)?

Seems like a bug, yes. Here the subclasses are simple leaf classes
which (correctly) don't set class_size, so the abstract parent
class must set class_size.

> ERROR: hw/core/register.c:328:1: instance_size should be set to 
> sizeof(RegisterInfo)?

Bug (we haven't noticed because the only thing that creates them
is register_info() and it does object_initialize() into an existing
struct rather than trying to object_new() it).

thanks
-- PMM



Re: [PATCH v5 01/15] block/nvme: Replace magic value by SCALE_MS definition

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:47PM +0200, Philippe Mathieu-Daudé wrote:
> Use self-explicit SCALE_MS definition instead of magic value.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 374e2689157..2f5e3c2adfa 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -715,7 +715,7 @@ static int nvme_init(BlockDriverState *bs, const char 
> *device, int namespace,
>  /* Reset device to get a clean state. */
>  s->regs->cc = cpu_to_le32(le32_to_cpu(s->regs->cc) & 0xFE);
>  /* Wait for CSTS.RDY = 0. */
> -deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * 
> 100ULL;
> +deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * 
> SCALE_MS;
>  while (le32_to_cpu(s->regs->csts) & 0x1) {
>  if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
>  error_setg(errp, "Timeout while waiting for device to reset (%"
> -- 
> 2.26.2
> 
> 




Re: Suspicious QOM types without instance/class size

2020-08-21 Thread David Hildenbrand
On 20.08.20 23:55, Eduardo Habkost wrote:
> While trying to convert TypeInfo declarations to the new
> OBJECT_DECLARE* macros, I've stumbled on a few suspicious cases
> where instance_size or class_size is not set, despite having type
> checker macros that use a specific type.
> 
> The ones with "WARNING" are abstract types (maybe not serious if
> subclasses set the appropriate sizes).  The ones with "ERROR"
> don't seem to be abstract types.
> 
> WARNING: hw/arm/armsse.c:1159:1: class_size should be set to 
> sizeof(ARMSSEClass)?
> WARNING: hw/audio/hda-codec.c:900:1: instance_size should be set to 
> sizeof(HDAAudioState)?
> ERROR: hw/core/register.c:328:1: instance_size should be set to 
> sizeof(RegisterInfo)?
> WARNING: hw/input/adb.c:310:1: class_size should be set to 
> sizeof(ADBDeviceClass)?
> WARNING: hw/isa/isa-superio.c:181:1: instance_size should be set to 
> sizeof(ISASuperIODevice)?
> WARNING: hw/ppc/pnv_lpc.c:771:1: instance_size should be set to 
> sizeof(PnvLpcController)?
> ERROR: hw/ppc/spapr_drc.c:771:1: instance_size should be set to 
> sizeof(SpaprDrc)?
> WARNING: hw/rtc/m48t59-isa.c:156:1: class_size should be set to 
> sizeof(M48txxISADeviceClass)?
> WARNING: hw/rtc/m48t59.c:691:1: class_size should be set to 
> sizeof(M48txxSysBusDeviceClass)?
> ERROR: hw/s390x/virtio-ccw.c:1237:1: class_size should be set to 
> sizeof(VirtioCcwBusClass)?

The parent of TYPE_VIRTIO_CCW_BUS is TYPE_VIRTIO_BUS.

typedef struct VirtioBusClass VirtioCcwBusClass;

So I guess the sizes match? Anyhow, setting doesn't hurt.

-- 
Thanks,

David / dhildenb




Re: Suspicious QOM types without instance/class size

2020-08-21 Thread Cornelia Huck
On Thu, 20 Aug 2020 17:55:29 -0400
Eduardo Habkost  wrote:

> While trying to convert TypeInfo declarations to the new
> OBJECT_DECLARE* macros, I've stumbled on a few suspicious cases
> where instance_size or class_size is not set, despite having type
> checker macros that use a specific type.
> 
> The ones with "WARNING" are abstract types (maybe not serious if
> subclasses set the appropriate sizes).  The ones with "ERROR"
> don't seem to be abstract types.
> 
> ERROR: hw/s390x/virtio-ccw.c:1237:1: class_size should be set to 
> sizeof(VirtioCcwBusClass)?
> ERROR: hw/virtio/virtio-pci.c:2101:1: class_size should be set to 
> sizeof(VirtioPCIBusClass)?

VirtioCcwBusClass and VirtioPCIBusClass are both simple typedefs of
VirtioBusClass (it's likely that I copied the ccw definition from the
pci one). virtio-mmio instead uses VirtioBusClass directly in its
checker macros.

I don't see a real reason for the typedefs, maybe ccw and pci should
use the mmio approach as well?




Re: [PATCH v5 03/15] block/nvme: Let nvme_create_queue_pair() fail gracefully

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:49PM +0200, Philippe Mathieu-Daudé wrote:
> As nvme_create_queue_pair() is allowed to fail, replace the
> alloc() calls by try_alloc() to avoid aborting QEMU.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 12 ++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 8c30a5fee28..e1893b4e792 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -213,14 +213,22 @@ static NVMeQueuePair 
> *nvme_create_queue_pair(BlockDriverState *bs,
>  int i, r;
>  BDRVNVMeState *s = bs->opaque;
>  Error *local_err = NULL;
> -NVMeQueuePair *q = g_new0(NVMeQueuePair, 1);
> +NVMeQueuePair *q;
>  uint64_t prp_list_iova;
>  
> +q = g_try_new0(NVMeQueuePair, 1);
> +if (!q) {
> +return NULL;
> +}
> +q->prp_list_pages = qemu_try_blockalign0(bs,
> +  s->page_size * NVME_QUEUE_SIZE);

Here you use NVME_QUEUE_SIZE instead of NVME_NUM_REQS, is that an
intentional change?

Maybe is not an issue, sice NVME_QUEUE_SIZE is bigger than
NVME_NUM_REQS, but we should mention in the commit message.

Thanks,
Stefano

> +if (!q->prp_list_pages) {
> +goto fail;
> +}
>  qemu_mutex_init(&q->lock);
>  q->s = s;
>  q->index = idx;
>  qemu_co_queue_init(&q->free_req_queue);
> -q->prp_list_pages = qemu_blockalign0(bs, s->page_size * NVME_NUM_REQS);
>  q->completion_bh = aio_bh_new(bdrv_get_aio_context(bs),
>nvme_process_completion_bh, q);
>  r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,
> -- 
> 2.26.2
> 
> 




Re: [PATCH v5 04/15] block/nvme: Define INDEX macros to ease code review

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:50PM +0200, Philippe Mathieu-Daudé wrote:
> Use definitions instead of '0' or '1' indexes. Also this will
> be useful when using multi-queues later.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 33 +++--
>  1 file changed, 19 insertions(+), 14 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index e1893b4e792..003809fbd83 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -103,6 +103,9 @@ typedef volatile struct {
>  
>  QEMU_BUILD_BUG_ON(offsetof(NVMeRegs, doorbells) != 0x1000);
>  
> +#define INDEX_ADMIN 0
> +#define INDEX_IO(n) (1 + n)
> +
>  struct BDRVNVMeState {
>  AioContext *aio_context;
>  QEMUVFIOState *vfio;
> @@ -531,7 +534,7 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  }
>  cmd.prp1 = cpu_to_le64(iova);
>  
> -if (nvme_cmd_sync(bs, s->queues[0], &cmd)) {
> +if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
>  error_setg(errp, "Failed to identify controller");
>  goto out;
>  }
> @@ -555,7 +558,7 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  
>  cmd.cdw10 = 0;
>  cmd.nsid = cpu_to_le32(namespace);
> -if (nvme_cmd_sync(bs, s->queues[0], &cmd)) {
> +if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
>  error_setg(errp, "Failed to identify namespace");
>  goto out;
>  }
> @@ -644,7 +647,7 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Error 
> **errp)
>  .cdw10 = cpu_to_le32(((queue_size - 1) << 16) | (n & 0x)),
>  .cdw11 = cpu_to_le32(0x3),
>  };
> -if (nvme_cmd_sync(bs, s->queues[0], &cmd)) {
> +if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
>  error_setg(errp, "Failed to create io queue [%d]", n);
>  nvme_free_queue_pair(q);
>  return false;
> @@ -655,7 +658,7 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Error 
> **errp)
>  .cdw10 = cpu_to_le32(((queue_size - 1) << 16) | (n & 0x)),
>  .cdw11 = cpu_to_le32(0x1 | (n << 16)),
>  };
> -if (nvme_cmd_sync(bs, s->queues[0], &cmd)) {
> +if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
>  error_setg(errp, "Failed to create io queue [%d]", n);
>  nvme_free_queue_pair(q);
>  return false;
> @@ -739,16 +742,18 @@ static int nvme_init(BlockDriverState *bs, const char 
> *device, int namespace,
>  
>  /* Set up admin queue. */
>  s->queues = g_new(NVMeQueuePair *, 1);
> -s->queues[0] = nvme_create_queue_pair(bs, 0, NVME_QUEUE_SIZE, errp);
> -if (!s->queues[0]) {
> +s->queues[INDEX_ADMIN] = nvme_create_queue_pair(bs, 0,
> +  NVME_QUEUE_SIZE,
> +  errp);
> +if (!s->queues[INDEX_ADMIN]) {
>  ret = -EINVAL;
>  goto out;
>  }
>  s->nr_queues = 1;
>  QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000);
>  s->regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
> -s->regs->asq = cpu_to_le64(s->queues[0]->sq.iova);
> -s->regs->acq = cpu_to_le64(s->queues[0]->cq.iova);
> +s->regs->asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
> +s->regs->acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
>  
>  /* After setting up all control registers we can enable device now. */
>  s->regs->cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
> @@ -839,7 +844,7 @@ static int 
> nvme_enable_disable_write_cache(BlockDriverState *bs, bool enable,
>  .cdw11 = cpu_to_le32(enable ? 0x01 : 0x00),
>  };
>  
> -ret = nvme_cmd_sync(bs, s->queues[0], &cmd);
> +ret = nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd);
>  if (ret) {
>  error_setg(errp, "Failed to configure NVMe write cache");
>  }
> @@ -1056,7 +1061,7 @@ static coroutine_fn int 
> nvme_co_prw_aligned(BlockDriverState *bs,
>  {
>  int r;
>  BDRVNVMeState *s = bs->opaque;
> -NVMeQueuePair *ioq = s->queues[1];
> +NVMeQueuePair *ioq = s->queues[INDEX_IO(0)];
>  NVMeRequest *req;
>  
>  uint32_t cdw12 = (((bytes >> s->blkshift) - 1) & 0x) |
> @@ -1171,7 +1176,7 @@ static coroutine_fn int 
> nvme_co_pwritev(BlockDriverState *bs,
>  static coroutine_fn int nvme_co_flush(BlockDriverState *bs)
>  {
>  BDRVNVMeState *s = bs->opaque;
> -NVMeQueuePair *ioq = s->queues[1];
> +NVMeQueuePair *ioq = s->queues[INDEX_IO(0)];
>  NVMeRequest *req;
>  NvmeCmd cmd = {
>  .opcode = NVME_CMD_FLUSH,
> @@ -1202,7 +1207,7 @@ static coroutine_fn int 
> nvme_co_pwrite_zeroes(BlockDriverState *bs,
>BdrvRequestFlags flags)
>  {
>  BDRVNVMeState *s = bs->opaque;
> -NVMeQueuePair *ioq = s->queues[1];
> +NVMeQueuePair *ioq = s->queues[

Re: [PATCH v5 05/15] block/nvme: Improve error message when IO queue creation failed

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:51PM +0200, Philippe Mathieu-Daudé wrote:
> Do not use the same error message for different failures.
> Display a different error whether it is the CQ or the SQ.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 003809fbd83..53448b7d230 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -648,7 +648,7 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Error 
> **errp)
>  .cdw11 = cpu_to_le32(0x3),
>  };
>  if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
> -error_setg(errp, "Failed to create io queue [%d]", n);
> +error_setg(errp, "Failed to create CQ io queue [%d]", n);
>  nvme_free_queue_pair(q);
>  return false;
>  }
> @@ -659,7 +659,7 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Error 
> **errp)
>  .cdw11 = cpu_to_le32(0x1 | (n << 16)),
>  };
>  if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
> -error_setg(errp, "Failed to create io queue [%d]", n);
> +error_setg(errp, "Failed to create SQ io queue [%d]", n);
>  nvme_free_queue_pair(q);
>  return false;
>  }
> -- 
> 2.26.2
> 
> 




Re: [PATCH v5 06/15] block/nvme: Use common error path in nvme_add_io_queue()

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:52PM +0200, Philippe Mathieu-Daudé wrote:
> Rearrange nvme_add_io_queue() by using a common error path.
> This will be proven useful in few commits where we add IRQ
> notification to the IO queues.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 9 +
>  1 file changed, 5 insertions(+), 4 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 53448b7d230..3101f1ad55d 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -649,8 +649,7 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Error 
> **errp)
>  };
>  if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
>  error_setg(errp, "Failed to create CQ io queue [%d]", n);
> -nvme_free_queue_pair(q);
> -return false;
> +goto out_error;
>  }
>  cmd = (NvmeCmd) {
>  .opcode = NVME_ADM_CMD_CREATE_SQ,
> @@ -660,13 +659,15 @@ static bool nvme_add_io_queue(BlockDriverState *bs, 
> Error **errp)
>  };
>  if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
>  error_setg(errp, "Failed to create SQ io queue [%d]", n);
> -nvme_free_queue_pair(q);
> -return false;
> +goto out_error;
>  }
>  s->queues = g_renew(NVMeQueuePair *, s->queues, n + 1);
>  s->queues[n] = q;
>  s->nr_queues++;
>  return true;
> +out_error:
> +nvme_free_queue_pair(q);
> +return false;
>  }
>  
>  static bool nvme_poll_cb(void *opaque)
> -- 
> 2.26.2
> 
> 




Re: deprecation of in-tree builds

2020-08-21 Thread Gerd Hoffmann
  Hi,

> If it's a matter of "I don't want my workflow to change too much", I'm
> not familiar with everybody's setups and requirements and it's not a
> precise-enough requirement for me to write the code (and block the
> inclusion of the pull request).

It's almost zero change to my workflow.  I only had to fix modular
builds (patch below).

I'm doing out-of-tree builds for ages.  First, because I consider it
good practice.  Second, because it allows me to easily test multiple
different configurations (modular vs. non-module, with/without spice,
...) without "make distclean; ./configure $otheropts", by simply having
multiple build trees.  Third, because I can just "rm -rf $builddir" and
start over in case I trap into one of the bugs our Makefiles have (not
rebuilding objects on cflags changes for example).

> What you're asking is not the final 0.1%; it's a completely different
> thing, because it _must_ be by definition half-baked.  For example here
> are all the tradeoffs that come to mind:

[ list snipped ]

> I'm rather happier to spend my time explaining details of the conversion
> if they have to hack on the build system in the near future, than to
> gather a detailed answer to these questions and any other that didn't
> come to mind.

Fully agree.

My personal take on this is we should just have configure catch in-tree
builds, print instructions for out-of-tree builds and throw an error.
Switching from in-tree to out-of-tree builds isn't that much of an issue
after all, and given that our current build system supports it too even
"git bisect" should be painless.

> (as long as you don't touch the build system of course).

And *this* should be easier once the series landed.
Also I'm looking forward to have less build system bugs.

take care,
  Gerd

---
diff --git a/util/module.c b/util/module.c
index c956ef096b3b..6e63006a8fb2 100644
--- a/util/module.c
+++ b/util/module.c
@@ -113,7 +113,7 @@ static int module_load_file(const char *fname)
 {
 GModule *g_module;
 void (*sym)(void);
-const char *dsosuf = HOST_DSOSUF;
+const char *dsosuf = CONFIG_HOST_DSOSUF;
 int len = strlen(fname);
 int suf_len = strlen(dsosuf);
 ModuleEntry *e, *next;
@@ -221,7 +221,7 @@ bool module_load_one(const char *prefix, const char 
*lib_name)
 
 for (i = 0; i < n_dirs; i++) {
 fname = g_strdup_printf("%s/%s%s",
-dirs[i], module_name, HOST_DSOSUF);
+dirs[i], module_name, CONFIG_HOST_DSOSUF);
 ret = module_load_file(fname);
 g_free(fname);
 fname = NULL;




Re: [PATCH v5 07/15] block/nvme: Rename local variable

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:53PM +0200, Philippe Mathieu-Daudé wrote:
> We are going to modify the code in the next commit. Renaming
> the 'resp' variable to 'id' first makes the next commit easier
> to review. No logical changes.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 19 +--
>  1 file changed, 9 insertions(+), 10 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 3101f1ad55d..99822d9fd36 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -510,8 +510,8 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  BDRVNVMeState *s = bs->opaque;
>  NvmeIdCtrl *idctrl;
>  NvmeIdNs *idns;
> +uint8_t *id;
>  NvmeLBAF *lbaf;
> -uint8_t *resp;
>  uint16_t oncs;
>  int r;
>  uint64_t iova;
> @@ -520,14 +520,14 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  .cdw10 = cpu_to_le32(0x1),
>  };
>  
> -resp = qemu_try_blockalign0(bs, sizeof(NvmeIdCtrl));
> -if (!resp) {
> +id = qemu_try_blockalign0(bs, sizeof(NvmeIdCtrl));
> +if (!id) {
>  error_setg(errp, "Cannot allocate buffer for identify response");
>  goto out;
>  }
> -idctrl = (NvmeIdCtrl *)resp;
> -idns = (NvmeIdNs *)resp;
> -r = qemu_vfio_dma_map(s->vfio, resp, sizeof(NvmeIdCtrl), true, &iova);
> +idctrl = (NvmeIdCtrl *)id;
> +idns = (NvmeIdNs *)id;
> +r = qemu_vfio_dma_map(s->vfio, id, sizeof(NvmeIdCtrl), true, &iova);
>  if (r) {
>  error_setg(errp, "Cannot map buffer for DMA");
>  goto out;
> @@ -554,8 +554,7 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  s->supports_write_zeroes = !!(oncs & NVME_ONCS_WRITE_ZEROS);
>  s->supports_discard = !!(oncs & NVME_ONCS_DSM);
>  
> -memset(resp, 0, 4096);
> -
> +memset(id, 0, 4096);
>  cmd.cdw10 = 0;
>  cmd.nsid = cpu_to_le32(namespace);
>  if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
> @@ -587,8 +586,8 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  
>  s->blkshift = lbaf->ds;
>  out:
> -qemu_vfio_dma_unmap(s->vfio, resp);
> -qemu_vfree(resp);
> +qemu_vfio_dma_unmap(s->vfio, id);
> +qemu_vfree(id);
>  }
>  
>  static bool nvme_poll_queues(BDRVNVMeState *s)
> -- 
> 2.26.2
> 
> 




Re: [PATCH] configure: add support for psuedo-"in source tree" builds

2020-08-21 Thread Kevin Wolf
Am 20.08.2020 um 19:42 hat Eric Blake geschrieben:
> On 8/20/20 11:55 AM, Daniel P. Berrangé wrote:
> > Meson requires the build dir to be separate from the source tree. Many
> > people are used to just running "./configure && make" though and the
> > meson conversion breaks that.
> > 
> > This introduces some backcompat support to make it appear as if an
> > "in source tree" build is being done, but with the the results in the
> > "build/" directory. This allows "./configure && make" to work as it
> > did historically, albeit with the output binaries staying under build/.
> > 
> > Signed-off-by: Daniel P. Berrangé 
> > ---
> 
> In addition to reviews you already have,
> 
> 
> > I've not tested it beyond that. Note it blows away the "build/"
> > dir each time ./configure is run so it is pristine each time.
> 
> I definitely like the idea of only blowing away what we created - but if we
> created build, then recreating it for each new configure run is nice.

I think I actually wouldn't automatically remove anything on configure.
It can be surprising behaviour for configure to delete directories, and
the old setup didn't do an automatic "make clean" either. By having a
separate build directory, manually emptying as needed has already become
easier.

> > We could optionally symlink binaries from build/ into $PWD
> > if poeople think that is important, eg by changing GNUmakefile
> > to have:
> > 
> > recurse: all
> > for bin in `find build -maxdepth 1 -type f -executable | grep -v -E 
> > '(ninjatool|config.status)'`; \
> 
> Using -maxdepth gets rid of the need to pre-create empty directories for
> nested binaries, but also loses out on binaries such as
> x86_64-softmmu/qemu-system-x86_64.  Oh, it looks like meson creates
> qemu-system-x86_64 as a binary in the top level, then a symlink in its old
> location.  Populating symlinks to ALL old locations is thus trickier than
> what you are proposing here, so it is fine to save that for a followup patch
> (let's get the bare minimum in first, so that at least ./configure && make
> works, before we worry about back-compat symlinks).

Having the system emulator symlinks in the top level would be a change,
but even more convenient than the original places. I'd vote for adding
the auto-symlinking at least for the tools; if the top-level symlinks
for system emulators get also symlinked by this, that's fine, too.

I was actually surprised that Dan reports "make check" from the source
tree to be working without the symlinks. Some code must be cleverer than
I thought!

Kevin




Re: [PATCH v3 03/10] migration/dirtyrate: Add dirtyrate statistics series functions

2020-08-21 Thread Zheng Chuan



On 2020/8/21 0:28, Dr. David Alan Gilbert wrote:
> * Chuan Zheng (zhengch...@huawei.com) wrote:
>> Add dirtyrate statistics to record/update dirtyrate info.
>>
>> Signed-off-by: Chuan Zheng 
>> Signed-off-by: YanYing Zhuang 
>> ---
>>  migration/dirtyrate.c | 30 ++
>>  migration/dirtyrate.h | 10 ++
>>  2 files changed, 40 insertions(+)
>>
>> diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
>> index bb0ebe9..8708090 100644
>> --- a/migration/dirtyrate.c
>> +++ b/migration/dirtyrate.c
>> @@ -24,6 +24,7 @@
>>  #include "dirtyrate.h"
>>  
>>  CalculatingDirtyRateState CalculatingState = CAL_DIRTY_RATE_INIT;
>> +static struct DirtyRateStat dirty_stat;
>>  
>>  static int dirty_rate_set_state(int new_state)
>>  {
>> @@ -40,6 +41,35 @@ static int dirty_rate_set_state(int new_state)
>>  return 0;
>>  }
>>  
>> +static void reset_dirtyrate_stat(void)
>> +{
>> +dirty_stat.total_dirty_samples = 0;
>> +dirty_stat.total_sample_count = 0;
>> +dirty_stat.total_block_mem_MB = 0;
>> +dirty_stat.dirty_rate = 0;
>> +}
>> +
>> +static void update_dirtyrate_stat(struct RamblockDirtyInfo *info)
>> +{
>> +dirty_stat.total_dirty_samples += info->sample_dirty_count;
>> +dirty_stat.total_sample_count += info->sample_pages_count;
>> +/* size of 4K pages in MB */
>> +dirty_stat.total_block_mem_MB += info->ramblock_pages / 256;
>> +}
>> +
>> +static void update_dirtyrate(uint64_t msec)
>> +{
>> +uint64_t dirty_rate;
>> +unsigned int total_dirty_samples = dirty_stat.total_dirty_samples;
>> +unsigned int total_sample_count = dirty_stat.total_sample_count;
>> +size_t total_block_mem_MB = dirty_stat.total_block_mem_MB;
>> +
>> +dirty_rate = total_dirty_samples * total_block_mem_MB *
>> + 1000 / (total_sample_count * msec);
>> +
>> +dirty_stat.dirty_rate = dirty_rate;
>> +}
>> +
>>  static void calculate_dirtyrate(struct DirtyRateConfig config)
>>  {
>>  /* todo */
>> diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
>> index 9650566..af57c80 100644
>> --- a/migration/dirtyrate.h
>> +++ b/migration/dirtyrate.h
>> @@ -57,6 +57,16 @@ struct RamblockDirtyInfo {
>>  uint8_t *hash_result; /* array of hash result for sampled pages */
>>  };
>>  
>> +/*
>> + * Store calculate statistics for each measure.
>> + */
>> +struct DirtyRateStat {
>> +unsigned int total_dirty_samples; /* total dirty pages for this measure 
>> */
>> +unsigned int total_sample_count; /* total sampled pages for this 
>> measure */
>> +size_t total_block_mem_MB; /* size of sampled pages in MB */
>> +int64_t dirty_rate; /* dirty rate for this measure */
> 
> As I said in the previous review, please comment 'dirty_rate' with it's
> units.
> 
Sorry, i missed that, will be fix in V4:)
> Dave
> 
>> +};
>> +
>>  void *get_dirtyrate_thread(void *arg);
>>  #endif
>>  
>> -- 
>> 1.8.3.1
>>




Re: [PATCH v3 09/10] migration/dirtyrate: Implement calculate_dirtyrate() function

2020-08-21 Thread Zheng Chuan



On 2020/8/21 1:57, Dr. David Alan Gilbert wrote:
> * Chuan Zheng (zhengch...@huawei.com) wrote:
>> Implement calculate_dirtyrate() function.
>>
>> Signed-off-by: Chuan Zheng 
>> Signed-off-by: YanYing Zhuang 
>> ---
>>  migration/dirtyrate.c | 46 --
>>  1 file changed, 44 insertions(+), 2 deletions(-)
>>
>> diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
>> index 4bbfcc3..041d0c6 100644
>> --- a/migration/dirtyrate.c
>> +++ b/migration/dirtyrate.c
>> @@ -184,6 +184,21 @@ static void get_ramblock_dirty_info(RAMBlock *block,
>>  strcpy(info->idstr, qemu_ram_get_idstr(block));
>>  }
>>  
>> +static void free_ramblock_dirty_info(struct RamblockDirtyInfo *infos, int 
>> count)
>> +{
>> +int i;
>> +
>> +if (!infos) {
>> +return;
>> +}
>> +
>> +for (i = 0; i < count; i++) {
>> +g_free(infos[i].sample_page_vfn);
>> +g_free(infos[i].hash_result);
>> +}
>> +g_free(infos);
>> +}
>> +
>>  static struct RamblockDirtyInfo *
>>  alloc_ramblock_dirty_info(int *block_index,
>>struct RamblockDirtyInfo *block_dinfo)
>> @@ -341,8 +356,35 @@ static int compare_page_hash_info(struct 
>> RamblockDirtyInfo *info,
>>  
>>  static void calculate_dirtyrate(struct DirtyRateConfig config)
>>  {
>> -/* todo */
>> -return;
>> +struct RamblockDirtyInfo *block_dinfo = NULL;
>> +int block_index = 0;
>> +int64_t msec = 0;
>> +int64_t initial_time;
>> +
>> +rcu_register_thread();
>> +reset_dirtyrate_stat();
>> +initial_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
>> +rcu_read_lock();
>> +if (record_ramblock_hash_info(&block_dinfo, config, &block_index) < 0) {
>> +goto out;
>> +}
>> +rcu_read_unlock();
>> +
>> +msec = config.sample_period_seconds * 1000;
>> +msec = set_sample_page_period(msec, initial_time);
>> +
>> +rcu_read_lock();
>> +if (compare_page_hash_info(block_dinfo, block_index) < 0) {
>> +goto out;
>> +}
>> +
>> +update_dirtyrate(msec);
> 
> I think this is OK, so:
> 
> Reviewed-by: Dr. David Alan Gilbert 
> 
> However, please try the following test,  set it to 60 seconds,
> start the dirty rate check, and in that time, shut the guest down
> (e.g. shutdown -h now in the guest) - what happens?
> 
> Dave
> 
It is ok when shutdown corcurrent with query dirtyrate, the get_dirtyrate
thread is terminated by qemu.

>> +
>> +out:
>> +rcu_read_unlock();
>> +free_ramblock_dirty_info(block_dinfo, block_index + 1);
>> +rcu_unregister_thread();
>> +
>>  }
>>  
>>  void *get_dirtyrate_thread(void *arg)
>> -- 
>> 1.8.3.1
>>




Re: [PATCH v3 01/10] migration/dirtyrate: Add get_dirtyrate_thread() function

2020-08-21 Thread Zheng Chuan



On 2020/8/21 0:11, Dr. David Alan Gilbert wrote:
> * Chuan Zheng (zhengch...@huawei.com) wrote:
>> Add get_dirtyrate_thread() functions
>>
>> Signed-off-by: Chuan Zheng 
>> Signed-off-by: YanYing Zhuang 
>> ---
>>  migration/Makefile.objs |  1 +
>>  migration/dirtyrate.c   | 64 
>> +
>>  migration/dirtyrate.h   | 44 ++
>>  3 files changed, 109 insertions(+)
>>  create mode 100644 migration/dirtyrate.c
>>  create mode 100644 migration/dirtyrate.h
>>
>> diff --git a/migration/Makefile.objs b/migration/Makefile.objs
>> index 0fc619e..12ae98c 100644
>> --- a/migration/Makefile.objs
>> +++ b/migration/Makefile.objs
>> @@ -6,6 +6,7 @@ common-obj-y += qemu-file.o global_state.o
>>  common-obj-y += qemu-file-channel.o
>>  common-obj-y += xbzrle.o postcopy-ram.o
>>  common-obj-y += qjson.o
>> +common-obj-y += dirtyrate.o
>>  common-obj-y += block-dirty-bitmap.o
>>  common-obj-y += multifd.o
>>  common-obj-y += multifd-zlib.o
>> diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
>> new file mode 100644
>> index 000..bb0ebe9
>> --- /dev/null
>> +++ b/migration/dirtyrate.c
>> @@ -0,0 +1,64 @@
>> +/*
>> + * Dirtyrate implement code
>> + *
>> + * Copyright (c) 2017-2020 HUAWEI TECHNOLOGIES CO.,LTD.
>> + *
>> + * Authors:
>> + *  Chuan Zheng 
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
>> + * See the COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qapi/error.h"
>> +#include "crypto/hash.h"
>> +#include "crypto/random.h"
>> +#include "qemu/config-file.h"
>> +#include "exec/memory.h"
>> +#include "exec/ramblock.h"
>> +#include "exec/target_page.h"
>> +#include "qemu/rcu_queue.h"
>> +#include "qapi/qapi-commands-migration.h"
>> +#include "migration.h"
>> +#include "dirtyrate.h"
>> +
>> +CalculatingDirtyRateState CalculatingState = CAL_DIRTY_RATE_INIT;
>> +
>> +static int dirty_rate_set_state(int new_state)
>> +{
>> +int old_state = CalculatingState;
>> +
>> +if (new_state == old_state) {
>> +return -1;
>> +}
>> +
>> +if (atomic_cmpxchg(&CalculatingState, old_state, new_state) != 
>> old_state) {
>> +return -1;
>> +}
> 
> This is a little unusual; this has removed your comment from v1 about
> what you're trying to protect; but not quite being clear about what it's
> doing.
> 
> I think what you want here is closer to migrate_set_state, ie you
> pass what you think the old state is, and the state you want to go to.
> 
Hi, Dave.
Thank you for your review.

Yes, what I want to do is to protect concurrent scene lockless, i'll rewrite
according to migrate_set_state().


>> +return 0;
>> +}
>> +
>> +static void calculate_dirtyrate(struct DirtyRateConfig config)
>> +{
>> +/* todo */
>> +return;
>> +}
>> +
>> +void *get_dirtyrate_thread(void *arg)
>> +{
>> +struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg;
>> +int ret;
>> +
>> +ret = dirty_rate_set_state(CAL_DIRTY_RATE_ACTIVE);
> 
> so this would become:
> ret = dirty_rate_set_state(CAL_DIRTY_RATE_INIT,
>   CAL_DIRTY_RATE_ACTIVE);
> 
>> +if (ret == -1) {
>> +return NULL;
>> +}
>> +
>> +calculate_dirtyrate(config);
>> +
>> +ret = dirty_rate_set_state(CAL_DIRTY_RATE_END);
>> +
>> +return NULL;
>> +}
>> diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
>> new file mode 100644
>> index 000..914c363
>> --- /dev/null
>> +++ b/migration/dirtyrate.h
>> @@ -0,0 +1,44 @@
>> +/*
>> + *  Dirtyrate common functions
>> + *
>> + *  Copyright (c) 2020 HUAWEI TECHNOLOGIES CO., LTD.
>> + *
>> + *  Authors:
>> + *  Chuan Zheng 
>> + *
>> + *  This work is licensed under the terms of the GNU GPL, version 2 or 
>> later.
>> + *  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#ifndef QEMU_MIGRATION_DIRTYRATE_H
>> +#define QEMU_MIGRATION_DIRTYRATE_H
>> +
>> +/*
>> + * Sample 256 pages per GB as default.
>> + * TODO: Make it configurable.
>> + */
>> +#define DIRTYRATE_DEFAULT_SAMPLE_PAGES256
>> +
>> +/* Take 1s as default for calculation duration */
>> +#define DEFAULT_FETCH_DIRTYRATE_TIME_SEC  1
>> +
>> +struct DirtyRateConfig {
>> +uint64_t sample_pages_per_gigabytes; /* sample pages per GB */
>> +int64_t sample_period_seconds; /* time duration between two sampling */
>> +};
>> +
>> +/*
>> + *  To record calculate dirty_rate status:
>> + *  0: initial status, calculating thread is not be created here.
>> + *  1: calculating thread is created.
>> + *  2: calculating thread is end, we can get result.
>> + */
>> +typedef enum {
>> +CAL_DIRTY_RATE_INIT = 0,
>> +CAL_DIRTY_RATE_ACTIVE,
>> +CAL_DIRTY_RATE_END,
>> +} CalculatingDirtyRateState;
>> +
>> +void *get_dirtyrate_thread(void *arg);
>> +#endif
>> +
>> -- 
>> 1.8.3.1
>>




Re: [PATCH v3 10/10] migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function

2020-08-21 Thread Zheng Chuan



On 2020/8/21 2:00, Dr. David Alan Gilbert wrote:
> * Eric Blake (ebl...@redhat.com) wrote:
>> On 8/16/20 10:20 PM, Chuan Zheng wrote:
>>> Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function which could be 
>>> called
>>>
>>> Signed-off-by: Chuan Zheng 
>>> ---
>>
>>> +++ b/qapi/migration.json
>>> @@ -1621,3 +1621,45 @@
>>>   ##
>>>   { 'event': 'UNPLUG_PRIMARY',
>>> 'data': { 'device-id': 'str' } }
>>> +
>>> +##
>>> +# @DirtyRateInfo:
>>> +#
>>> +# Information about current dirty page rate of vm.
>>> +#
>>> +# @dirty-rate: @dirtyrate describing the dirty page rate of vm
>>> +#  in units of MB/s.
>>> +#  If this field return '-1', it means querying is not
>>> +#  start or not complete.
>>> +#
>>> +# @status: @status containing dirtyrate query status includes
>>> +#   status with 'not start measuring' or
>>> +#   'Still measuring' or 'measured'(since 5.2)
>>
>> Missing space before (
>>
>>> +##
>>> +{ 'struct': 'DirtyRateInfo',
>>> +  'data': {'dirty-rate': 'int64',
>>> +   'status': 'str'} }
>>
>> Based on your description, this should be an enum type rather than an
>> open-coded string.  Something like:
>>
>> { 'enum': 'DirtyRateStatus', 'data': [ 'unstarted', 'measuring', 'measured'
>> ] }
>> { 'struct': 'DirtyRateInfo', 'data': { 'dirty-rate': 'int64', 'status':
>> 'DirtyRateStatus' } }
> 
> Yes, and if you do that I think you'll find qmp would automatically
> define a C enum type for you, so you don't need to define the
> CalculatingDirtyRateStage yourself;   see how MigrationStatus works.
> 
> Dave
> 
Sure, it could be better,will fix it in V4:)
> 
>>
>>> +
>>> +##
>>> +# @calc-dirty-rate:
>>> +#
>>> +# start calculating dirty page rate for vm
>>> +#
>>> +# @calc-time: time in units of second for sample dirty pages
>>> +#
>>> +# Since: 5.2
>>> +#
>>> +# Example:
>>> +#   {"command": "cal-dirty-rate", "data": {"calc-time": 1} }
>>> +#
>>> +##
>>> +{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64'} }
>>> +
>>> +##
>>> +# @query-dirty-rate:
>>> +#
>>> +# query dirty page rate in units of MB/s for vm
>>> +#
>>> +# Since: 5.2
>>> +##
>>> +{ 'command': 'query-dirty-rate', 'returns': 'DirtyRateInfo' }
>>>
>>
>> -- 
>> Eric Blake, Principal Software Engineer
>> Red Hat, Inc.   +1-919-301-3226
>> Virtualization:  qemu.org | libvirt.org




Re: [PATCH v3 02/10] migration/dirtyrate: Add RamlockDirtyInfo to store sampled page info

2020-08-21 Thread Zheng Chuan



On 2020/8/21 0:20, Dr. David Alan Gilbert wrote:
> * Chuan Zheng (zhengch...@huawei.com) wrote:
>> Add RamlockDirtyInfo to store sampled page info of each ramblock.
>>
>> Signed-off-by: Chuan Zheng 
>> Signed-off-by: YanYing Zhuang 
>> ---
>>  migration/dirtyrate.h | 18 ++
>>  1 file changed, 18 insertions(+)
>>
>> diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
>> index 914c363..9650566 100644
>> --- a/migration/dirtyrate.h
>> +++ b/migration/dirtyrate.h
>> @@ -19,6 +19,11 @@
>>   */
>>  #define DIRTYRATE_DEFAULT_SAMPLE_PAGES256
>>  
>> +/*
>> + * Record ramblock idstr
>> + */
>> +#define RAMBLOCK_INFO_MAX_LEN 256
>> +
>>  /* Take 1s as default for calculation duration */
>>  #define DEFAULT_FETCH_DIRTYRATE_TIME_SEC  1
>>  
>> @@ -39,6 +44,19 @@ typedef enum {
>>  CAL_DIRTY_RATE_END,
>>  } CalculatingDirtyRateState;
>>  
>> +/*
>> + * Store dirtypage info for each ramblock.
>> + */
>> +struct RamblockDirtyInfo {
>> +char idstr[RAMBLOCK_INFO_MAX_LEN]; /* idstr for each ramblock */
>> Can you remind me; why not just use RAMBlock* here of the block you're
> interested in, rather than storing the name?
>
idstr is used to store which ramblock is sampled page in, we test it in
find_page_matched().
so you mean we just RAMBlock*, and take idstr out of RAMBlock* when it need to
find matched page?

>> +uint8_t *ramblock_addr; /* base address of ramblock we measure */
>> +size_t ramblock_pages; /* sum of dividation by 4K pages for ramblock */
> 
> 'dividation' is the wrong word, and 'sum' is only needed where you're
> adding things together.  I think this is 'ramblock size in TARGET_PAGEs'
> 
>> +size_t *sample_page_vfn; /* relative offset address for sampled page */
>> +unsigned int sample_pages_count; /* sum of sampled pages */
>> +unsigned int sample_dirty_count; /* sum of dirty pages we measure */
> 
> These are both 'count' rather than 'sum'
> 
OK, will be fixed in V4:)

>> +uint8_t *hash_result; /* array of hash result for sampled pages */
>> +};
>> +
>>  void *get_dirtyrate_thread(void *arg);
>>  #endif
>>  
>> -- 
>> 1.8.3.1
>>




Re: [PATCH v5 08/15] block/nvme: Use union of NvmeIdCtrl / NvmeIdNs structures

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:54PM +0200, Philippe Mathieu-Daudé wrote:
> We allocate an unique chunk of memory then use it for two
> different structures. By using an union, we make it clear
> the data is overlapping (and we can remove the casts).
> 
> Suggested-by: Stefan Hajnoczi 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 31 +++
>  1 file changed, 15 insertions(+), 16 deletions(-)
> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 99822d9fd36..2bd1935f951 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -508,9 +508,10 @@ static int nvme_cmd_sync(BlockDriverState *bs, 
> NVMeQueuePair *q,
>  static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp)
>  {
>  BDRVNVMeState *s = bs->opaque;
> -NvmeIdCtrl *idctrl;
> -NvmeIdNs *idns;
> -uint8_t *id;
> +union {
> +NvmeIdCtrl ctrl;
> +NvmeIdNs ns;
> +} *id;

What about defining a new 'NvmeId' type with this union?

>  NvmeLBAF *lbaf;
>  uint16_t oncs;
>  int r;
> @@ -520,14 +521,12 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  .cdw10 = cpu_to_le32(0x1),
>  };
>  
> -id = qemu_try_blockalign0(bs, sizeof(NvmeIdCtrl));
> +id = qemu_try_blockalign0(bs, sizeof(*id));
>  if (!id) {
>  error_setg(errp, "Cannot allocate buffer for identify response");
>  goto out;
>  }
> -idctrl = (NvmeIdCtrl *)id;
> -idns = (NvmeIdNs *)id;
> -r = qemu_vfio_dma_map(s->vfio, id, sizeof(NvmeIdCtrl), true, &iova);
> +r = qemu_vfio_dma_map(s->vfio, id, sizeof(*id), true, &iova);
>  if (r) {
>  error_setg(errp, "Cannot map buffer for DMA");
>  goto out;
> @@ -539,22 +538,22 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  goto out;
>  }
>  
> -if (le32_to_cpu(idctrl->nn) < namespace) {
> +if (le32_to_cpu(id->ctrl.nn) < namespace) {
>  error_setg(errp, "Invalid namespace");
>  goto out;
>  }
> -s->write_cache_supported = le32_to_cpu(idctrl->vwc) & 0x1;
> -s->max_transfer = (idctrl->mdts ? 1 << idctrl->mdts : 0) * s->page_size;
> +s->write_cache_supported = le32_to_cpu(id->ctrl.vwc) & 0x1;
> +s->max_transfer = (id->ctrl.mdts ? 1 << id->ctrl.mdts : 0) * 
> s->page_size;
>  /* For now the page list buffer per command is one page, to hold at most
>   * s->page_size / sizeof(uint64_t) entries. */
>  s->max_transfer = MIN_NON_ZERO(s->max_transfer,
>s->page_size / sizeof(uint64_t) * s->page_size);
>  
> -oncs = le16_to_cpu(idctrl->oncs);
> +oncs = le16_to_cpu(id->ctrl.oncs);
>  s->supports_write_zeroes = !!(oncs & NVME_ONCS_WRITE_ZEROS);
>  s->supports_discard = !!(oncs & NVME_ONCS_DSM);
>  
> -memset(id, 0, 4096);
> +memset(id, 0, sizeof(*id));
>  cmd.cdw10 = 0;
>  cmd.nsid = cpu_to_le32(namespace);
>  if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
> @@ -562,11 +561,11 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  goto out;
>  }
>  
> -s->nsze = le64_to_cpu(idns->nsze);
> -lbaf = &idns->lbaf[NVME_ID_NS_FLBAS_INDEX(idns->flbas)];
> +s->nsze = le64_to_cpu(id->ns.nsze);
> +lbaf = &id->ns.lbaf[NVME_ID_NS_FLBAS_INDEX(id->ns.flbas)];
>  
> -if (NVME_ID_NS_DLFEAT_WRITE_ZEROES(idns->dlfeat) &&
> -NVME_ID_NS_DLFEAT_READ_BEHAVIOR(idns->dlfeat) ==
> +if (NVME_ID_NS_DLFEAT_WRITE_ZEROES(id->ns.dlfeat) &&
> +NVME_ID_NS_DLFEAT_READ_BEHAVIOR(id->ns.dlfeat) ==
>  NVME_ID_NS_DLFEAT_READ_BEHAVIOR_ZEROES) {
>  bs->supported_write_flags |= BDRV_REQ_MAY_UNMAP;
>  }
> -- 
> 2.26.2
> 
> 

With or without the new tyoe, the patch looks good to me:

Reviewed-by: Stefano Garzarella 




Re: [PATCH v2] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Kevin Wolf
Am 20.08.2020 um 23:15 hat Eric Blake geschrieben:
> On 8/20/20 12:31 PM, Daniel P. Berrangé wrote:
> > Meson requires the build dir to be separate from the source tree. Many
> > people are used to just running "./configure && make" though and the
> > meson conversion breaks that.
> > 
> > This introduces some backcompat support to make it appear as if an
> > "in source tree" build is being done, but with the the results in the
> > "build/" directory. This allows "./configure && make" to work as it
> > did historically, albeit with the output binaries staying under build/.
> 
> An observation:
> 
> If you already had out-of-tree builds, this does not change anything. You
> can do an incremental build, where a tree that builds pre-merge should
> continue to build incrementally with 'make -C dir' post-merge.
> 
> If you are used to in-tree builds, and do a fresh checkout, this lets you
> maintain the illusion of an in-tree build although binaries may be located
> differently than you are used to.
> 
> But if you do an incremental update to an in-tree build, this will cause
> some odd behaviors, starting with the git update:
> 
> $ git merge $paolos_tag
> error: The following untracked working tree files would be overwritten by
> merge:
>   accel/kvm/trace.h
> ...
>   util/trace.h
> Please move or remove them before you merge.
> Aborting
> $ find -name trace.h -delete
> $ git merge $paolos_tag
> $ git am $this_patch
> $ make
> config-host.mak is out-of-date, running configure
> Using './build' as the directory for build output
> Submodule 'meson' (https://github.com/mesonbuild/meson/) registered for path
> 'meson'
> Cloning into '/home/eblake/qemu-tmp2/meson'...
> ...
> Command line for building ['libcommon.fa'] is long, using a response file
> ./ninjatool -t ninja2make --omit clean dist uninstall < build.ninja >
> Makefile.ninja
> /bin/sh: build.ninja: No such file or directory
>   GEN tests/test-qapi-gen
> make: Nothing to be done for 'all'.
> $ echo $?
> 0
> $ make
> changing dir to build for make ""...
> make[1]: Entering directory '/home/eblake/qemu-tmp2/build'
> Makefile:84: *** This is an out of tree build but your source tree
> (/home/eblake/qemu-tmp2) seems to have been used for an in-tree build. You
> can fix this by running "make distclean && rm -rf *-linux-user *-softmmu" in
> your source tree.  Stop.
> make[1]: Leaving directory '/home/eblake/qemu-tmp2/build'
> make: *** [GNUmakefile:11: all] Error 2
> $ echo $?
> 2
> 
> So I'm not sure why the first build gets as far as it does, but does NOT
> complete things and yet does not fail make, but my advice is that you should
> NOT try to an incremental build on in-tree build when crossing the meson
> epoch.  If you are a fan of in-tree convenience, you need a ONE-TIME
> distclean when pulling in these changes (the fact that you HAVE to clean up
> trace.h files to merge in the meson stuff should be a hint for that).  After
> that has been done, you can go back to pretending meson supports in-tree.

Sounds like it will be painful to switch between branches based on make
and branches based on meson. By extension, it will also be painful to
check out and build old versions for comparison, or doing that even more
than once during git bisect. :-(

Kevin




[PATCH] numa: hmat: fix cache size check

2020-08-21 Thread Igor Mammedov
when QEMU is started like:

qemu-system-x86_64 -smp 2 -machine hmat=on \
 -m 2G \
 -object memory-backend-ram,size=1G,id=m0 \
 -object memory-backend-ram,size=1G,id=m1 \
 -numa node,nodeid=0,memdev=m0 \
 -numa node,nodeid=1,memdev=m1,initiator=0 \
 -numa cpu,node-id=0,socket-id=0 \
 -numa cpu,node-id=0,socket-id=1 \
 -numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=5
 \
 -numa 
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=200M
 \
 -numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=10
 \
 -numa 
hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=100M
 \
 -numa 
hmat-cache,node-id=0,size=8K,level=1,associativity=direct,policy=write-back,line=5
 \
 -numa 
hmat-cache,node-id=0,size=16K,level=2,associativity=direct,policy=write-back,line=5

it errors out with:
 -numa 
hmat-cache,node-id=0,size=16K,level=2,associativity=direct,policy=write-back,line=5:
Invalid size=16384, the size of level=2 should be less than the 
size(8192) of level=1

which doesn't look right as one would expect that L1 < L2 < L3 ...
Fix it by sawpping relevant size checks.

Fixes: c412a48d4d91 (numa: Extend CLI to provide memory side cache information)
Signed-off-by: Igor Mammedov 
---
CC: jingqi@intel.com

 hw/core/numa.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/core/numa.c b/hw/core/numa.c
index d1a94a14f8..f9593ec716 100644
--- a/hw/core/numa.c
+++ b/hw/core/numa.c
@@ -425,10 +425,10 @@ void parse_numa_hmat_cache(MachineState *ms, 
NumaHmatCacheOptions *node,
 
 if ((node->level > 1) &&
 ms->numa_state->hmat_cache[node->node_id][node->level - 1] &&
-(node->size >=
+(node->size <=
 ms->numa_state->hmat_cache[node->node_id][node->level - 1]->size)) 
{
 error_setg(errp, "Invalid size=%" PRIu64 ", the size of level=%" PRIu8
-   " should be less than the size(%" PRIu64 ") of "
+   " should be larger than the size(%" PRIu64 ") of "
"level=%u", node->size, node->level,
ms->numa_state->hmat_cache[node->node_id]
  [node->level - 1]->size,
@@ -438,10 +438,10 @@ void parse_numa_hmat_cache(MachineState *ms, 
NumaHmatCacheOptions *node,
 
 if ((node->level < HMAT_LB_LEVELS - 1) &&
 ms->numa_state->hmat_cache[node->node_id][node->level + 1] &&
-(node->size <=
+(node->size >=
 ms->numa_state->hmat_cache[node->node_id][node->level + 1]->size)) 
{
 error_setg(errp, "Invalid size=%" PRIu64 ", the size of level=%" PRIu8
-   " should be larger than the size(%" PRIu64 ") of "
+   " should be less than the size(%" PRIu64 ") of "
"level=%u", node->size, node->level,
ms->numa_state->hmat_cache[node->node_id]
  [node->level + 1]->size,
-- 
2.26.2




Re: [PATCH v5 09/15] block/nvme: Replace qemu_try_blockalign0 by qemu_try_blockalign/memset

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:55PM +0200, Philippe Mathieu-Daudé wrote:
> In the next commit we'll get rid of qemu_try_blockalign().
> To ease review, first replace qemu_try_blockalign0() by explicit
> calls to qemu_try_blockalign() and memset().
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 16 +---
>  1 file changed, 9 insertions(+), 7 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 2bd1935f951..ac6bb52043d 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -174,12 +174,12 @@ static void nvme_init_queue(BlockDriverState *bs, 
> NVMeQueue *q,
>  
>  bytes = ROUND_UP(nentries * entry_bytes, s->page_size);
>  q->head = q->tail = 0;
> -q->queue = qemu_try_blockalign0(bs, bytes);
> -
> +q->queue = qemu_try_blockalign(bs, bytes);
>  if (!q->queue) {
>  error_setg(errp, "Cannot allocate queue");
>  return;
>  }
> +memset(q->queue, 0, bytes);
>  r = qemu_vfio_dma_map(s->vfio, q->queue, bytes, false, &q->iova);
>  if (r) {
>  error_setg(errp, "Cannot map queue");
> @@ -223,11 +223,12 @@ static NVMeQueuePair 
> *nvme_create_queue_pair(BlockDriverState *bs,
>  if (!q) {
>  return NULL;
>  }
> -q->prp_list_pages = qemu_try_blockalign0(bs,
> +q->prp_list_pages = qemu_try_blockalign(bs,
>s->page_size * NVME_QUEUE_SIZE);
>  if (!q->prp_list_pages) {
>  goto fail;
>  }
> +memset(q->prp_list_pages, 0, s->page_size * NVME_QUEUE_SIZE);
>  qemu_mutex_init(&q->lock);
>  q->s = s;
>  q->index = idx;
> @@ -521,7 +522,7 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  .cdw10 = cpu_to_le32(0x1),
>  };
>  
> -id = qemu_try_blockalign0(bs, sizeof(*id));
> +id = qemu_try_blockalign(bs, sizeof(*id));
>  if (!id) {
>  error_setg(errp, "Cannot allocate buffer for identify response");
>  goto out;
> @@ -531,8 +532,9 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  error_setg(errp, "Cannot map buffer for DMA");
>  goto out;
>  }
> -cmd.prp1 = cpu_to_le64(iova);
>  
> +memset(id, 0, sizeof(*id));
> +cmd.prp1 = cpu_to_le64(iova);
>  if (nvme_cmd_sync(bs, s->queues[INDEX_ADMIN], &cmd)) {
>  error_setg(errp, "Failed to identify controller");
>  goto out;
> @@ -1283,11 +1285,11 @@ static int coroutine_fn 
> nvme_co_pdiscard(BlockDriverState *bs,
>  
>  assert(s->nr_queues > 1);
>  
> -buf = qemu_try_blockalign0(bs, s->page_size);
> +buf = qemu_try_blockalign(bs, s->page_size);
>  if (!buf) {
>  return -ENOMEM;
>  }
> -
> +memset(buf, 0, s->page_size);
>  buf->nlb = cpu_to_le32(bytes >> s->blkshift);
>  buf->slba = cpu_to_le64(offset >> s->blkshift);
>  buf->cattr = 0;
> -- 
> 2.26.2
> 
> 




Re: [PATCH 07/18] hw/sd: sd: Fix incorrect populated function switch status data structure

2020-08-21 Thread Bin Meng
Hi Sai,

On Fri, Aug 21, 2020 at 6:04 PM Sai Pavan Boddu
 wrote:
>
> Hi Philippe,
>
> First two patch of SD look good. Tested them over zynqmp and versal
> platforms.
>

Thanks for testing.

Can I add your Tested-by tag?

Regards,
Bin



Re: [PATCH] virtio-gpu: fix unmap the already mapped items

2020-08-21 Thread Gerd Hoffmann
On Fri, Aug 21, 2020 at 04:49:45PM +0800, Li Zhijian wrote:
> we go here either (!(*iov)[i].iov_base) or (len != l), so we need to consider
> to unmap the 'i'th item as well when the 'i'th item is not nil
> 
> Signed-off-by: Li Zhijian 
> ---
>  hw/display/virtio-gpu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
> index 5f0dd7c150..1f777e43ff 100644
> --- a/hw/display/virtio-gpu.c
> +++ b/hw/display/virtio-gpu.c
> @@ -656,7 +656,7 @@ int virtio_gpu_create_mapping_iov(VirtIOGPU *g,
>  qemu_log_mask(LOG_GUEST_ERROR, "%s: failed to map MMIO memory 
> for"
>" resource %d element %d\n",
>__func__, ab->resource_id, i);
> -virtio_gpu_cleanup_mapping_iov(g, *iov, i);
> +virtio_gpu_cleanup_mapping_iov(g, *iov, i + 
> !!(*iov)[i].iov_base);

Cute trick, but the code should be readable without having to dig out
the commit message which explains it.  Can we have something simpler
along the lines of "if (iov_base) { i++; /* cleanup partial map */ }"
please?

thanks,
  Gerd




Re: [PATCH v5 10/15] block/nvme: Replace qemu_try_blockalign(bs) by qemu_try_memalign(pg_sz)

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:56PM +0200, Philippe Mathieu-Daudé wrote:
> qemu_try_blockalign() is a generic API that call back to the
> block driver to return its page alignment. As we call from
> within the very same driver, we already know to page alignment
> stored in our state. Remove indirections and use the value from
> BDRVNVMeState.
> This change is required to later remove the BlockDriverState
> argument, to make nvme_init_queue() per hardware, and not per
> block driver.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 10 +-
>  1 file changed, 5 insertions(+), 5 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index ac6bb52043d..f180078e781 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -174,7 +174,7 @@ static void nvme_init_queue(BlockDriverState *bs, 
> NVMeQueue *q,
>  
>  bytes = ROUND_UP(nentries * entry_bytes, s->page_size);
>  q->head = q->tail = 0;
> -q->queue = qemu_try_blockalign(bs, bytes);
> +q->queue = qemu_try_memalign(s->page_size, bytes);
>  if (!q->queue) {
>  error_setg(errp, "Cannot allocate queue");
>  return;
> @@ -223,7 +223,7 @@ static NVMeQueuePair 
> *nvme_create_queue_pair(BlockDriverState *bs,
>  if (!q) {
>  return NULL;
>  }
> -q->prp_list_pages = qemu_try_blockalign(bs,
> +q->prp_list_pages = qemu_try_memalign(s->page_size,
>s->page_size * NVME_QUEUE_SIZE);
>  if (!q->prp_list_pages) {
>  goto fail;
> @@ -522,7 +522,7 @@ static void nvme_identify(BlockDriverState *bs, int 
> namespace, Error **errp)
>  .cdw10 = cpu_to_le32(0x1),
>  };
>  
> -id = qemu_try_blockalign(bs, sizeof(*id));
> +id = qemu_try_memalign(s->page_size, sizeof(*id));
>  if (!id) {
>  error_setg(errp, "Cannot allocate buffer for identify response");
>  goto out;
> @@ -1141,7 +1141,7 @@ static int nvme_co_prw(BlockDriverState *bs, uint64_t 
> offset, uint64_t bytes,
>  return nvme_co_prw_aligned(bs, offset, bytes, qiov, is_write, flags);
>  }
>  trace_nvme_prw_buffered(s, offset, bytes, qiov->niov, is_write);
> -buf = qemu_try_blockalign(bs, bytes);
> +buf = qemu_try_memalign(s->page_size, bytes);
>  
>  if (!buf) {
>  return -ENOMEM;
> @@ -1285,7 +1285,7 @@ static int coroutine_fn 
> nvme_co_pdiscard(BlockDriverState *bs,
>  
>  assert(s->nr_queues > 1);
>  
> -buf = qemu_try_blockalign(bs, s->page_size);
> +buf = qemu_try_memalign(s->page_size, s->page_size);
>  if (!buf) {
>  return -ENOMEM;
>  }
> -- 
> 2.26.2
> 
> 




Re: [PATCH v5 11/15] block/nvme: Simplify nvme_init_queue() arguments

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:57PM +0200, Philippe Mathieu-Daudé wrote:
> nvme_init_queue() doesn't require BlockDriverState anymore.
> Replace it by BDRVNVMeState to simplify.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 7 +++
>  1 file changed, 3 insertions(+), 4 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index f180078e781..5b69fc75a60 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -165,10 +165,9 @@ static QemuOptsList runtime_opts = {
>  },
>  };
>  
> -static void nvme_init_queue(BlockDriverState *bs, NVMeQueue *q,
> +static void nvme_init_queue(BDRVNVMeState *s, NVMeQueue *q,
>  int nentries, int entry_bytes, Error **errp)
>  {
> -BDRVNVMeState *s = bs->opaque;
>  size_t bytes;
>  int r;
>  
> @@ -251,14 +250,14 @@ static NVMeQueuePair 
> *nvme_create_queue_pair(BlockDriverState *bs,
>  req->prp_list_iova = prp_list_iova + i * s->page_size;
>  }
>  
> -nvme_init_queue(bs, &q->sq, size, NVME_SQ_ENTRY_BYTES, &local_err);
> +nvme_init_queue(s, &q->sq, size, NVME_SQ_ENTRY_BYTES, &local_err);
>  if (local_err) {
>  error_propagate(errp, local_err);
>  goto fail;
>  }
>  q->sq.doorbell = &s->regs->doorbells[idx * 2 * s->doorbell_scale];
>  
> -nvme_init_queue(bs, &q->cq, size, NVME_CQ_ENTRY_BYTES, &local_err);
> +nvme_init_queue(s, &q->cq, size, NVME_CQ_ENTRY_BYTES, &local_err);
>  if (local_err) {
>  error_propagate(errp, local_err);
>  goto fail;
> -- 
> 2.26.2
> 
> 




[PATCH 0/2] fdmon-poll: reset npfd when upgrading to fdmon-epoll

2020-08-21 Thread Stefan Hajnoczi
Fix an assertion failure when aio_poll() is called in a aio_disable_external()
region. The failure happens when fdmon-poll is first upgraded to fdmon-epoll
and then downgraded again due to aio_disable_external().

Stefan Hajnoczi (2):
  fdmon-poll: reset npfd when upgrading to fdmon-epoll
  tests: add test-fdmon-epoll

 MAINTAINERS  |  1 +
 tests/Makefile.include   |  4 +++
 tests/test-fdmon-epoll.c | 73 
 util/fdmon-poll.c|  1 +
 4 files changed, 79 insertions(+)
 create mode 100644 tests/test-fdmon-epoll.c

-- 
2.26.2



[PATCH 2/2] tests: add test-fdmon-epoll

2020-08-21 Thread Stefan Hajnoczi
Test aio_disable_external(), which switches from fdmon-epoll back to
fdmon-poll. This resulted in an assertion failure that was fixed in the
previous patch.

Signed-off-by: Stefan Hajnoczi 
---
 MAINTAINERS  |  1 +
 tests/Makefile.include   |  4 +++
 tests/test-fdmon-epoll.c | 73 
 3 files changed, 78 insertions(+)
 create mode 100644 tests/test-fdmon-epoll.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 0886eb3d2b..6d2e99b94f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2109,6 +2109,7 @@ F: migration/block*
 F: include/block/aio.h
 F: include/block/aio-wait.h
 F: scripts/qemugdb/aio.py
+F: tests/test-fdmon-epoll.c
 T: git https://github.com/stefanha/qemu.git block
 
 Block SCSI subsystem
diff --git a/tests/Makefile.include b/tests/Makefile.include
index c7e4646ded..b785e487b7 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -78,6 +78,9 @@ check-unit-$(CONFIG_SOFTMMU) += tests/test-iov$(EXESUF)
 check-unit-y += tests/test-bitmap$(EXESUF)
 check-unit-$(CONFIG_BLOCK) += tests/test-aio$(EXESUF)
 check-unit-$(CONFIG_BLOCK) += tests/test-aio-multithread$(EXESUF)
+ifeq ($(CONFIG_EPOLL_CREATE1),y)
+check-unit-$(CONFIG_BLOCK) += tests/test-fdmon-epoll$(EXESUF)
+endif
 check-unit-$(CONFIG_BLOCK) += tests/test-throttle$(EXESUF)
 check-unit-$(CONFIG_BLOCK) += tests/test-thread-pool$(EXESUF)
 check-unit-$(CONFIG_BLOCK) += tests/test-hbitmap$(EXESUF)
@@ -408,6 +411,7 @@ tests/test-char$(EXESUF): tests/test-char.o 
$(test-util-obj-y) $(test-io-obj-y)
 tests/test-coroutine$(EXESUF): tests/test-coroutine.o $(test-block-obj-y)
 tests/test-aio$(EXESUF): tests/test-aio.o $(test-block-obj-y)
 tests/test-aio-multithread$(EXESUF): tests/test-aio-multithread.o 
$(test-block-obj-y)
+tests/test-fdmon-epoll$(EXESUF): tests/test-fdmon-epoll.o $(test-block-obj-y)
 tests/test-throttle$(EXESUF): tests/test-throttle.o $(test-block-obj-y)
 tests/test-bdrv-drain$(EXESUF): tests/test-bdrv-drain.o $(test-block-obj-y) 
$(test-util-obj-y)
 tests/test-bdrv-graph-mod$(EXESUF): tests/test-bdrv-graph-mod.o 
$(test-block-obj-y) $(test-util-obj-y)
diff --git a/tests/test-fdmon-epoll.c b/tests/test-fdmon-epoll.c
new file mode 100644
index 00..11fd8a2fa9
--- /dev/null
+++ b/tests/test-fdmon-epoll.c
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * fdmon-epoll tests
+ *
+ * Copyright (c) 2020 Red Hat, Inc.
+ */
+
+#include "qemu/osdep.h"
+#include "block/aio.h"
+#include "qapi/error.h"
+#include "qemu/main-loop.h"
+
+static AioContext *ctx;
+
+static void dummy_fd_handler(EventNotifier *notifier)
+{
+event_notifier_test_and_clear(notifier);
+}
+
+static void add_event_notifiers(EventNotifier *notifiers, size_t n)
+{
+for (size_t i = 0; i < n; i++) {
+event_notifier_init(¬ifiers[i], false);
+aio_set_event_notifier(ctx, ¬ifiers[i], false,
+   dummy_fd_handler, NULL);
+}
+}
+
+static void remove_event_notifiers(EventNotifier *notifiers, size_t n)
+{
+for (size_t i = 0; i < n; i++) {
+aio_set_event_notifier(ctx, ¬ifiers[i], false, NULL, NULL);
+event_notifier_cleanup(¬ifiers[i]);
+}
+}
+
+/* Check that fd handlers work when external clients are disabled */
+static void test_external_disabled(void)
+{
+EventNotifier notifiers[100];
+
+/* fdmon-epoll is only enabled when many fd handlers are registered */
+add_event_notifiers(notifiers, G_N_ELEMENTS(notifiers));
+
+event_notifier_set(¬ifiers[0]);
+assert(aio_poll(ctx, true));
+
+aio_disable_external(ctx);
+event_notifier_set(¬ifiers[0]);
+assert(aio_poll(ctx, true));
+aio_enable_external(ctx);
+
+remove_event_notifiers(notifiers, G_N_ELEMENTS(notifiers));
+}
+
+int main(int argc, char **argv)
+{
+/*
+ * This code relies on the fact that fdmon-io_uring disables itself when
+ * the glib main loop is in use. The main loop uses fdmon-poll and upgrades
+ * to fdmon-epoll when the number of fds exceeds a threshold.
+ */
+qemu_init_main_loop(&error_fatal);
+ctx = qemu_get_aio_context();
+
+while (g_main_context_iteration(NULL, false)) {
+/* Do nothing */
+}
+
+g_test_init(&argc, &argv, NULL);
+g_test_add_func("/fdmon-epoll/external-disabled", test_external_disabled);
+return g_test_run();
+}
-- 
2.26.2



Re: [PATCH] configure: add support for psuedo-"in source tree" builds

2020-08-21 Thread Daniel P . Berrangé
On Fri, Aug 21, 2020 at 11:58:21AM +0200, Kevin Wolf wrote:
> Am 20.08.2020 um 19:42 hat Eric Blake geschrieben:
> > On 8/20/20 11:55 AM, Daniel P. Berrangé wrote:
> > > Meson requires the build dir to be separate from the source tree. Many
> > > people are used to just running "./configure && make" though and the
> > > meson conversion breaks that.
> > > 
> > > This introduces some backcompat support to make it appear as if an
> > > "in source tree" build is being done, but with the the results in the
> > > "build/" directory. This allows "./configure && make" to work as it
> > > did historically, albeit with the output binaries staying under build/.
> > > 
> > > Signed-off-by: Daniel P. Berrangé 
> > > ---
> > 
> > In addition to reviews you already have,
> > 
> > 
> > > I've not tested it beyond that. Note it blows away the "build/"
> > > dir each time ./configure is run so it is pristine each time.
> > 
> > I definitely like the idea of only blowing away what we created - but if we
> > created build, then recreating it for each new configure run is nice.
> 
> I think I actually wouldn't automatically remove anything on configure.
> It can be surprising behaviour for configure to delete directories, and
> the old setup didn't do an automatic "make clean" either. By having a
> separate build directory, manually emptying as needed has already become
> easier.

The issue is that previously you could do

  ./configure
  make
  ./configure
  make

This isn't possible with the new system because meson will refuse
to use the "build/" directory if it already contains a previous
configured build.

Doing "rm -rf build" in configure lets the above sequence work.

I can remove the "rm -rf biuld" in configure if we are happy
to require


  ./configure
  make
  make distclean
  ./configure
  make

because the "GNUmakefile" wires up "distclean" to purge the
build/ directory.

> > > We could optionally symlink binaries from build/ into $PWD
> > > if poeople think that is important, eg by changing GNUmakefile
> > > to have:
> > > 
> > > recurse: all
> > >   for bin in `find build -maxdepth 1 -type f -executable | grep -v -E 
> > > '(ninjatool|config.status)'`; \
> > 
> > Using -maxdepth gets rid of the need to pre-create empty directories for
> > nested binaries, but also loses out on binaries such as
> > x86_64-softmmu/qemu-system-x86_64.  Oh, it looks like meson creates
> > qemu-system-x86_64 as a binary in the top level, then a symlink in its old
> > location.  Populating symlinks to ALL old locations is thus trickier than
> > what you are proposing here, so it is fine to save that for a followup patch
> > (let's get the bare minimum in first, so that at least ./configure && make
> > works, before we worry about back-compat symlinks).
> 
> Having the system emulator symlinks in the top level would be a change,
> but even more convenient than the original places. I'd vote for adding
> the auto-symlinking at least for the tools; if the top-level symlinks
> for system emulators get also symlinked by this, that's fine, too.
> 
> I was actually surprised that Dan reports "make check" from the source
> tree to be working without the symlinks. Some code must be cleverer than
> I thought!

That's because "make check" is not actually running from the source
tree. When you run "make check" in the source tree, what's acutally
happening is

  cd build
  make check

so it is actually running from build-dir context.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|




[PATCH 1/2] fdmon-poll: reset npfd when upgrading to fdmon-epoll

2020-08-21 Thread Stefan Hajnoczi
npfd keeps track of how many pollfds are currently being monitored. It
must be reset to 0 when fdmon_poll_wait() returns.

When npfd reaches a treshold we switch to fdmon-epoll because it scales
better.

This patch resets npfd in the case where we switch to fdmon-epoll.
Forgetting to do so results in the following assertion failure:

  util/fdmon-poll.c:65: fdmon_poll_wait: Assertion `npfd == 0' failed.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1869952
Fixes: 1f050a4690f62a1e7dabc4f44141e9f762c3769f ("aio-posix: extract ppoll(2) 
and epoll(7) fd monitoring")
Signed-off-by: Stefan Hajnoczi 
---
 util/fdmon-poll.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/util/fdmon-poll.c b/util/fdmon-poll.c
index 488067b679..5fe3b47865 100644
--- a/util/fdmon-poll.c
+++ b/util/fdmon-poll.c
@@ -73,6 +73,7 @@ static int fdmon_poll_wait(AioContext *ctx, AioHandlerList 
*ready_list,
 
 /* epoll(7) is faster above a certain number of fds */
 if (fdmon_epoll_try_upgrade(ctx, npfd)) {
+npfd = 0; /* we won't need pollfds[], reset npfd */
 return ctx->fdmon_ops->wait(ctx, ready_list, timeout);
 }
 
-- 
2.26.2



Re: [PATCH v2] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Paolo Bonzini
On 21/08/20 12:04, Kevin Wolf wrote:
>> So I'm not sure why the first build gets as far as it does, but does NOT
>> complete things and yet does not fail make, but my advice is that you should
>> NOT try to an incremental build on in-tree build when crossing the meson
>> epoch.  If you are a fan of in-tree convenience, you need a ONE-TIME
>> distclean when pulling in these changes (the fact that you HAVE to clean up
>> trace.h files to merge in the meson stuff should be a hint for that).  After
>> that has been done, you can go back to pretending meson supports in-tree.
> Sounds like it will be painful to switch between branches based on make
> and branches based on meson. By extension, it will also be painful to
> check out and build old versions for comparison, or doing that even more
> than once during git bisect. :-(

Not if you switch to out-of-tree builds...

Paolo




Re: [PATCH v5 12/15] block/nvme: Replace BDRV_POLL_WHILE by AIO_WAIT_WHILE

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:58PM +0200, Philippe Mathieu-Daudé wrote:
> BDRV_POLL_WHILE() is defined as:
> 
>   #define BDRV_POLL_WHILE(bs, cond) ({  \
>   BlockDriverState *bs_ = (bs); \
>   AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \
>  cond); })
> 
> As we will remove the BlockDriverState use in the next commit,
> start by using the exploded version of BDRV_POLL_WHILE().
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 5b69fc75a60..456fe61f5ea 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -493,6 +493,7 @@ static void nvme_cmd_sync_cb(void *opaque, int ret)
>  static int nvme_cmd_sync(BlockDriverState *bs, NVMeQueuePair *q,
>   NvmeCmd *cmd)
>  {
> +AioContext *aio_context = bdrv_get_aio_context(bs);
>  NVMeRequest *req;
>  int ret = -EINPROGRESS;
>  req = nvme_get_free_req(q);
> @@ -501,7 +502,7 @@ static int nvme_cmd_sync(BlockDriverState *bs, 
> NVMeQueuePair *q,
>  }
>  nvme_submit_command(q, req, cmd, nvme_cmd_sync_cb, &ret);
>  
> -BDRV_POLL_WHILE(bs, ret == -EINPROGRESS);
> +AIO_WAIT_WHILE(aio_context, ret == -EINPROGRESS);

Maybe I would have:

AIO_WAIT_WHILE(bdrv_get_aio_context(bs), ret == -EINPROGRESS);

But it doesn't matter, LGTM:

Reviewed-by: Stefano Garzarella 

>  return ret;
>  }
>  
> -- 
> 2.26.2
> 
> 




Re: [PATCH v2] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Paolo Bonzini
On 20/08/20 19:31, Daniel P. Berrangé wrote:
> Meson requires the build dir to be separate from the source tree. Many
> people are used to just running "./configure && make" though and the
> meson conversion breaks that.
> 
> This introduces some backcompat support to make it appear as if an
> "in source tree" build is being done, but with the the results in the
> "build/" directory. This allows "./configure && make" to work as it
> did historically, albeit with the output binaries staying under build/.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
> 
> This is a simple integration of Eric's proposal from
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg07290.html
> 
> with a bit of configure magic. It is enough to enable
> 
>./configure
>make
>make check
> 
> I've not tested it beyond that. Note it blows away the "build/"
> dir each time ./configure is run so it is pristine each time.
> 
> We could optionally symlink binaries from build/ into $PWD
> if people think that is important, eg by changing GNUmakefile
> to have:
> 
> recurse: all
> for bin in `find build -maxdepth 1 -type f -executable | grep -v -E 
> '(ninjatool|config.status)'`; \
> do \
>   ln -f -s $$bin . ; \
> done
> 
> and some cleanup logic to purge the symlinks for "make clean"
> 
> This goes on top of Paolo's most recent meson port v175 posting,
> or whatever number it is upto now :-)
> 
> In v2:
> 
>  - Use a marker file so we don't blow away a build/ dir
>we didn't create
>  - Silence the distclean rule
>  - Fix broken use of error_exit that's not defined yet
>  - Add comment to GNUmakefile
> 
>  .gitignore |  2 ++
>  configure  | 48 +---
>  2 files changed, 47 insertions(+), 3 deletions(-)
> 
> diff --git a/.gitignore b/.gitignore
> index 92311284ef..4ccb9ed975 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -1,3 +1,5 @@
> +/GNUmakefile
> +/build/
>  /.doctrees
>  /config-devices.*
>  /config-all-devices.*
> diff --git a/configure b/configure
> index cc5f58f31a..cdffe221c7 100755
> --- a/configure
> +++ b/configure
> @@ -11,6 +11,51 @@ unset CLICOLOR_FORCE GREP_OPTIONS
>  # Don't allow CCACHE, if present, to use cached results of compile tests!
>  export CCACHE_RECACHE=yes
>  
> +# make source path absolute
> +source_path=$(cd "$(dirname -- "$0")"; pwd)
> +
> +if test "$PWD" == "$source_path"
> +then
> +echo "Using './build' as the directory for build output"
> +
> +MARKER=build/auto-created-by-configure
> +
> +if test -e build
> +then
> + if test -f $MARKER
> + then
> +rm -rf build
> + else
> + echo "ERROR: ./build dir already exists and was not previously 
> created by configure"
> + exit 1
> + fi
> +fi
> +
> +mkdir build
> +touch $MARKER
> +
> +cat > GNUmakefile < +# This file is auto-generated by configure to support in-source tree
> +# 'make' command invokation
> +
> +ifeq (\$(MAKECMDGOALS),)
> +recurse: all
> +endif
> +
> +.NOTPARALLEL: %
> +%: force
> + @echo 'changing dir to build for \$(MAKE) "\$(MAKECMDGOALS)"...'
> + @\$(MAKE) -C build -f Makefile \$(MAKECMDGOALS)
> + @if test "\$(MAKECMDGOALS)" = "distclean" && test -e 
> build/auto-created-by-configure ; then rm -rf build GNUmakefile ; fi
> +force: ;
> +.PHONY: force
> +GNUmakefile: ;
> +
> +EOF
> +cd build
> +exec $source_path/configure "$@"
> +fi
> +
>  # Temporary directory used for files created while
>  # configure runs. Since it is in the build directory
>  # we can safely blow away any previous version of it
> @@ -297,9 +342,6 @@ ld_has() {
>  $ld --help 2>/dev/null | grep ".$1" >/dev/null 2>&1
>  }
>  
> -# make source path absolute
> -source_path=$(cd "$(dirname -- "$0")"; pwd)
> -
>  if printf %s\\n "$source_path" "$PWD" | grep -q "[[:space:]:]";
>  then
>error_exit "main directory cannot contain spaces nor colons"
> 

Queued, thanks!

Paolo




Re: [PATCH v2] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Daniel P . Berrangé
On Fri, Aug 21, 2020 at 12:04:26PM +0200, Kevin Wolf wrote:
> Am 20.08.2020 um 23:15 hat Eric Blake geschrieben:
> > On 8/20/20 12:31 PM, Daniel P. Berrangé wrote:
> > > Meson requires the build dir to be separate from the source tree. Many
> > > people are used to just running "./configure && make" though and the
> > > meson conversion breaks that.
> > > 
> > > This introduces some backcompat support to make it appear as if an
> > > "in source tree" build is being done, but with the the results in the
> > > "build/" directory. This allows "./configure && make" to work as it
> > > did historically, albeit with the output binaries staying under build/.
> > 
> > An observation:
> > 
> > If you already had out-of-tree builds, this does not change anything. You
> > can do an incremental build, where a tree that builds pre-merge should
> > continue to build incrementally with 'make -C dir' post-merge.
> > 
> > If you are used to in-tree builds, and do a fresh checkout, this lets you
> > maintain the illusion of an in-tree build although binaries may be located
> > differently than you are used to.
> > 
> > But if you do an incremental update to an in-tree build, this will cause
> > some odd behaviors, starting with the git update:
> > 
> > $ git merge $paolos_tag
> > error: The following untracked working tree files would be overwritten by
> > merge:
> > accel/kvm/trace.h
> > ...
> > util/trace.h
> > Please move or remove them before you merge.
> > Aborting
> > $ find -name trace.h -delete
> > $ git merge $paolos_tag
> > $ git am $this_patch
> > $ make
> > config-host.mak is out-of-date, running configure
> > Using './build' as the directory for build output
> > Submodule 'meson' (https://github.com/mesonbuild/meson/) registered for path
> > 'meson'
> > Cloning into '/home/eblake/qemu-tmp2/meson'...
> > ...
> > Command line for building ['libcommon.fa'] is long, using a response file
> > ./ninjatool -t ninja2make --omit clean dist uninstall < build.ninja >
> > Makefile.ninja
> > /bin/sh: build.ninja: No such file or directory
> >   GEN tests/test-qapi-gen
> > make: Nothing to be done for 'all'.
> > $ echo $?
> > 0
> > $ make
> > changing dir to build for make ""...
> > make[1]: Entering directory '/home/eblake/qemu-tmp2/build'
> > Makefile:84: *** This is an out of tree build but your source tree
> > (/home/eblake/qemu-tmp2) seems to have been used for an in-tree build. You
> > can fix this by running "make distclean && rm -rf *-linux-user *-softmmu" in
> > your source tree.  Stop.
> > make[1]: Leaving directory '/home/eblake/qemu-tmp2/build'
> > make: *** [GNUmakefile:11: all] Error 2
> > $ echo $?
> > 2
> > 
> > So I'm not sure why the first build gets as far as it does, but does NOT
> > complete things and yet does not fail make, but my advice is that you should
> > NOT try to an incremental build on in-tree build when crossing the meson
> > epoch.  If you are a fan of in-tree convenience, you need a ONE-TIME
> > distclean when pulling in these changes (the fact that you HAVE to clean up
> > trace.h files to merge in the meson stuff should be a hint for that).  After
> > that has been done, you can go back to pretending meson supports in-tree.
> 
> Sounds like it will be painful to switch between branches based on make
> and branches based on meson. By extension, it will also be painful to
> check out and build old versions for comparison, or doing that even more
> than once during git bisect. :-(

Such is life when using in-source-dir builds.

Even with our current build system, using a separate build directory
makes it easier to work across multiple branches, as you can trivially
keep the builds from each branch separate and they won't interfere
with each other.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|




Re: [PATCH v2] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Kevin Wolf
Am 20.08.2020 um 19:31 hat Daniel P. Berrangé geschrieben:
> Meson requires the build dir to be separate from the source tree. Many
> people are used to just running "./configure && make" though and the
> meson conversion breaks that.
> 
> This introduces some backcompat support to make it appear as if an
> "in source tree" build is being done, but with the the results in the
> "build/" directory. This allows "./configure && make" to work as it
> did historically, albeit with the output binaries staying under build/.
> 
> Signed-off-by: Daniel P. Berrangé 
> ---
> 
> This is a simple integration of Eric's proposal from
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg07290.html
> 
> with a bit of configure magic. It is enough to enable
> 
>./configure
>make
>make check
> 
> I've not tested it beyond that. Note it blows away the "build/"
> dir each time ./configure is run so it is pristine each time.

I guess "make install" is the only other one that normal users would
care about. We shoud make sure that it works (I don't see why it
wouldn't, but worth testing anyway).

Kevin




Re: [PATCH 07/18] hw/sd: sd: Fix incorrect populated function switch status data structure

2020-08-21 Thread Sai Pavan Boddu
Hi Philippe,

First two patch of SD look good. Tested them over zynqmp and versal
platforms.

Thanks,
Sai Pavan
On Tue, Aug 18, 2020 at 04:30:15PM +, Sai Pavan Boddu wrote:
> Thanks Philippe,
> 
> I would test this and get back.
> 
> Regards,
> Sai Pavan
> 
> > -Original Message-
> > From: Philippe Mathieu-Daudé  On
> > Behalf Of Philippe Mathieu-Daudé
> > Sent: Saturday, August 15, 2020 1:29 PM
> > To: Bin Meng ; Alistair Francis
> > ; Bastian Koppelmann  > paderborn.de>; Palmer Dabbelt ; Sagar
> > Karandikar ; qemu-devel@nongnu.org; qemu-
> > ri...@nongnu.org; Sai Pavan Boddu 
> > Cc: Bin Meng ; qemu-bl...@nongnu.org
> > Subject: Re: [PATCH 07/18] hw/sd: sd: Fix incorrect populated function 
> > switch
> > status data structure
> > 
> > +Sai Pavan
> > 
> > On 8/14/20 6:40 PM, Bin Meng wrote:
> > > From: Bin Meng 
> > >
> > > At present the function switch status data structure bit [399:376] are
> > > wrongly pupulated. These 3 bytes encode function switch status for the
> > > 6 function groups, with 4 bits per group, starting from function group
> > > 6 at bit 399, then followed by function group 5 at bit 395, and so on.
> > >
> > > However the codes mistakenly fills in the function group 1 status at
> > > bit 399. This fixes the code logic.
> > >
> > 
> > Fixes: a1bb27b1e9 ("SD card emulation (initial implementation)")
> > 
> > > Signed-off-by: Bin Meng 
> > > ---
> > >
> > >  hw/sd/sd.c | 6 +-
> > >  1 file changed, 5 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/hw/sd/sd.c b/hw/sd/sd.c
> > > index fad9cf1..51f5900 100644
> > > --- a/hw/sd/sd.c
> > > +++ b/hw/sd/sd.c
> > > @@ -806,11 +806,15 @@ static void sd_function_switch(SDState *sd,
> > uint32_t arg)
> > >  sd->data[11] = 0x43;
> > >  sd->data[12] = 0x80; /* Supported group 1 functions */
> > >  sd->data[13] = 0x03;
> > > +
> > > +sd->data[14] = 0;
> > > +sd->data[15] = 0;
> > > +sd->data[16] = 0;
> > 
> > Pointless zero initialization.
> > 
> > Reviewed-by: Philippe Mathieu-Daudé 
> > 
> > I'll wait until next week in case Sai Pavan wants to test this patch (I 
> > don't have
> > access to the Xilinx images anymore) then I'll queue this via my sd-next 
> > tree.
> > 
> > Regards,
> > 
> > Phil.
> > 
> > >  for (i = 0; i < 6; i ++) {
> > >  new_func = (arg >> (i * 4)) & 0x0f;
> > >  if (mode && new_func != 0x0f)
> > >  sd->function_group[i] = new_func;
> > > -sd->data[14 + (i >> 1)] = new_func << ((i * 4) & 4);
> > > +sd->data[16 - (i >> 1)] |= new_func << ((i % 2) * 4);
> > >  }
> > >  memset(&sd->data[17], 0, 47);
> > >  stw_be_p(sd->data + 64, sd_crc16(sd->data, 64));
> > >
> 



Re: [PATCH v5 13/15] block/nvme: Simplify nvme_create_queue_pair() arguments

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:58:59PM +0200, Philippe Mathieu-Daudé wrote:
> nvme_create_queue_pair() doesn't require BlockDriverState anymore.
> Replace it by BDRVNVMeState and AioContext to simplify.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 13 +++--
>  1 file changed, 7 insertions(+), 6 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 456fe61f5ea..1f67e888c84 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -208,12 +208,12 @@ static void nvme_free_req_queue_cb(void *opaque)
>  qemu_mutex_unlock(&q->lock);
>  }
>  
> -static NVMeQueuePair *nvme_create_queue_pair(BlockDriverState *bs,
> +static NVMeQueuePair *nvme_create_queue_pair(BDRVNVMeState *s,
> + AioContext *aio_context,
>   int idx, int size,
>   Error **errp)
>  {
>  int i, r;
> -BDRVNVMeState *s = bs->opaque;
>  Error *local_err = NULL;
>  NVMeQueuePair *q;
>  uint64_t prp_list_iova;
> @@ -232,8 +232,7 @@ static NVMeQueuePair 
> *nvme_create_queue_pair(BlockDriverState *bs,
>  q->s = s;
>  q->index = idx;
>  qemu_co_queue_init(&q->free_req_queue);
> -q->completion_bh = aio_bh_new(bdrv_get_aio_context(bs),
> -  nvme_process_completion_bh, q);
> +q->completion_bh = aio_bh_new(aio_context, nvme_process_completion_bh, 
> q);
>  r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,
>s->page_size * NVME_NUM_REQS,
>false, &prp_list_iova);
> @@ -637,7 +636,8 @@ static bool nvme_add_io_queue(BlockDriverState *bs, Error 
> **errp)
>  NvmeCmd cmd;
>  int queue_size = NVME_QUEUE_SIZE;
>  
> -q = nvme_create_queue_pair(bs, n, queue_size, errp);
> +q = nvme_create_queue_pair(s, bdrv_get_aio_context(bs),
> +   n, queue_size, errp);
>  if (!q) {
>  return false;
>  }
> @@ -683,6 +683,7 @@ static int nvme_init(BlockDriverState *bs, const char 
> *device, int namespace,
>   Error **errp)
>  {
>  BDRVNVMeState *s = bs->opaque;
> +AioContext *aio_context = bdrv_get_aio_context(bs);
>  int ret;
>  uint64_t cap;
>  uint64_t timeout_ms;
> @@ -743,7 +744,7 @@ static int nvme_init(BlockDriverState *bs, const char 
> *device, int namespace,
>  
>  /* Set up admin queue. */
>  s->queues = g_new(NVMeQueuePair *, 1);
> -s->queues[INDEX_ADMIN] = nvme_create_queue_pair(bs, 0,
> +s->queues[INDEX_ADMIN] = nvme_create_queue_pair(s, aio_context, 0,
>NVME_QUEUE_SIZE,
>errp);
>  if (!s->queues[INDEX_ADMIN]) {
> -- 
> 2.26.2
> 
> 




[PATCH v3] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Daniel P . Berrangé
Meson requires the build dir to be separate from the source tree. Many
people are used to just running "./configure && make" though and the
meson conversion breaks that.

This introduces some backcompat support to make it appear as if an
"in source tree" build is being done, but with the results in the
"build/" directory. This allows "./configure && make" to work as it
did historically, albeit with the output binaries staying under build/.

Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrangé 
---

Changed in v3:

 - remove bashism  ==
 - avoid need for quoting when generating GNUmakefile
 - change line breaks in GNUmakefile for clarity

 .gitignore |  2 ++
 configure  | 52 +---
 2 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/.gitignore b/.gitignore
index 92311284ef..4ccb9ed975 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,5 @@
+/GNUmakefile
+/build/
 /.doctrees
 /config-devices.*
 /config-all-devices.*
diff --git a/configure b/configure
index cc5f58f31a..4e5fe33211 100755
--- a/configure
+++ b/configure
@@ -11,6 +11,55 @@ unset CLICOLOR_FORCE GREP_OPTIONS
 # Don't allow CCACHE, if present, to use cached results of compile tests!
 export CCACHE_RECACHE=yes
 
+# make source path absolute
+source_path=$(cd "$(dirname -- "$0")"; pwd)
+
+if test "$PWD" = "$source_path"
+then
+echo "Using './build' as the directory for build output"
+
+MARKER=build/auto-created-by-configure
+
+if test -e build
+then
+if test -f $MARKER
+then
+   rm -rf build
+else
+echo "ERROR: ./build dir already exists and was not previously 
created by configure"
+exit 1
+fi
+fi
+
+mkdir build
+touch $MARKER
+
+cat > GNUmakefile <<'EOF'
+# This file is auto-generated by configure to support in-source tree
+# 'make' command invocation
+
+ifeq ($(MAKECMDGOALS),)
+recurse: all
+endif
+
+.NOTPARALLEL: %
+%: force
+   @echo 'changing dir to build for $(MAKE) "$(MAKECMDGOALS)"...'
+   @$(MAKE) -C build -f Makefile $(MAKECMDGOALS)
+   @if test "$(MAKECMDGOALS)" = "distclean" && \
+   test -e build/auto-created-by-configure ; \
+   then \
+   rm -rf build GNUmakefile ; \
+   fi
+force: ;
+.PHONY: force
+GNUmakefile: ;
+
+EOF
+cd build
+exec $source_path/configure "$@"
+fi
+
 # Temporary directory used for files created while
 # configure runs. Since it is in the build directory
 # we can safely blow away any previous version of it
@@ -297,9 +346,6 @@ ld_has() {
 $ld --help 2>/dev/null | grep ".$1" >/dev/null 2>&1
 }
 
-# make source path absolute
-source_path=$(cd "$(dirname -- "$0")"; pwd)
-
 if printf %s\\n "$source_path" "$PWD" | grep -q "[[:space:]:]";
 then
   error_exit "main directory cannot contain spaces nor colons"
-- 
2.26.2




[PULL v8 001/152] oss-fuzz/build: remove LIB_FUZZING_ENGINE

2020-08-21 Thread Paolo Bonzini
Meson build scripts will only include qemu-fuzz-TARGET rules if configured
with --enable-fuzzing, and that takes care of adding -fsanitize=fuzzer.
Therefore we can just specify the configure option and stop modifying
the CFLAGS and CONFIG_FUZZ options in the "make" invocation.

Signed-off-by: Paolo Bonzini 
---
 scripts/oss-fuzz/build.sh | 16 +---
 1 file changed, 5 insertions(+), 11 deletions(-)

diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh
index a07b3022e8..52430cb620 100755
--- a/scripts/oss-fuzz/build.sh
+++ b/scripts/oss-fuzz/build.sh
@@ -20,7 +20,7 @@
 # e.g.
 # $CXX $CXXFLAGS -std=c++11 -Iinclude \
 # /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \
-# $LIB_FUZZING_ENGINE /path/to/library.a
+# -fsanitize=fuzzer /path/to/library.a
 
 fatal () {
 echo "Error : ${*}, exiting."
@@ -54,10 +54,6 @@ mkdir -p $OSS_FUZZ_BUILD_DIR || fatal "mkdir 
$OSS_FUZZ_BUILD_DIR failed"
 cd $OSS_FUZZ_BUILD_DIR || fatal "cd $OSS_FUZZ_BUILD_DIR failed"
 
 
-if [ -z ${LIB_FUZZING_ENGINE+x} ]; then
-LIB_FUZZING_ENGINE="-fsanitize=fuzzer"
-fi
-
 if [ -z ${OUT+x} ]; then
 DEST_DIR=$(realpath "./DEST_DIR")
 else
@@ -67,14 +63,12 @@ fi
 mkdir -p "$DEST_DIR/lib/"  # Copy the shared libraries here
 
 # Build once to get the list of dynamic lib paths, and copy them over
-../configure --disable-werror --cc="$CC" --cxx="$CXX" \
+../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \
 --extra-cflags="$EXTRA_CFLAGS" --target-list="i386-softmmu"
 
-if ! make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" "-j$(nproc)" \
-i386-softmmu/fuzz; then
+if ! make "-j$(nproc)" i386-softmmu/fuzz; then
 fatal "Build failed. Please specify a compiler with fuzzing support"\
-  "using the \$CC and \$CXX environemnt variables, or specify a"\
-  "\$LIB_FUZZING_ENGINE compatible with your compiler"\
+  "using the \$CC and \$CXX environemnt variables"\
   "\nFor example: CC=clang CXX=clang++ $0"
 fi
 
@@ -87,7 +81,7 @@ rm ./i386-softmmu/qemu-fuzz-i386
 ../configure --bindir="$DEST_DIR" --datadir="$DEST_DIR/data/" --disable-werror 
\
 --cc="$CC" --cxx="$CXX" --extra-cflags="$EXTRA_CFLAGS" \
 --extra-ldflags="-Wl,-rpath,'\$\$ORIGIN/lib'"
-make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" "-j$(nproc)" i386-softmmu/fuzz
+make "-j$(nproc)" i386-softmmu/fuzz
 
 # Copy over the datadir
 cp  -r ../pc-bios/ "$DEST_DIR/pc-bios"
-- 
2.26.2





[PULL v8 002/152] optionrom: simplify Makefile

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Make it independent from the rules.mak, and clean up to use pattern rules.

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 pc-bios/optionrom/Makefile | 67 +++---
 1 file changed, 41 insertions(+), 26 deletions(-)

diff --git a/pc-bios/optionrom/Makefile b/pc-bios/optionrom/Makefile
index e33a24da0d..51cb6ca9d8 100644
--- a/pc-bios/optionrom/Makefile
+++ b/pc-bios/optionrom/Makefile
@@ -1,13 +1,16 @@
-all: build-all
+CURRENT_MAKEFILE := $(realpath $(word $(words 
$(MAKEFILE_LIST)),$(MAKEFILE_LIST)))
+SRC_DIR := $(dir $(CURRENT_MAKEFILE))
+TOPSRC_DIR := $(SRC_DIR)/../..
+VPATH = $(SRC_DIR)
+
+all: multiboot.bin linuxboot.bin linuxboot_dma.bin kvmvapic.bin pvh.bin
 # Dummy command so that make thinks it has done something
@true
 
 include ../../config-host.mak
-include $(SRC_PATH)/rules.mak
-
-$(call set-vpath, $(SRC_PATH)/pc-bios/optionrom)
 
-.PHONY : all clean build-all
+quiet-command = $(if $(V),$1,$(if $(2),@printf "  %-7s %s\n" $2 $3 && $1, @$1))
+cc-option = $(if $(shell $(CC) $1 -S -o /dev/null -xc /dev/null >/dev/null 
2>&1 && echo OK), $1, $2)
 
 # Compiling with no optimization creates ROMs that are too large
 ifeq ($(lastword $(filter -O%, -O0 $(CFLAGS))),-O0)
@@ -15,48 +18,60 @@ override CFLAGS += -O2
 endif
 override CFLAGS += -march=i486
 
-# Drop -fstack-protector and the like
-QEMU_CFLAGS := $(filter -W%, $(QEMU_CFLAGS)) $(CFLAGS_NOPIE) -ffreestanding
-QEMU_CFLAGS += $(call cc-option, $(QEMU_CFLAGS), -fno-stack-protector)
-QEMU_CFLAGS += $(call cc-option, $(QEMU_CFLAGS), -m16)
-ifeq ($(filter -m16, $(QEMU_CFLAGS)),)
+# Flags for dependency generation
+override CPPFLAGS += -MMD -MP -MT $@ -MF $(@D)/$(*F).d
+
+override CFLAGS += $(filter -W%, $(QEMU_CFLAGS))
+override CFLAGS += $(CFLAGS_NOPIE) -ffreestanding -I$(TOPSRC_DIR)/include
+override CFLAGS += $(call cc-option, -fno-stack-protector)
+override CFLAGS += $(call cc-option, -m16)
+
+ifeq ($(filter -m16, $(CFLAGS)),)
 # Attempt to work around compilers that lack -m16 (GCC <= 4.8, clang <= ??)
 # On GCC we add -fno-toplevel-reorder to keep the order of asm blocks with
 # respect to the rest of the code.  clang does not have -fno-toplevel-reorder,
 # but it places all asm blocks at the beginning and we're relying on it for
 # the option ROM header.  So just force clang not to use the integrated
 # assembler, which doesn't support .code16gcc.
-QEMU_CFLAGS += $(call cc-option, $(QEMU_CFLAGS), -fno-toplevel-reorder)
-QEMU_CFLAGS += $(call cc-option, $(QEMU_CFLAGS), -no-integrated-as)
-QEMU_CFLAGS += -m32 -include $(SRC_PATH)/pc-bios/optionrom/code16gcc.h
+override CFLAGS += $(call cc-option, -fno-toplevel-reorder)
+override CFLAGS += $(call cc-option, -no-integrated-as)
+override CFLAGS += -m32 -include $(SRC_DIR)/code16gcc.h
 endif
 
-QEMU_INCLUDES += -I$(SRC_PATH)
-
 Wa = -Wa,
-ASFLAGS += -32
-QEMU_CFLAGS += $(call cc-c-option, $(QEMU_CFLAGS), $(Wa)-32)
+override ASFLAGS += -32
+override CFLAGS += $(call cc-option, $(Wa)-32)
 
-build-all: multiboot.bin linuxboot.bin linuxboot_dma.bin kvmvapic.bin pvh.bin
 
-# suppress auto-removal of intermediate files
-.SECONDARY:
+LD_I386_EMULATION ?= elf_i386
+override LDFLAGS = -m $(LD_I386_EMULATION) -T $(SRC_DIR)/flat.lds
+override LDFLAGS += $(LDFLAGS_NOPIE)
 
+all: multiboot.bin linuxboot.bin linuxboot_dma.bin kvmvapic.bin pvh.bin
+
+pvh.img: pvh.o pvh_main.o
 
 %.o: %.S
-   $(call quiet-command,$(CPP) $(QEMU_INCLUDES) $(QEMU_DGFLAGS) -c -o - $< 
| $(AS) $(ASFLAGS) -o $@,"AS","$(TARGET_DIR)$@")
+   $(call quiet-command,$(CPP) $(CPPFLAGS) -c -o - $< | $(AS) $(ASFLAGS) 
-o $@,"AS","$@")
 
-pvh.img: pvh.o pvh_main.o
-   $(call quiet-command,$(LD) $(LDFLAGS_NOPIE) -m $(LD_I386_EMULATION) -T 
$(SRC_PATH)/pc-bios/optionrom/flat.lds -s -o $@ $^,"BUILD","$(TARGET_DIR)$@")
+%.o: %.c
+   $(call quiet-command,$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@,"CC","$@")
 
 %.img: %.o
-   $(call quiet-command,$(LD) $(LDFLAGS_NOPIE) -m $(LD_I386_EMULATION) -T 
$(SRC_PATH)/pc-bios/optionrom/flat.lds -s -o $@ $<,"BUILD","$(TARGET_DIR)$@")
+   $(call quiet-command,$(LD) $(LDFLAGS) -s -o $@ $^,"BUILD","$@")
 
 %.raw: %.img
-   $(call quiet-command,$(OBJCOPY) -O binary -j .text $< 
$@,"BUILD","$(TARGET_DIR)$@")
+   $(call quiet-command,$(OBJCOPY) -O binary -j .text $< $@,"BUILD","$@")
 
 %.bin: %.raw
-   $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/signrom.py $< 
$@,"SIGN","$(TARGET_DIR)$@")
+   $(call quiet-command,$(PYTHON) $(TOPSRC_DIR)/scripts/signrom.py $< 
$@,"SIGN","$@")
+
+include $(wildcard *.d)
 
 clean:
rm -f *.o *.d *.raw *.img *.bin *~
+
+# suppress auto-removal of intermediate files
+.SECONDARY:
+
+.PHONY: all clean
-- 
2.26.2





[PULL v8 016/152] configure: add support for pseudo-"in source tree" builds

2020-08-21 Thread Paolo Bonzini
From: Daniel P. Berrangé 

Meson requires the build dir to be separate from the source tree. Many
people are used to just running "./configure && make" though and the
meson conversion breaks that.

This introduces some backcompat support to make it appear as if an
"in source tree" build is being done, but with the the results in the
"build/" directory. This allows "./configure && make" to work as it
did historically, albeit with the output binaries staying under build/.

Signed-off-by: Daniel P. Berrangé 
Signed-off-by: Paolo Bonzini 
---
 .gitignore |  2 ++
 configure  | 48 +---
 2 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/.gitignore b/.gitignore
index f8b3cd6fd5..d1e5e06242 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,5 @@
+/GNUmakefile
+/build/
 /.doctrees
 /config-devices.*
 /config-all-devices.*
diff --git a/configure b/configure
index 35d6492343..cc04a560a0 100755
--- a/configure
+++ b/configure
@@ -11,6 +11,51 @@ unset CLICOLOR_FORCE GREP_OPTIONS
 # Don't allow CCACHE, if present, to use cached results of compile tests!
 export CCACHE_RECACHE=yes
 
+# make source path absolute
+source_path=$(cd "$(dirname -- "$0")"; pwd)
+
+if test "$PWD" == "$source_path"
+then
+echo "Using './build' as the directory for build output"
+
+MARKER=build/auto-created-by-configure
+
+if test -e build
+then
+   if test -f $MARKER
+   then
+  rm -rf build
+   else
+   echo "ERROR: ./build dir already exists and was not previously 
created by configure"
+   exit 1
+   fi
+fi
+
+mkdir build
+touch $MARKER
+
+cat > GNUmakefile /dev/null 2>&1
 }
 
-# make source path absolute
-source_path=$(cd "$(dirname -- "$0")"; pwd)
-
 if printf %s\\n "$source_path" "$PWD" | grep -q "[[:space:]:]";
 then
   error_exit "main directory cannot contain spaces nor colons"
-- 
2.26.2





[PULL v8 007/152] build-sys hack: ensure target directory is there

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

By removing some unnest-vars calls, we miss some directory creation
that may be required by some/dir/object.d.

This will go away once everything is converted to Meson.

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 rules.mak | 1 +
 1 file changed, 1 insertion(+)

diff --git a/rules.mak b/rules.mak
index 694865b63e..56ba540a32 100644
--- a/rules.mak
+++ b/rules.mak
@@ -66,6 +66,7 @@ expand-objs = $(strip $(sort $(filter %.o,$1)) \
   $(filter-out %.o %.mo,$1))
 
 %.o: %.c
+   @mkdir -p $(dir $@)
$(call quiet-command,$(CC) $(QEMU_LOCAL_INCLUDES) $(QEMU_INCLUDES) \
   $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) \
   -c -o $@ $<,"CC","$(TARGET_DIR)$@")
-- 
2.26.2





[PULL v8 000/152] Meson-based build system

2020-08-21 Thread Paolo Bonzini
The following changes since commit 1d806cef0e38b5db8347a8e12f214d543204a314:

  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-08-11' 
into staging (2020-08-19 22:19:11 +0100)

are available in the Git repository at:

  https://gitlab.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 9cc97511bbc6729875d0f9366fec5babde4349bb:

  docs: convert build system documentation to rST (2020-08-21 06:19:12 -0400)


New build system.

Missing:
* converting configure tests
* converting unit tests
* converting some remaining parts of the installation


Daniel P. Berrangé (1):
  configure: add support for pseudo-"in source tree" builds

Marc-André Lureau (90):
  optionrom: simplify Makefile
  build-sys hack: ensure target directory is there
  configure: expand path variables for meson configure
  configure: generate Meson cross file
  build-sys hack: link with whole .fa archives
  build-sys: add meson submodule
  meson: enable pie
  meson: use coverage option
  meson: add remaining generated tcg trace helpers
  meson: add version.o
  contrib/vhost-user-input: convert to meson
  contrib/vhost-user-gpu: convert to meson
  contrib/ivshmem: convert to meson
  contrib/elf2dmp: convert to meson
  meson: add macos dependencies
  meson: convert vss-win32
  meson: add msi generation
  meson: add qemu-bridge-helper
  meson: add qemu-keymap
  meson: add qemu-edid
  meson: add virtfs-proxy-helper
  meson: keymap-gen
  meson: generate qemu-version.h
  meson: generate shader headers
  meson: generate hxtool files
  meson: handle edk2 bios and descriptors
  meson: convert qom directory to Meson (tools part)
  meson: convert authz directory to Meson
  meson: convert crypto directory to Meson
  meson: convert io directory to Meson
  meson: convert target/s390x/gen-features.h
  meson: add modules infrastructure
  meson: convert chardev directory to Meson (tools part)
  meson: convert block
  meson: qemu-{img,io,nbd}
  meson: qemu-pr-helper
  meson: convert ui directory to Meson
  meson: convert trace/
  meson: convert dump/
  meson: convert replay directory to Meson
  meson: convert migration directory to Meson
  meson: convert net directory to Meson
  meson: convert backends directory to Meson
  meson: convert fsdev/
  meson: convert disas directory to Meson
  meson: convert qapi-specific to meson
  meson: convert hw/xen
  meson: convert hw/core
  meson: convert hw/smbios
  meson: convert hw/mem
  meson: convert hw/watchdog
  meson: convert hw/virtio
  meson: convert hw/vfio
  meson: convert hw/ssi
  meson: convert hw/sd
  meson: convert hw/scsi
  meson: convert hw/pcmcia
  meson: convert hw/pci-host
  meson: convert hw/pci-bridge
  meson: convert hw/pci
  meson: convert hw/nvram
  meson: convert hw/rdma
  meson: convert hw/net
  meson: convert hw/misc
  meson: convert hw/isa
  meson: convert hw/ipmi
  meson: convert hw/ipack
  meson: convert hw/intc
  meson: convert hw/input
  meson: convert hw/ide
  meson: convert hw/i2c
  meson: convert hw/hyperv
  meson: convert hw/gpio
  meson: convert hw/dma
  meson: convert hw/display
  meson: convert hw/cpu
  meson: convert hw/char
  meson: convert hw/block
  meson: convert hw/audio
  meson: convert hw/adc
  meson: convert hw/acpi
  meson: convert hw/9pfs, cleanup
  meson: convert hw/arch*
  meson: accel
  meson: linux-user
  meson: bsd-user
  meson: cpu-emu
  meson: convert systemtap files
  rules.mak: remove version.o
  meson: convert po/

Paolo Bonzini (60):
  oss-fuzz/build: remove LIB_FUZZING_ENGINE
  trace: switch position of headers to what Meson requires
  meson: rename included C source files to .c.inc
  meson: rename .inc.h files to .h.inc
  tests/vm: do not pollute configure with --efi-aarch64
  tests/vm: check for Python YAML parser in the Makefile
  tests/docker: add test script for static linux-user builds
  nsis: use "make DESTDIR=" instead of "make prefix="
  configure: do not include $(...) variables in config-host.mak
  configure: prepare CFLAGS/CXXFLAGS/LDFLAGS for Meson
  tests/vm: include setuptools
  configure: integrate Meson in the build system
  meson: move summary to meson.build
  meson: add sparse support
  meson: add testsuite Makefile generator
  libqemuutil, qapi, trace: convert to meson
  contrib/libvhost-user: convert to Meson
  tools/virtiofsd: convert to Meson
  contrib/vhost-user-blk: convert to Meson
  vhost-user-scsi: add compatibility for libiscsi 1.9.0
   

[PULL v8 022/152] meson: enable pie

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 configure | 1 +
 1 file changed, 1 insertion(+)

diff --git a/configure b/configure
index 40c8462539..7c2a46ade0 100755
--- a/configure
+++ b/configure
@@ -8514,6 +8514,7 @@ NINJA=$PWD/ninjatool $meson setup \
 -Ddebug=$(if test "$debug_info" = yes; then echo true; else echo 
false; fi) \
 -Dwerror=$(if test "$werror" = yes; then echo true; else echo false; 
fi) \
 -Dstrip=$(if test "$strip_opt" = yes; then echo true; else echo false; 
fi) \
+-Db_pie=$(if test "$pie" = yes; then echo true; else echo false; fi) \
 $cross_arg \
 "$PWD" "$source_path"
 
-- 
2.26.2





[PULL v8 010/152] tests/docker: add test script for static linux-user builds

2020-08-21 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini 
---
 tests/docker/test-static | 24 
 1 file changed, 24 insertions(+)
 create mode 100755 tests/docker/test-static

diff --git a/tests/docker/test-static b/tests/docker/test-static
new file mode 100755
index 00..372ef6fac7
--- /dev/null
+++ b/tests/docker/test-static
@@ -0,0 +1,24 @@
+#!/bin/bash -e
+#
+# Compile QEMU user mode emulators as static binaries on Linux.
+#
+# Copyright (c) 2020 Red Hat Inc.
+#
+# Authors:
+#  Paolo Bonzini 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+build_qemu \
+--disable-system \
+--disable-tools \
+--disable-guest-agent \
+--disable-docs \
+--static
+install_qemu
-- 
2.26.2





[PULL v8 006/152] meson: rename .inc.h files to .h.inc

2020-08-21 Thread Paolo Bonzini
Make it consistent with '.c.inc' and '.rst.inc'.

Signed-off-by: Paolo Bonzini 
---
 include/exec/cpu-all.h   | 10 +-
 include/exec/memory.h| 12 ++--
 .../exec/{memory_ldst.inc.h => memory_ldst.h.inc}|  0
 ...ry_ldst_cached.inc.h => memory_ldst_cached.h.inc} |  0
 ...memory_ldst_phys.inc.h => memory_ldst_phys.h.inc} |  0
 target/s390x/cpu_features.c  |  2 +-
 target/s390x/cpu_features_def.h  |  2 +-
 ...cpu_features_def.inc.h => cpu_features_def.h.inc} |  0
 8 files changed, 13 insertions(+), 13 deletions(-)
 rename include/exec/{memory_ldst.inc.h => memory_ldst.h.inc} (100%)
 rename include/exec/{memory_ldst_cached.inc.h => memory_ldst_cached.h.inc} 
(100%)
 rename include/exec/{memory_ldst_phys.inc.h => memory_ldst_phys.h.inc} (100%)
 rename target/s390x/{cpu_features_def.inc.h => cpu_features_def.h.inc} (100%)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index fc403d456b..f6439c4705 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -189,13 +189,13 @@ extern unsigned long reserved_va;
 #define ARG1 as
 #define ARG1_DECLAddressSpace *as
 #define TARGET_ENDIANNESS
-#include "exec/memory_ldst.inc.h"
+#include "exec/memory_ldst.h.inc"
 
 #define SUFFIX   _cached_slow
 #define ARG1 cache
 #define ARG1_DECLMemoryRegionCache *cache
 #define TARGET_ENDIANNESS
-#include "exec/memory_ldst.inc.h"
+#include "exec/memory_ldst.h.inc"
 
 static inline void stl_phys_notdirty(AddressSpace *as, hwaddr addr, uint32_t 
val)
 {
@@ -207,17 +207,17 @@ static inline void stl_phys_notdirty(AddressSpace *as, 
hwaddr addr, uint32_t val
 #define ARG1 as
 #define ARG1_DECLAddressSpace *as
 #define TARGET_ENDIANNESS
-#include "exec/memory_ldst_phys.inc.h"
+#include "exec/memory_ldst_phys.h.inc"
 
 /* Inline fast path for direct RAM access.  */
 #define ENDIANNESS
-#include "exec/memory_ldst_cached.inc.h"
+#include "exec/memory_ldst_cached.h.inc"
 
 #define SUFFIX   _cached
 #define ARG1 cache
 #define ARG1_DECLMemoryRegionCache *cache
 #define TARGET_ENDIANNESS
-#include "exec/memory_ldst_phys.inc.h"
+#include "exec/memory_ldst_phys.h.inc"
 #endif
 
 /* page related stuff */
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 307e527835..0cfe987ab4 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -2133,12 +2133,12 @@ MemTxResult address_space_write_rom(AddressSpace *as, 
hwaddr addr,
 #define SUFFIX
 #define ARG1 as
 #define ARG1_DECLAddressSpace *as
-#include "exec/memory_ldst.inc.h"
+#include "exec/memory_ldst.h.inc"
 
 #define SUFFIX
 #define ARG1 as
 #define ARG1_DECLAddressSpace *as
-#include "exec/memory_ldst_phys.inc.h"
+#include "exec/memory_ldst_phys.h.inc"
 
 struct MemoryRegionCache {
 void *ptr;
@@ -2179,7 +2179,7 @@ struct MemoryRegionCache {
 #define SUFFIX   _cached_slow
 #define ARG1 cache
 #define ARG1_DECLMemoryRegionCache *cache
-#include "exec/memory_ldst.inc.h"
+#include "exec/memory_ldst.h.inc"
 
 /* Inline fast path for direct RAM access.  */
 static inline uint8_t address_space_ldub_cached(MemoryRegionCache *cache,
@@ -2205,15 +2205,15 @@ static inline void 
address_space_stb_cached(MemoryRegionCache *cache,
 }
 
 #define ENDIANNESS   _le
-#include "exec/memory_ldst_cached.inc.h"
+#include "exec/memory_ldst_cached.h.inc"
 
 #define ENDIANNESS   _be
-#include "exec/memory_ldst_cached.inc.h"
+#include "exec/memory_ldst_cached.h.inc"
 
 #define SUFFIX   _cached
 #define ARG1 cache
 #define ARG1_DECLMemoryRegionCache *cache
-#include "exec/memory_ldst_phys.inc.h"
+#include "exec/memory_ldst_phys.h.inc"
 
 /* address_space_cache_init: prepare for repeated access to a physical
  * memory region
diff --git a/include/exec/memory_ldst.inc.h b/include/exec/memory_ldst.h.inc
similarity index 100%
rename from include/exec/memory_ldst.inc.h
rename to include/exec/memory_ldst.h.inc
diff --git a/include/exec/memory_ldst_cached.inc.h 
b/include/exec/memory_ldst_cached.h.inc
similarity index 100%
rename from include/exec/memory_ldst_cached.inc.h
rename to include/exec/memory_ldst_cached.h.inc
diff --git a/include/exec/memory_ldst_phys.inc.h 
b/include/exec/memory_ldst_phys.h.inc
similarity index 100%
rename from include/exec/memory_ldst_phys.inc.h
rename to include/exec/memory_ldst_phys.h.inc
diff --git a/target/s390x/cpu_features.c b/target/s390x/cpu_features.c
index 9f817e3cfa..31ea8df246 100644
--- a/target/s390x/cpu_features.c
+++ b/target/s390x/cpu_features.c
@@ -23,7 +23,7 @@
 .desc = _DESC, \
 },
 static const S390FeatDef s390_features[S390_FEAT_MAX] = {
-#include "cpu_features_def.inc.h"
+#include "cpu_features_def.h.inc"
 };
 #undef DEF_FEAT
 
diff --git a/target/s390x/cpu_features_def.h b/target/s390x/cpu_features_def.h
index 412d356feb..87df31848e 100644
--- a/target/s3

[PULL v8 003/152] pc-bios/s390-ccw: do not use rules.mak

2020-08-21 Thread Paolo Bonzini
From: Thomas Huth 

Signed-off-by: Paolo Bonzini 
---
 pc-bios/s390-ccw/Makefile | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/pc-bios/s390-ccw/Makefile b/pc-bios/s390-ccw/Makefile
index 50bc880272..cc0f77baa6 100644
--- a/pc-bios/s390-ccw/Makefile
+++ b/pc-bios/s390-ccw/Makefile
@@ -3,10 +3,26 @@ all: build-all
@true
 
 include ../../config-host.mak
-include $(SRC_PATH)/rules.mak
 
+quiet-command = $(if $(V),$1,$(if $(2),@printf "  %-7s %s\n" $2 $3 && $1, @$1))
+cc-option = $(if $(shell $(CC) $1 -S -o /dev/null -xc /dev/null > /dev/null \
+ 2>&1 && echo OK), $1, $2)
+
+VPATH_SUFFIXES = %.c %.h %.S %.m %.mak %.sh %.rc Kconfig% %.json.in
+set-vpath = $(if $1,$(foreach PATTERN,$(VPATH_SUFFIXES),$(eval vpath 
$(PATTERN) $1)))
 $(call set-vpath, $(SRC_PATH)/pc-bios/s390-ccw)
 
+# Flags for dependency generation
+QEMU_DGFLAGS = -MMD -MP -MT $@ -MF $(@D)/$(*F).d
+
+%.o: %.c
+   $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) \
+  -c -o $@ $<,"CC","$(TARGET_DIR)$@")
+
+%.o: %.S
+   $(call quiet-command,$(CCAS) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) \
+  -c -o $@ $<,"CCAS","$(TARGET_DIR)$@")
+
 .PHONY : all clean build-all
 
 OBJECTS = start.o main.o bootmap.o jump2ipl.o sclp.o menu.o \
-- 
2.26.2





[PULL v8 009/152] tests/vm: check for Python YAML parser in the Makefile

2020-08-21 Thread Paolo Bonzini
No need to do it in the configure file if it is only used for a help message.

Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Paolo Bonzini 
---
 configure | 9 -
 tests/vm/Makefile.include | 6 +-
 2 files changed, 5 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index 1c17a0f27f..625c14c500 100755
--- a/configure
+++ b/configure
@@ -959,13 +959,6 @@ do
 fi
 done
 
-# Check for existence of python3 yaml, needed to
-# import yaml config files into vm-build.
-python_yaml="no"
-if $(python3 -c "import yaml" 2> /dev/null); then
-python_yaml="yes"
-fi
-
 : ${smbd=${SMBD-/usr/sbin/smbd}}
 
 # Default objcc to clang if available, otherwise use CC
@@ -6844,7 +6837,6 @@ if test "$docs" != "no"; then
 echo "sphinx-build  $sphinx_build"
 fi
 echo "genisoimage   $genisoimage"
-echo "python_yaml   $python_yaml"
 echo "slirp support $slirp $(echo_version $slirp $slirp_version)"
 if test "$slirp" != "no" ; then
 echo "smbd  $smbd"
@@ -7945,7 +7937,6 @@ echo "PYTHON=$python" >> $config_host_mak
 echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak
 echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak
 echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
-echo "PYTHON_YAML=$python_yaml" >> $config_host_mak
 echo "CC=$cc" >> $config_host_mak
 if $iasl -h > /dev/null 2>&1; then
   echo "IASL=$iasl" >> $config_host_mak
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index a599d1994d..61f893ffdc 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -17,6 +17,10 @@ IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
 
 .PRECIOUS: $(IMAGE_FILES)
 
+ifneq ($(PYTHON),)
+HAVE_PYTHON_YAML = $(shell $(PYTHON) -c "import yaml" 2> /dev/null && echo yes)
+endif
+
 # 'vm-help' target was historically named 'vm-test'
 vm-help vm-test:
@echo "vm-help: Test QEMU in preconfigured virtual machines"
@@ -56,7 +60,7 @@ endif
@echo "QEMU_LOCAL=1 - Use QEMU binary local to this 
build."
@echo "QEMU=/path/to/qemu- Change path to QEMU binary"
@echo "QEMU_IMG=/path/to/qemu-img- Change path to qemu-img tool"
-ifeq ($(PYTHON_YAML),yes)
+ifeq ($(HAVE_PYTHON_YAML),yes)
@echo "QEMU_CONFIG=/path/conf.yml   - Change path to VM 
configuration .yml file."
 else
@echo "(install python3-yaml to enable support for yaml file to 
configure a VM.)"
-- 
2.26.2





[PULL v8 021/152] meson: move summary to meson.build

2020-08-21 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini 
---
 configure   | 237 --
 meson.build | 245 
 2 files changed, 245 insertions(+), 237 deletions(-)

diff --git a/configure b/configure
index 3360f40a33..40c8462539 100755
--- a/configure
+++ b/configure
@@ -570,8 +570,6 @@ libdaxctl=""
 meson=""
 skip_meson=no
 
-supported_cpu="no"
-supported_os="no"
 bogus_os="no"
 malloc_trim=""
 
@@ -810,35 +808,27 @@ ARCH=
 # Note that this case should only have supported host CPUs, not guests.
 case "$cpu" in
   ppc|ppc64|s390x|sparc64|x32|riscv32|riscv64)
-supported_cpu="yes"
   ;;
   ppc64le)
 ARCH="ppc64"
-supported_cpu="yes"
   ;;
   i386|i486|i586|i686|i86pc|BePC)
 cpu="i386"
-supported_cpu="yes"
   ;;
   x86_64|amd64)
 cpu="x86_64"
-supported_cpu="yes"
   ;;
   armv*b|armv*l|arm)
 cpu="arm"
-supported_cpu="yes"
   ;;
   aarch64)
 cpu="aarch64"
-supported_cpu="yes"
   ;;
   mips*)
 cpu="mips"
-supported_cpu="yes"
   ;;
   sparc|sun4[cdmuv])
 cpu="sparc"
-supported_cpu="yes"
   ;;
   *)
 # This will result in either an error or falling back to TCI later
@@ -866,7 +856,6 @@ MINGW32*)
 audio_drv_list=""
   fi
   supported_os="yes"
-  pie="no"
 ;;
 GNU/kFreeBSD)
   bsd="yes"
@@ -884,7 +873,6 @@ FreeBSD)
   libs_qga="-lutil $libs_qga"
   netmap=""  # enable netmap autodetect
   HOST_VARIANT_DIR="freebsd"
-  supported_os="yes"
 ;;
 DragonFly)
   bsd="yes"
@@ -901,7 +889,6 @@ NetBSD)
   audio_possible_drivers="oss sdl"
   oss_lib="-lossaudio"
   HOST_VARIANT_DIR="netbsd"
-  supported_os="yes"
 ;;
 OpenBSD)
   bsd="yes"
@@ -909,7 +896,6 @@ OpenBSD)
   audio_drv_list="try-sdl"
   audio_possible_drivers="sdl"
   HOST_VARIANT_DIR="openbsd"
-  supported_os="yes"
 ;;
 Darwin)
   bsd="yes"
@@ -930,7 +916,6 @@ Darwin)
   # won't work when we're compiling with gcc as a C compiler.
   QEMU_CFLAGS="-DOS_OBJECT_USE_OBJC=0 $QEMU_CFLAGS"
   HOST_VARIANT_DIR="darwin"
-  supported_os="yes"
 ;;
 SunOS)
   solaris="yes"
@@ -961,7 +946,6 @@ Linux)
   linux_user="yes"
   kvm="yes"
   QEMU_INCLUDES="-isystem ${source_path}/linux-headers -I$PWD/linux-headers 
$QEMU_INCLUDES"
-  supported_os="yes"
   libudev="yes"
 ;;
 esac
@@ -5317,8 +5301,6 @@ EOF
 spice="yes"
 libs_softmmu="$libs_softmmu $spice_libs"
 QEMU_CFLAGS="$QEMU_CFLAGS $spice_cflags"
-spice_protocol_version=$($pkg_config --modversion spice-protocol)
-spice_server_version=$($pkg_config --modversion spice-server)
   else
 if test "$spice" = "yes" ; then
   feature_not_found "spice" \
@@ -6935,225 +6917,6 @@ QEMU_CFLAGS="$pixman_cflags $fdt_cflags $QEMU_CFLAGS"
 QEMU_LDFLAGS="$fdt_ldflags $QEMU_LDFLAGS"
 libs_softmmu="$pixman_libs $libs_softmmu"
 
-echo "Install prefix$prefix"
-echo "BIOS directory$(eval echo $qemu_datadir)"
-echo "firmware path $(eval echo $firmwarepath)"
-echo "binary directory  $(eval echo $bindir)"
-echo "library directory $(eval echo $libdir)"
-echo "module directory  $(eval echo $qemu_moddir)"
-echo "libexec directory $(eval echo $libexecdir)"
-echo "include directory $(eval echo $includedir)"
-echo "config directory  $(eval echo $sysconfdir)"
-if test "$mingw32" = "no" ; then
-echo "local state directory   $(eval echo $local_statedir)"
-echo "Manual directory  $(eval echo $mandir)"
-echo "ELF interp prefix $interp_prefix"
-else
-echo "local state directory   queried at runtime"
-echo "Windows SDK   $win_sdk"
-fi
-echo "Build directory   $(pwd)"
-echo "Source path   $source_path"
-echo "GIT binary$git"
-echo "GIT submodules$git_submodules"
-echo "C compiler$cc"
-echo "Host C compiler   $host_cc"
-echo "C++ compiler  $cxx"
-echo "Objective-C compiler $objcc"
-echo "ARFLAGS   $ARFLAGS"
-echo "CFLAGS$CFLAGS"
-echo "QEMU_CFLAGS   $QEMU_CFLAGS"
-echo "QEMU_LDFLAGS  $QEMU_LDFLAGS"
-echo "make  $make"
-echo "install   $install"
-echo "python$python ($python_version)"
-if test "$docs" != "no"; then
-echo "sphinx-build  $sphinx_build"
-fi
-echo "genisoimage   $genisoimage"
-echo "slirp support $slirp $(echo_version $slirp $slirp_version)"
-if test "$slirp" != "no" ; then
-echo "smbd  $smbd"
-fi
-echo "module support$modules"
-echo "alt path mod load $module_upgrades"
-echo "host CPU  $cpu"
-echo "host big endian   $bigendian"
-echo "target list   $target_list"
-echo "gprof enabled $gprof"
-echo "sparse enabled$sparse"
-echo "strip binaries$strip_opt"
-echo "profiler  $profiler"
-echo "static build  $static"
-echo "safe stack$safe_stack"
-if test "$darwin" = "yes" ; then
-echo "Cocoa support $cocoa"
-fi
-echo "SDL support   $sdl $(echo_version $sdl $sdlversion)"
-echo "SDL image support $sdl_image"
-echo "GTK support   $gtk $(echo_version $gtk $gtk_version)"
-echo "GTK GL support$gtk_gl"
-echo "VTE 

[PULL v8 015/152] tests/vm: include setuptools

2020-08-21 Thread Paolo Bonzini
They are a dependency of Meson, so install them.

Signed-off-by: Paolo Bonzini 
---
 tests/vm/freebsd | 1 +
 tests/vm/netbsd  | 1 +
 tests/vm/openbsd | 1 +
 3 files changed, 3 insertions(+)

diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 29252fa4a6..b34b14fc53 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -33,6 +33,7 @@ class FreeBSDVM(basevm.BaseVM):
 "pkgconf",
 "bzip2",
 "python37",
+"py37-setuptools",
 
 # gnu tools
 "bash",
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 2e87199211..93d48b6fdd 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -31,6 +31,7 @@ class NetBSDVM(basevm.BaseVM):
 "pkgconf",
 "xz",
 "python37",
+"py37-setuptools",
 
 # gnu tools
 "bash",
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index dfe633e453..7e27fda642 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -30,6 +30,7 @@ class OpenBSDVM(basevm.BaseVM):
 "git",
 "pkgconf",
 "bzip2", "xz",
+"py3-setuptools",
 
 # gnu tools
 "bash",
-- 
2.26.2





[PULL v8 018/152] configure: generate Meson cross file

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 configure   | 72 ++---
 meson.build |  7 +-
 2 files changed, 69 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index ce32c568f8..a82045352c 100755
--- a/configure
+++ b/configure
@@ -8626,21 +8626,75 @@ echo >> "$iotests_common_env"
 echo "export PYTHON='$python'" >> "$iotests_common_env"
 
 if test "$skip_meson" = no; then
+cross="config-meson.cross.new"
+meson_quote() {
+echo "['$(echo $* | sed "s/ /','/g")']"
+}
+
+echo "# Automatically generated by configure - do not modify" > $cross
+echo "[properties]" >> $cross
+test -z "$cxx" && echo "link_language = 'c'" >> $cross
+echo "[binaries]" >> $cross
+echo "c = $(meson_quote $cc)" >> $cross
+test -n "$cxx" && echo "cpp = $(meson_quote $cxx)" >> $cross
+echo "ar = $(meson_quote $ar)" >> $cross
+echo "nm = $(meson_quote $nm)" >> $cross
+echo "pkgconfig = $(meson_quote $pkg_config_exe)" >> $cross
+echo "ranlib = $(meson_quote $ranlib)" >> $cross
+echo "strip = $(meson_quote $strip)" >> $cross
+echo "windres = $(meson_quote $windres)" >> $cross
+if test -n "$cross_prefix"; then
+cross_arg="--cross-file config-meson.cross"
+# Hack: Meson expects an absolute path for the *build* machine
+# for the prefix, so add a slash in front of a Windows path that
+# includes a drive letter.
+#
+# See https://github.com/mesonbuild/meson/issues/7577.
+echo "[host_machine]" >> $cross
+if test "$mingw32" = "yes" ; then
+echo "system = 'windows'" >> $cross
+case $prefix in
+?:*) pre_prefix=/ ;;
+esac
+fi
+case "$ARCH" in
+i386|x86_64)
+echo "cpu_family = 'x86'" >> $cross
+;;
+ppc64le)
+echo "cpu_family = 'ppc64'" >> $cross
+;;
+*)
+echo "cpu_family = '$ARCH'" >> $cross
+;;
+esac
+echo "cpu = '$cpu'" >> $cross
+if test "$bigendian" = "yes" ; then
+echo "endian = 'big'" >> $cross
+else
+echo "endian = 'little'" >> $cross
+fi
+else
+cross_arg="--native-file config-meson.cross"
+fi
+mv $cross config-meson.cross
+
 rm -rf meson-private meson-info meson-logs
 NINJA=$PWD/ninjatool $meson setup \
---prefix "$prefix" \
---libdir "$libdir" \
---libexecdir "$libexecdir" \
---bindir "$bindir" \
---includedir "$includedir" \
---datadir "$datadir" \
---mandir "$mandir" \
---sysconfdir "$sysconfdir" \
---localstatedir "$local_statedir" \
+--prefix "${pre_prefix}$prefix" \
+--libdir "${pre_prefix}$libdir" \
+--libexecdir "${pre_prefix}$libexecdir" \
+--bindir "${pre_prefix}$bindir" \
+--includedir "${pre_prefix}$includedir" \
+--datadir "${pre_prefix}$datadir" \
+--mandir "${pre_prefix}$mandir" \
+--sysconfdir "${pre_prefix}$sysconfdir" \
+--localstatedir "${pre_prefix}$local_statedir" \
 -Doptimization=$(if test "$debug" = yes; then echo 0; else echo 2; fi) 
\
 -Ddebug=$(if test "$debug_info" = yes; then echo true; else echo 
false; fi) \
 -Dwerror=$(if test "$werror" = yes; then echo true; else echo false; 
fi) \
 -Dstrip=$(if test "$strip_opt" = yes; then echo true; else echo false; 
fi) \
+$cross_arg \
 "$PWD" "$source_path"
 
 if test "$?" -ne 0 ; then
diff --git a/meson.build b/meson.build
index 613983e464..5ad85a7314 100644
--- a/meson.build
+++ b/meson.build
@@ -15,7 +15,12 @@ 
add_project_link_arguments(config_host['QEMU_LDFLAGS'].split(),
 add_project_arguments(config_host['QEMU_INCLUDES'].split(),
   language: ['c', 'cpp', 'objc'])
 
-add_languages('cpp', required: false, native: false)
+python = import('python').find_installation()
+
+link_language = meson.get_external_property('link_language', 'cpp')
+if link_language == 'cpp'
+  add_languages('cpp', required: true, native: false)
+endif
 if host_machine.system() == 'darwin'
   add_languages('objc', required: false, native: false)
 endif
-- 
2.26.2





[PULL v8 011/152] nsis: use "make DESTDIR=" instead of "make prefix="

2020-08-21 Thread Paolo Bonzini
The next patch will prevent modifying the prefix on "make install".  Adjust the
creation of the installer.

Signed-off-by: Paolo Bonzini 
---
 Makefile | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 5596482dbd..ec12101a84 100644
--- a/Makefile
+++ b/Makefile
@@ -1188,16 +1188,16 @@ installer: $(INSTALLER)
 
 INSTDIR=/tmp/qemu-nsis
 
-$(INSTALLER): install-doc $(SRC_PATH)/qemu.nsi
-   $(MAKE) install prefix=${INSTDIR}
+$(INSTALLER): $(SRC_PATH)/qemu.nsi
+   $(MAKE) install DESTDIR=${INSTDIR}
 ifdef SIGNCODE
-   (cd ${INSTDIR}; \
+   (cd ${INSTDIR}/${bindir}; \
  for i in *.exe; do \
$(SIGNCODE) $${i}; \
  done \
 )
 endif # SIGNCODE
-   (cd ${INSTDIR}; \
+   (cd ${INSTDIR}/${bindir}; \
  for i in qemu-system-*.exe; do \
arch=$${i%.exe}; \
arch=$${arch#qemu-system-}; \
@@ -1206,11 +1206,11 @@ endif # SIGNCODE
echo File \"\$${BINDIR}\\$$i\"; \
echo SectionEnd; \
  done \
-) >${INSTDIR}/system-emulations.nsh
+) >${INSTDIR}/${bindir}/system-emulations.nsh
makensis $(nsisflags) \
 $(if $(BUILD_DOCS),-DCONFIG_DOCUMENTATION="y") \
 $(if $(CONFIG_GTK),-DCONFIG_GTK="y") \
--DBINDIR="${INSTDIR}" \
+-DBINDIR="${INSTDIR}/${bindir}" \
 $(if $(DLL_PATH),-DDLLDIR="$(DLL_PATH)") \
 -DSRCDIR="$(SRC_PATH)" \
 -DOUTFILE="$(INSTALLER)" \
-- 
2.26.2





[PULL v8 028/152] meson: add version.o

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 meson.build | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meson.build b/meson.build
index 4e9eb5da5c..feb3c8a30c 100644
--- a/meson.build
+++ b/meson.build
@@ -49,8 +49,14 @@ targetos = host_machine.system()
 m = cc.find_library('m', required: false)
 util = cc.find_library('util', required: false)
 socket = []
+version_res = []
 if targetos == 'windows'
   socket = cc.find_library('ws2_32')
+
+  win = import('windows')
+  version_res = win.compile_resources('version.rc',
+  depend_files: 
files('pc-bios/qemu-nsis.ico'),
+  include_directories: 
include_directories('.'))
 endif
 glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
   link_args: config_host['GLIB_LIBS'].split())
@@ -235,7 +241,7 @@ libqemuutil = static_library('qemuutil',
  sources: util_ss.sources() + stub_ss.sources() + 
genh,
  dependencies: [util_ss.dependencies(), m, glib, 
socket])
 qemuutil = declare_dependency(link_with: libqemuutil,
-  sources: genh)
+  sources: genh + version_res)
 
 summary_info = {}
 summary_info += {'Install prefix':config_host['prefix']}
-- 
2.26.2





[PULL v8 008/152] tests/vm: do not pollute configure with --efi-aarch64

2020-08-21 Thread Paolo Bonzini
Just make EFI_AARCH64 a variable in the makefile that defaults to the efi
firmware included with QEMU.  It can be redefined on the "make" command
line.

Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Paolo Bonzini 
---
 configure | 19 ---
 tests/vm/Makefile.include |  2 ++
 2 files changed, 2 insertions(+), 19 deletions(-)

diff --git a/configure b/configure
index 2acc4d1465..1c17a0f27f 100755
--- a/configure
+++ b/configure
@@ -418,7 +418,6 @@ prefix="/usr/local"
 mandir="\${prefix}/share/man"
 datadir="\${prefix}/share"
 firmwarepath="\${prefix}/share/qemu-firmware"
-efi_aarch64=""
 qemu_docdir="\${prefix}/share/doc/qemu"
 bindir="\${prefix}/bin"
 libdir="\${prefix}/lib"
@@ -1109,8 +1108,6 @@ for opt do
   ;;
   --firmwarepath=*) firmwarepath="$optarg"
   ;;
-  --efi-aarch64=*) efi_aarch64="$optarg"
-  ;;
   --host=*|--build=*|\
   --disable-dependency-tracking|\
   --sbindir=*|--sharedstatedir=*|\
@@ -3650,20 +3647,6 @@ EOF
   fi
 fi
 
-
-# efi-aarch64 probe
-# Check for efi files needed by aarch64 VMs.
-# By default we will use the efi included with QEMU.
-# Allow user to override the path for efi also.
-if ! test -f "$efi_aarch64"; then
-  if test -f $source_path/pc-bios/edk2-aarch64-code.fd.bz2; then
-# valid after build
-efi_aarch64=$PWD/pc-bios/edk2-aarch64-code.fd
-  else
-efi_aarch64=""
-  fi
-fi
-
 ##
 # libcap-ng library probe
 if test "$cap_ng" != "no" ; then
@@ -6861,7 +6844,6 @@ if test "$docs" != "no"; then
 echo "sphinx-build  $sphinx_build"
 fi
 echo "genisoimage   $genisoimage"
-echo "efi_aarch64   $efi_aarch64"
 echo "python_yaml   $python_yaml"
 echo "slirp support $slirp $(echo_version $slirp $slirp_version)"
 if test "$slirp" != "no" ; then
@@ -7963,7 +7945,6 @@ echo "PYTHON=$python" >> $config_host_mak
 echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak
 echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak
 echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
-echo "EFI_AARCH64=$efi_aarch64" >> $config_host_mak
 echo "PYTHON_YAML=$python_yaml" >> $config_host_mak
 echo "CC=$cc" >> $config_host_mak
 if $iasl -h > /dev/null 2>&1; then
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index f21948c46a..a599d1994d 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -2,6 +2,8 @@
 
 .PHONY: vm-build-all vm-clean-all
 
+EFI_AARCH64 = $(wildcard $(BUILD_DIR)/pc-bios/edk2-aarch64-code.fd)
+
 IMAGES := freebsd netbsd openbsd centos fedora
 ifneq ($(GENISOIMAGE),)
 IMAGES += ubuntu.i386 centos
-- 
2.26.2





[PULL v8 013/152] configure: expand path variables for meson configure

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 configure | 35 ---
 1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/configure b/configure
index 606c327cca..373bb22287 100755
--- a/configure
+++ b/configure
@@ -415,16 +415,7 @@ LDFLAGS_SHARED="-shared"
 modules="no"
 module_upgrades="no"
 prefix="/usr/local"
-mandir="\${prefix}/share/man"
-datadir="\${prefix}/share"
 firmwarepath="\${prefix}/share/qemu-firmware"
-qemu_docdir="\${prefix}/share/doc/qemu"
-bindir="\${prefix}/bin"
-libdir="\${prefix}/lib"
-libexecdir="\${prefix}/libexec"
-includedir="\${prefix}/include"
-sysconfdir="\${prefix}/etc"
-local_statedir="\${prefix}/var"
 confsuffix="/qemu"
 slirp=""
 oss_lib=""
@@ -979,12 +970,6 @@ if test "$mingw32" = "yes" ; then
 LIBS="-liberty $LIBS"
   fi
   prefix="c:/Program Files/QEMU"
-  mandir="\${prefix}"
-  datadir="\${prefix}"
-  qemu_docdir="\${prefix}"
-  bindir="\${prefix}"
-  sysconfdir="\${prefix}"
-  local_statedir=
   confsuffix=""
   libs_qga="-lws2_32 -lwinmm -lpowrprof -lwtsapi32 -lwininet -liphlpapi 
-lnetapi32 $libs_qga"
 fi
@@ -1638,6 +1623,26 @@ for opt do
   esac
 done
 
+libdir="${libdir:-$prefix/lib}"
+libexecdir="${libexecdir:-$prefix/libexec}"
+includedir="${includedir:-$prefix/include}"
+
+if test "$mingw32" = "yes" ; then
+mandir="$prefix"
+datadir="$prefix"
+qemu_docdir="$prefix"
+bindir="$prefix"
+sysconfdir="$prefix"
+local_statedir=
+else
+mandir="${mandir:-$prefix/share/man}"
+datadir="${datadir:-$prefix/share}"
+qemu_docdir="${qemu_docdir:-$prefix/share/doc/qemu}"
+bindir="${bindir:-$prefix/bin}"
+sysconfdir="${sysconfdir:-$prefix/etc}"
+local_statedir="${local_statedir:-$prefix/var}"
+fi
+
 case "$cpu" in
 ppc)
CPU_CFLAGS="-m32"
-- 
2.26.2





[PULL v8 004/152] trace: switch position of headers to what Meson requires

2020-08-21 Thread Paolo Bonzini
Meson doesn't enjoy the same flexibility we have with Make in choosing
the include path.  In particular the tracing headers are using
$(build_root)/$(
---
 .gitignore   |  1 -
 Makefile | 46 ++--
 Makefile.objs|  1 -
 accel/kvm/trace.h|  1 +
 accel/tcg/cputlb.c   |  2 +-
 accel/tcg/trace.h|  1 +
 accel/tcg/user-exec.c|  2 +-
 audio/trace.h|  1 +
 authz/trace.h|  1 +
 backends/tpm/trace.h |  1 +
 backends/trace.h |  1 +
 block/trace.h|  1 +
 chardev/trace.h  |  1 +
 crypto/trace.h   |  1 +
 dma-helpers.c|  2 +-
 exec.c   |  2 +-
 gdbstub.c|  2 +-
 hw/9pfs/trace.h  |  1 +
 hw/acpi/trace.h  |  1 +
 hw/alpha/trace.h |  1 +
 hw/arm/trace.h   |  1 +
 hw/audio/trace.h |  1 +
 hw/block/dataplane/trace.h   |  1 +
 hw/block/trace.h |  1 +
 hw/char/trace.h  |  1 +
 hw/core/cpu.c|  2 +-
 hw/core/trace.h  |  1 +
 hw/display/trace.h   |  1 +
 hw/dma/trace.h   |  1 +
 hw/gpio/trace.h  |  1 +
 hw/hppa/trace.h  |  1 +
 hw/hyperv/trace.h|  1 +
 hw/i2c/trace.h   |  1 +
 hw/i386/trace.h  |  1 +
 hw/i386/xen/trace.h  |  1 +
 hw/ide/trace.h   |  1 +
 hw/input/trace.h |  1 +
 hw/intc/trace.h  |  1 +
 hw/isa/trace.h   |  1 +
 hw/mem/trace.h   |  1 +
 hw/mips/trace.h  |  1 +
 hw/misc/macio/trace.h|  1 +
 hw/misc/trace.h  |  1 +
 hw/net/trace.h   |  1 +
 hw/nvram/trace.h |  1 +
 hw/pci-host/trace.h  |  1 +
 hw/pci/trace.h   |  1 +
 hw/ppc/trace.h   |  1 +
 hw/rdma/trace.h  |  1 +
 hw/rdma/vmw/trace.h  |  1 +
 hw/riscv/trace.h |  1 +
 hw/rtc/trace.h   |  1 +
 hw/s390x/trace.h |  1 +
 hw/scsi/trace.h  |  1 +
 hw/sd/trace.h|  1 +
 hw/sparc/trace.h |  1 +
 hw/sparc64/trace.h   |  1 +
 hw/ssi/trace.h   |  1 +
 hw/timer/trace.h |  1 +
 hw/tpm/trace.h   |  1 +
 hw/usb/trace.h   |  1 +
 hw/vfio/trace.h  |  1 +
 hw/virtio/trace.h|  1 +
 hw/watchdog/trace.h  |  1 +
 hw/xen/trace.h   |  1 +
 include/user/syscall-trace.h |  2 +-
 io/trace.h   |  1 +
 job-qmp.c|  2 +-
 job.c|  2 +-
 linux-user/trace.h   |  1 +
 migration/trace.h|  1 +
 monitor/trace.h  |  1 +
 nbd/trace.h  |  1 +
 net/trace.h  |  1 +
 qapi/trace.h |  1 +
 qom/trace.h  |  1 +
 scripts/tracetool/format/c.py|  5 +--
 scripts/tracetool/format/tcg_h.py|  2 +-
 scripts/tracetool/format/tcg_helper_c.py |  2 +-
 scsi/trace.h |  1 +
 softmmu/balloon.c|  2 +-
 softmmu/ioport.c |  2 +-
 softmmu/memory.c |  2 +-
 softmmu/vl.c |  2 +-
 target/arm/trace.h   |  1 +
 target/hppa/trace.h  |  1 +
 target/i386/trace.h  |  1 +
 target/mips/trace.h  |  1 +
 target/ppc/trace.h   |  1 +
 target/riscv/trace.h |  1 +
 target/s390x/trace.h |  1 +
 target/sparc/trace.h |  1 +
 trace/control-target.c   |  2 +-
 trace/control.c  |  2 +-
 ui/trace.h   |  1 +
 util/trace.h |  1 +
 96 files changed, 121 insertions(+), 41 deletions(-)
 create mode 100644 accel/kvm/trace.h
 creat

[PULL v8 026/152] libqemuutil, qapi, trace: convert to meson

2020-08-21 Thread Paolo Bonzini
This shows how to do some "computations" in meson.build using its array
and dictionary data structures, and also a basic usage of the sourceset
module for conditional compilation.

Notice the new "if have_system" part of util/meson.build, which fixes
a bug in the old build system was buggy: util/dbus.c was built even for
non-softmmu builds, but the dependency on -lgio was lost when the linking
was done through libqemuutil.a.  Because all of its users required gio
otherwise, the bug was hidden.  Meson instead propagates libqemuutil's
dependencies down to its users, and shows the problem.

Signed-off-by: Paolo Bonzini 
---
 Makefile| 228 +---
 Makefile.objs   |  95 
 Makefile.target |   8 +-
 configure   |   8 +
 crypto/Makefile.objs|   6 -
 crypto/meson.build  |  11 ++
 docs/devel/tracing.txt  |   2 +-
 meson.build | 194 +++
 qapi/Makefile.objs  |  23 ---
 qapi/meson.build| 121 +++
 qobject/Makefile.objs   |   3 -
 qobject/meson.build |   3 +
 rules.mak   |   2 +-
 scripts/qapi-gen.py |   0
 scripts/tracetool.py|   0
 scripts/tracetool/backend/dtrace.py |   2 +-
 scripts/tracetool/backend/ust.py|   6 +-
 storage-daemon/Makefile.objs|   4 +-
 storage-daemon/meson.build  |   1 +
 storage-daemon/qapi/Makefile.objs   |   1 -
 storage-daemon/qapi/meson.build |  10 ++
 stubs/Makefile.objs |  56 ---
 stubs/meson.build   |  50 ++
 trace/Makefile.objs |  51 ---
 trace/meson.build   |  76 ++
 util/Makefile.objs  |  83 --
 util/meson.build|  78 ++
 27 files changed, 597 insertions(+), 525 deletions(-)
 create mode 100644 crypto/meson.build
 create mode 100644 qapi/meson.build
 delete mode 100644 qobject/Makefile.objs
 create mode 100644 qobject/meson.build
 mode change 100755 => 100644 scripts/qapi-gen.py
 mode change 100755 => 100644 scripts/tracetool.py
 create mode 100644 storage-daemon/meson.build
 delete mode 100644 storage-daemon/qapi/Makefile.objs
 create mode 100644 storage-daemon/qapi/meson.build
 delete mode 100644 stubs/Makefile.objs
 create mode 100644 stubs/meson.build
 create mode 100644 trace/meson.build
 delete mode 100644 util/Makefile.objs
 create mode 100644 util/meson.build

diff --git a/Makefile b/Makefile
index 5f9aae6e3e..e5d217d4d2 100644
--- a/Makefile
+++ b/Makefile
@@ -137,184 +137,60 @@ FULL_VERSION := $(if $(QEMU_PKGVERSION),$(VERSION) 
($(QEMU_PKGVERSION)),$(VERSIO
 
 generated-files-y = qemu-version.h config-host.h qemu-options.def
 
-GENERATED_QAPI_FILES = qapi/qapi-builtin-types.h qapi/qapi-builtin-types.c
-GENERATED_QAPI_FILES += qapi/qapi-types.h qapi/qapi-types.c
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-types-%.h)
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-types-%.c)
-GENERATED_QAPI_FILES += qapi/qapi-builtin-visit.h qapi/qapi-builtin-visit.c
-GENERATED_QAPI_FILES += qapi/qapi-visit.h qapi/qapi-visit.c
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-visit-%.h)
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-visit-%.c)
-GENERATED_QAPI_FILES += qapi/qapi-init-commands.h qapi/qapi-init-commands.c
-GENERATED_QAPI_FILES += qapi/qapi-commands.h qapi/qapi-commands.c
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-commands-%.h)
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-commands-%.c)
-GENERATED_QAPI_FILES += qapi/qapi-emit-events.h qapi/qapi-emit-events.c
-GENERATED_QAPI_FILES += qapi/qapi-events.h qapi/qapi-events.c
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-events-%.h)
-GENERATED_QAPI_FILES += $(QAPI_MODULES:%=qapi/qapi-events-%.c)
-GENERATED_QAPI_FILES += qapi/qapi-introspect.c qapi/qapi-introspect.h
-GENERATED_QAPI_FILES += qapi/qapi-doc.texi
-
-# The following list considers only the storage daemon main module. All other
-# modules are currently shared with the main schema, so we don't actually
-# generate additional files.
-
-GENERATED_STORAGE_DAEMON_QAPI_FILES = storage-daemon/qapi/qapi-commands.h
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-commands.c
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-emit-events.h
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-emit-events.c
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-events.h
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-events.c
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-init-commands.h
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-init-commands.c
-GENERATED_STORAGE_DAEMON_QAPI_FILES += storage-daemon/qapi/qapi-introspect.h
-GENERATED_STORAGE_DAEMON_QAP

[PULL v8 020/152] build-sys: add meson submodule

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 .gitmodules   |  3 ++
 configure | 71 +++
 meson |  1 +
 scripts/archive-source.sh |  3 +-
 4 files changed, 70 insertions(+), 8 deletions(-)
 create mode 16 meson

diff --git a/.gitmodules b/.gitmodules
index 9c0501a4d4..ce979398a8 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -58,3 +58,6 @@
 [submodule "roms/qboot"]
path = roms/qboot
url = https://github.com/bonzini/qboot
+[submodule "meson"]
+   path = meson
+   url = https://github.com/mesonbuild/meson/
diff --git a/configure b/configure
index a82045352c..3360f40a33 100755
--- a/configure
+++ b/configure
@@ -219,6 +219,25 @@ path_of() {
 return 1
 }
 
+version_ge () {
+local_ver1=`echo $1 | tr . ' '`
+local_ver2=`echo $2 | tr . ' '`
+while true; do
+set x $local_ver1
+local_first=${2-0}
+# shift 2 does nothing if there are less than 2 arguments
+shift; shift
+local_ver1=$*
+set x $local_ver2
+# the second argument finished, the first must be greater or equal
+test $# = 1 && return 0
+test $local_first -lt $2 && return 1
+test $local_first -gt $2 && return 0
+shift; shift
+local_ver2=$*
+done
+}
+
 have_backend () {
 echo "$trace_backends" | grep "$1" >/dev/null
 }
@@ -959,6 +978,7 @@ fi
 # python 2.x, but some distros have it as python 3.x, so
 # we check that too
 python=
+explicit_python=no
 for binary in "${PYTHON-python3}" python
 do
 if has "$binary"
@@ -1042,7 +1062,7 @@ for opt do
   ;;
   --install=*) install="$optarg"
   ;;
-  --python=*) python="$optarg"
+  --python=*) python="$optarg" ; explicit_python=yes
   ;;
   --sphinx-build=*) sphinx_build="$optarg"
   ;;
@@ -2023,15 +2043,52 @@ python_version=$($python -c 'import sys; 
print("%d.%d.%d" % (sys.version_info[0]
 # Suppress writing compiled files
 python="$python -B"
 
-if ! has "$meson"
-then
-error_exit "Meson not found. Use --meson=/path/to/meson"
+if test -z "$meson"; then
+if test "$explicit_python" = no && has meson && version_ge "$(meson 
--version)" 0.55.0; then
+meson=meson
+elif test -e "${source_path}/.git" && test $git_update = 'yes' ; then
+meson=git
+elif test -e "${source_path}/meson/meson.py" ; then
+meson=internal
+else
+if test "$explicit_python" = yes; then
+error_exit "--python requires using QEMU's embedded Meson 
distribution, but it was not found."
+else
+error_exit "Meson not found.  Use --meson=/path/to/meson"
+fi
+fi
+else
+# Meson uses its own Python interpreter to invoke other Python scripts,
+# but the user wants to use the one they specified with --python.
+#
+# We do not want to override the distro Python interpreter (and sometimes
+# cannot: for example in Homebrew /usr/bin/meson is a bash script), so
+# just require --meson=git|internal together with --python.
+if test "$explicit_python" = yes; then
+case "$meson" in
+git | internal) ;;
+*) error_exit "--python requires using QEMU's embedded Meson 
distribution." ;;
+esac
+fi
 fi
-meson=$(command -v $meson)
 
-if ! $python -c 'import pkg_resources' > /dev/null 2>&1; then
-  error_exit "Python setuptools not found"
+if test "$meson" = git; then
+git_submodules="${git_submodules} meson"
 fi
+if test "$git_update" = yes; then
+(cd "${source_path}" && GIT="$git" "./scripts/git-submodule.sh" update 
"$git_submodules")
+fi
+
+case "$meson" in
+git | internal)
+if ! $python -c 'import pkg_resources' > /dev/null 2>&1; then
+error_exit "Python setuptools not found"
+fi
+meson="$python ${source_path}/meson/meson.py"
+;;
+*) meson=$(command -v meson) ;;
+esac
+
 
 # Check that the C compiler works. Doing this here before testing
 # the host CPU ensures that we had a valid CC to autodetect the
diff --git a/meson b/meson
new file mode 16
index 00..d0c68dc115
--- /dev/null
+++ b/meson
@@ -0,0 +1 @@
+Subproject commit d0c68dc11507a47b9b85de508e023d9590d60565
diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
index fb5d6b3918..c6169db69f 100755
--- a/scripts/archive-source.sh
+++ b/scripts/archive-source.sh
@@ -26,7 +26,8 @@ sub_file="${sub_tdir}/submodule.tar"
 # independent of what the developer currently has initialized
 # in their checkout, because the build environment is completely
 # different to the host OS.
-submodules="dtc slirp ui/keycodemapdb tests/fp/berkeley-softfloat-3 
tests/fp/berkeley-testfloat-3"
+submodules="dtc slirp meson ui/keycodemapdb"
+submodules="$submodules tests/fp/berkeley-softfloat-3 
tests/fp/berkeley-testfloat-3"
 sub_deinit=""
 
 function cleanup() {
-- 
2.26.2





[PULL v8 014/152] configure: prepare CFLAGS/CXXFLAGS/LDFLAGS for Meson

2020-08-21 Thread Paolo Bonzini
Split between CFLAGS/QEMU_CFLAGS and CXXFLAGS/QEMU_CXXFLAGS so that
we will use CFLAGS and CXXFLAGS for flags that we do not want to
pass to add_project_arguments.

Signed-off-by: Paolo Bonzini 
---
 configure | 80 ---
 rules.mak |  4 +--
 softmmu/Makefile.objs |  2 +-
 3 files changed, 41 insertions(+), 45 deletions(-)

diff --git a/configure b/configure
index 373bb22287..35d6492343 100755
--- a/configure
+++ b/configure
@@ -107,15 +107,12 @@ update_cxxflags() {
 # options which some versions of GCC's C++ compiler complain about
 # because they only make sense for C programs.
 QEMU_CXXFLAGS="$QEMU_CXXFLAGS -D__STDC_LIMIT_MACROS 
-D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS"
-
+CXXFLAGS=$(echo "$CFLAGS" | sed s/-std=gnu99/-std=gnu++11/)
 for arg in $QEMU_CFLAGS; do
 case $arg in
 -Wstrict-prototypes|-Wmissing-prototypes|-Wnested-externs|\
 -Wold-style-declaration|-Wold-style-definition|-Wredundant-decls)
 ;;
--std=gnu99)
-QEMU_CXXFLAGS=${QEMU_CXXFLAGS:+$QEMU_CXXFLAGS }"-std=gnu++98"
-;;
 *)
 QEMU_CXXFLAGS=${QEMU_CXXFLAGS:+$QEMU_CXXFLAGS }$arg
 ;;
@@ -125,13 +122,13 @@ update_cxxflags() {
 
 compile_object() {
   local_cflags="$1"
-  do_cc $QEMU_CFLAGS $local_cflags -c -o $TMPO $TMPC
+  do_cc $CFLAGS $QEMU_CFLAGS $local_cflags -c -o $TMPO $TMPC
 }
 
 compile_prog() {
   local_cflags="$1"
   local_ldflags="$2"
-  do_cc $QEMU_CFLAGS $local_cflags -o $TMPE $TMPC $QEMU_LDFLAGS $local_ldflags
+  do_cc $CFLAGS $QEMU_CFLAGS $local_cflags -o $TMPE $TMPC $LDFLAGS 
$QEMU_LDFLAGS $local_ldflags
 }
 
 # symbolically link $1 to $2.  Portable version of "ln -sf".
@@ -597,15 +594,14 @@ ARFLAGS="${ARFLAGS-rv}"
 # left shift of signed integers is well defined and has the expected
 # 2s-complement style results. (Both clang and gcc agree that it
 # provides these semantics.)
-QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv -std=gnu99 $QEMU_CFLAGS"
-QEMU_CFLAGS="-Wall -Wundef -Wwrite-strings -Wmissing-prototypes $QEMU_CFLAGS"
+QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv $QEMU_CFLAGS"
+QEMU_CFLAGS="-Wundef -Wwrite-strings -Wmissing-prototypes $QEMU_CFLAGS"
 QEMU_CFLAGS="-Wstrict-prototypes -Wredundant-decls $QEMU_CFLAGS"
 QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
$QEMU_CFLAGS"
 QEMU_INCLUDES="-iquote . -iquote ${source_path} -iquote 
${source_path}/accel/tcg -iquote ${source_path}/include"
 QEMU_INCLUDES="$QEMU_INCLUDES -iquote ${source_path}/disas/libvixl"
-if test "$debug_info" = "yes"; then
-CFLAGS="-g $CFLAGS"
-fi
+CFLAGS="-std=gnu99 -Wall"
+
 
 # running configure in the source tree?
 # we know that's the case if configure is there.
@@ -886,7 +882,6 @@ SunOS)
   QEMU_CFLAGS="-D_XOPEN_SOURCE=600 $QEMU_CFLAGS"
 # needed for TIOCWIN* defines in termios.h
   QEMU_CFLAGS="-D__EXTENSIONS__ $QEMU_CFLAGS"
-  QEMU_CFLAGS="-std=gnu99 $QEMU_CFLAGS"
   solarisnetlibs="-lsocket -lnsl -lresolv"
   LIBS="$solarisnetlibs $LIBS"
   libs_qga="$solarisnetlibs $libs_qga"
@@ -963,7 +958,7 @@ if test "$mingw32" = "yes" ; then
   EXESUF=".exe"
   DSOSUF=".dll"
   # MinGW needs -mthreads for TLS and macro _MT.
-  QEMU_CFLAGS="-mthreads $QEMU_CFLAGS"
+  CFLAGS="-mthreads $CFLAGS"
   LIBS="-lwinmm -lws2_32 $LIBS"
   write_c_skeleton;
   if compile_prog "" "-liberty" ; then
@@ -2109,7 +2104,7 @@ EOF
   for flag in $gcc_flags; do
 # We need to check both a compile and a link, since some compiler
 # setups fail only on a .c->.o compile and some only at link time
-if do_cc $QEMU_CFLAGS -Werror $flag -c -o $TMPO $TMPC &&
+if compile_object "-Werror $flag" &&
compile_prog "-Werror $flag" ""; then
   QEMU_CFLAGS="$QEMU_CFLAGS $flag"
   QEMU_LDFLAGS="$QEMU_LDFLAGS $flag"
@@ -2184,7 +2179,7 @@ fi
 
 if test "$static" = "yes"; then
   if test "$pie" != "no" && compile_prog "-Werror -fPIE -DPIE" "-static-pie"; 
then
-QEMU_CFLAGS="-fPIE -DPIE $QEMU_CFLAGS"
+CFLAGS="-fPIE -DPIE $CFLAGS"
 QEMU_LDFLAGS="-static-pie $QEMU_LDFLAGS"
 pie="yes"
   elif test "$pie" = "yes"; then
@@ -2194,11 +2189,11 @@ if test "$static" = "yes"; then
 pie="no"
   fi
 elif test "$pie" = "no"; then
-  QEMU_CFLAGS="$CFLAGS_NOPIE $QEMU_CFLAGS"
-  QEMU_LDFLAGS="$LDFLAGS_NOPIE $QEMU_LDFLAGS"
+  CFLAGS="$CFLAGS_NOPIE $CFLAGS"
+  LDFLAGS="$LDFLAGS_NOPIE $LDFLAGS"
 elif compile_prog "-Werror -fPIE -DPIE" "-pie"; then
-  QEMU_CFLAGS="-fPIE -DPIE $QEMU_CFLAGS"
-  QEMU_LDFLAGS="-pie $QEMU_LDFLAGS"
+  CFLAGS="-fPIE -DPIE $CFLAGS"
+  LDFLAGS="-pie $LDFLAGS"
   pie="yes"
 elif test "$pie" = "yes"; then
   error_exit "PIE not available due to missing toolchain support"
@@ -3981,7 +3976,7 @@ EOF
 if ! compile_prog "$glib_cflags -Werror" "$glib_libs" ; then
 if cc_has_warning_flag "-Wno-unknown-attributes"; then
 glib_cflags="-Wno-unknown-attributes $glib_cflags"
-   

[PULL v8 005/152] meson: rename included C source files to .c.inc

2020-08-21 Thread Paolo Bonzini
With Makefiles that have automatically generated dependencies, you
generated includes are set as dependencies of the Makefile, so that they
are built before everything else and they are available when first
building the .c files.

Alternatively you can use a fine-grained dependency, e.g.

target/arm/translate.o: target/arm/decode-neon-shared.inc.c

With Meson you have only one choice and it is a third option, namely
"build at the beginning of the corresponding target"; the way you
express it is to list the includes in the sources of that target.

The problem is that Meson decides if something is a source vs. a
generated include by looking at the extension: '.c', '.cc', '.m', '.C'
are sources, while everything else is considered an include---including
'.inc.c'.

Use '.c.inc' to avoid this, as it is consistent with our other convention
of using '.rst.inc' for included reStructuredText files.  The editorconfig
file is adjusted.

Signed-off-by: Paolo Bonzini 
---
 .editorconfig |  3 +-
 .gitignore|  4 +-
 Makefile  | 36 -
 ...tomic_common.inc.c => atomic_common.c.inc} |  0
 accel/tcg/cputlb.c|  2 +-
 accel/tcg/user-exec.c |  2 +-
 contrib/gitdm/filetypes.txt   |  2 +-
 exec.c|  4 +-
 ...alize.inc.c => softfloat-specialize.c.inc} |  0
 fpu/softfloat.c   |  2 +-
 include/tcg/tcg.h |  2 +-
 memory_ldst.inc.c => memory_ldst.c.inc|  0
 scripts/clean-includes|  2 +-
 target/arm/Makefile.objs  | 40 +--
 ...nslate-neon.inc.c => translate-neon.c.inc} |  6 +--
 target/arm/translate-sve.c|  2 +-
 ...ranslate-vfp.inc.c => translate-vfp.c.inc} |  4 +-
 target/arm/translate.c| 12 +++---
 target/avr/Makefile.objs  |  4 +-
 target/avr/disas.c|  2 +-
 target/avr/translate.c|  2 +-
 target/cris/translate.c   |  2 +-
 ...ranslate_v10.inc.c => translate_v10.c.inc} |  0
 target/hppa/Makefile.objs |  4 +-
 target/hppa/translate.c   |  2 +-
 target/mips/translate.c   |  2 +-
 ...nslate_init.inc.c => translate_init.c.inc} |  0
 target/openrisc/Makefile.objs |  6 +--
 target/openrisc/disas.c   |  2 +-
 target/openrisc/translate.c   |  2 +-
 target/ppc/int_helper.c   |  2 +-
 .../{mfrom_table.inc.c => mfrom_table.c.inc}  |  0
 target/ppc/translate.c| 22 +-
 .../{dfp-impl.inc.c => dfp-impl.c.inc}|  0
 .../{dfp-ops.inc.c => dfp-ops.c.inc}  |  0
 .../{fp-impl.inc.c => fp-impl.c.inc}  |  0
 .../translate/{fp-ops.inc.c => fp-ops.c.inc}  |  0
 .../{spe-impl.inc.c => spe-impl.c.inc}|  0
 .../{spe-ops.inc.c => spe-ops.c.inc}  |  0
 .../{vmx-impl.inc.c => vmx-impl.c.inc}|  0
 .../{vmx-ops.inc.c => vmx-ops.c.inc}  |  0
 .../{vsx-impl.inc.c => vsx-impl.c.inc}|  0
 .../{vsx-ops.inc.c => vsx-ops.c.inc}  |  0
 ...nslate_init.inc.c => translate_init.c.inc} |  0
 target/riscv/Makefile.objs|  8 ++--
 ...rivileged.inc.c => trans_privileged.c.inc} |  0
 .../{trans_rva.inc.c => trans_rva.c.inc}  |  0
 .../{trans_rvd.inc.c => trans_rvd.c.inc}  |  0
 .../{trans_rvf.inc.c => trans_rvf.c.inc}  |  0
 .../{trans_rvh.inc.c => trans_rvh.c.inc}  |  0
 .../{trans_rvi.inc.c => trans_rvi.c.inc}  |  0
 .../{trans_rvm.inc.c => trans_rvm.c.inc}  |  0
 .../{trans_rvv.inc.c => trans_rvv.c.inc}  |  0
 target/riscv/translate.c  | 20 +-
 target/rx/Makefile.objs   |  6 +--
 target/rx/disas.c |  2 +-
 target/rx/translate.c |  2 +-
 target/s390x/translate.c  |  2 +-
 ...{translate_vx.inc.c => translate_vx.c.inc} |  0
 target/xtensa/core-dc232b.c   |  4 +-
 .../{gdb-config.inc.c => gdb-config.c.inc}|  0
 ...nsa-modules.inc.c => xtensa-modules.c.inc} |  0
 target/xtensa/core-dc233c.c   |  4 +-
 .../{gdb-config.inc.c => gdb-config.c.inc}|  0
 ...nsa-modules.inc.c => xtensa-modules.c.inc} |  0
 target/xtensa/core-de212.c|  4 +-
 .../{gdb-config.inc.c => gdb-config.c.inc}|  0
 ...nsa-modules.inc.c => xtensa-modules.c.inc} |  0
 target/xtensa/core-fsf.c  |  2 +-
 ...nsa-modules.inc.c => xtensa-modules.c.inc} |  0
 target/xtensa/core-sample_controller.c|  4 +-
 .../{gdb-config.inc.c => gdb-config.c.inc}|  0
 ...nsa-modules.inc.c => xtensa-modules.c.inc} |  0
 target/xtensa/core-test_kc705_be.c|  

[PULL v8 037/152] contrib/ivshmem: convert to meson

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile | 9 -
 Makefile.objs| 2 --
 configure| 3 ---
 contrib/ivshmem-client/Makefile.objs | 1 -
 contrib/ivshmem-client/meson.build   | 4 
 contrib/ivshmem-server/Makefile.objs | 1 -
 contrib/ivshmem-server/meson.build   | 4 
 meson.build  | 5 +
 8 files changed, 13 insertions(+), 16 deletions(-)
 delete mode 100644 contrib/ivshmem-client/Makefile.objs
 create mode 100644 contrib/ivshmem-client/meson.build
 delete mode 100644 contrib/ivshmem-server/Makefile.objs
 create mode 100644 contrib/ivshmem-server/meson.build

diff --git a/Makefile b/Makefile
index ed6494b771..e16cbe6191 100644
--- a/Makefile
+++ b/Makefile
@@ -303,8 +303,6 @@ dummy := $(call unnest-vars,, \
 chardev-obj-y \
 qga-obj-y \
 elf2dmp-obj-y \
-ivshmem-client-obj-y \
-ivshmem-server-obj-y \
 qga-vss-dll-obj-y \
 block-obj-y \
 block-obj-m \
@@ -517,13 +515,6 @@ endif
 elf2dmp$(EXESUF): $(elf2dmp-obj-y)
$(call LINK, $^)
 
-ifdef CONFIG_IVSHMEM
-ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) $(COMMON_LDADDS)
-   $(call LINK, $^)
-ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
-   $(call LINK, $^)
-endif
-
 module_block.h: $(SRC_PATH)/scripts/modules/module_block.py config-host.mak
$(call quiet-command,$(PYTHON) $< $@ \
$(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \
diff --git a/Makefile.objs b/Makefile.objs
index 2f2d4b2066..336a684ff3 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -104,7 +104,5 @@ qga-vss-dll-obj-y = qga/
 ##
 # contrib
 elf2dmp-obj-y = contrib/elf2dmp/
-ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
-ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
 
 ##
diff --git a/configure b/configure
index 40c033a48c..1d00af339c 100755
--- a/configure
+++ b/configure
@@ -6725,9 +6725,6 @@ if test "$want_tools" = "yes" ; then
   if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then
 tools="qemu-nbd\$(EXESUF) qemu-storage-daemon\$(EXESUF) $tools"
   fi
-  if [ "$ivshmem" = "yes" ]; then
-tools="ivshmem-client\$(EXESUF) ivshmem-server\$(EXESUF) $tools"
-  fi
   if [ "$curl" = "yes" ]; then
   tools="elf2dmp\$(EXESUF) $tools"
   fi
diff --git a/contrib/ivshmem-client/Makefile.objs 
b/contrib/ivshmem-client/Makefile.objs
deleted file mode 100644
index bfab2d20dd..00
--- a/contrib/ivshmem-client/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-ivshmem-client-obj-y = ivshmem-client.o main.o
diff --git a/contrib/ivshmem-client/meson.build 
b/contrib/ivshmem-client/meson.build
new file mode 100644
index 00..1b171efb4f
--- /dev/null
+++ b/contrib/ivshmem-client/meson.build
@@ -0,0 +1,4 @@
+executable('ivshmem-client', files('ivshmem-client.c', 'main.c'),
+   dependencies: glib,
+   build_by_default: targetos == 'linux',
+   install: false)
diff --git a/contrib/ivshmem-server/Makefile.objs 
b/contrib/ivshmem-server/Makefile.objs
deleted file mode 100644
index c060dd3698..00
--- a/contrib/ivshmem-server/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-ivshmem-server-obj-y = ivshmem-server.o main.o
diff --git a/contrib/ivshmem-server/meson.build 
b/contrib/ivshmem-server/meson.build
new file mode 100644
index 00..3a53942201
--- /dev/null
+++ b/contrib/ivshmem-server/meson.build
@@ -0,0 +1,4 @@
+executable('ivshmem-server', files('ivshmem-server.c', 'main.c'),
+   dependencies: [qemuutil, rt],
+   build_by_default: targetos == 'linux',
+   install: false)
diff --git a/meson.build b/meson.build
index 71ef66e1d6..c1078b6759 100644
--- a/meson.build
+++ b/meson.build
@@ -94,6 +94,7 @@ libcap_ng = not_found
 if 'CONFIG_LIBCAP_NG' in config_host
   libcap_ng = declare_dependency(link_args: 
config_host['LIBCAP_NG_LIBS'].split())
 endif
+rt = cc.find_library('rt', required: false)
 libiscsi = not_found
 if 'CONFIG_LIBISCSI' in config_host
   libiscsi = declare_dependency(compile_args: 
config_host['LIBISCSI_CFLAGS'].split(),
@@ -283,6 +284,10 @@ if have_tools
 subdir('contrib/vhost-user-input')
 subdir('contrib/vhost-user-scsi')
   endif
+  if 'CONFIG_IVSHMEM' in config_host
+subdir('contrib/ivshmem-client')
+subdir('contrib/ivshmem-server')
+  endif
 endif
 
 subdir('tools')
-- 
2.26.2





[PULL v8 025/152] meson: add testsuite Makefile generator

2020-08-21 Thread Paolo Bonzini
Rules to execute tests are generated by a simple Python program
that integrates into the existing "make check" mechanism.  This
provides familiarity for developers, and also allows piecewise
conversion of the testsuite Makefiles to meson.

The generated rules are based on QEMU's existing test harness
Makefile and TAP parser.

Signed-off-by: Paolo Bonzini 
---
 Makefile   |  10 +++-
 scripts/mtest2make.py  | 102 +
 tests/Makefile.include |   1 -
 3 files changed, 111 insertions(+), 2 deletions(-)
 create mode 100644 scripts/mtest2make.py

diff --git a/Makefile b/Makefile
index 81c9642a30..5f9aae6e3e 100644
--- a/Makefile
+++ b/Makefile
@@ -68,6 +68,14 @@ Makefile.ninja: build.ninja ninjatool
 
 ${ninja-targets-c_COMPILER} ${ninja-targets-cpp_COMPILER}: .var.command += -MP
 
+# If MESON is empty, the rule will be re-evaluated after Makefiles are
+# reread (and MESON won't be empty anymore).
+ifneq ($(MESON),)
+Makefile.mtest: build.ninja scripts/mtest2make.py
+   $(MESON) introspect --tests | $(PYTHON) scripts/mtest2make.py > $@
+-include Makefile.mtest
+endif
+
 .git-submodule-status: git-submodule-update config-host.mak
 
 # Check that we're not trying to do an out-of-tree build from
@@ -825,7 +833,7 @@ distclean: clean ninja-distclean
rm -f roms/seabios/config.mak roms/vgabios/config.mak
rm -f qemu-plugins-ld.symbols qemu-plugins-ld64.symbols
rm -rf meson-private meson-logs meson-info compile_commands.json
-   rm -f Makefile.ninja ninjatool ninjatool.stamp
+   rm -f Makefile.ninja ninjatool ninjatool.stamp Makefile.mtest
rm -f config.log
rm -f linux-headers/asm
rm -f docs/version.texi
diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
new file mode 100644
index 00..bdb257bbd9
--- /dev/null
+++ b/scripts/mtest2make.py
@@ -0,0 +1,102 @@
+#! /usr/bin/env python3
+
+# Create Makefile targets to run tests, from Meson's test introspection data.
+#
+# Author: Paolo Bonzini 
+
+from collections import defaultdict
+import json
+import os
+import shlex
+import sys
+
+class Suite(object):
+def __init__(self):
+self.tests = list()
+self.slow_tests = list()
+self.executables = set()
+
+print('''
+SPEED = quick
+
+# $1 = test command, $2 = test name
+.test-human-tap = $1 < /dev/null | ./scripts/tap-driver.pl --test-name="$2" 
$(if $(V),,--show-failures-only)
+.test-human-exitcode = $1 < /dev/null
+.test-tap-tap = $1 < /dev/null | sed "s/^[a-z][a-z]* [0-9]*/& $2/" || true
+.test-tap-exitcode = printf "%s\\n" 1..1 "`$1 < /dev/null > /dev/null || echo 
"not "`ok 1 $2"
+.test.print = echo $(if $(V),'$1','Running test $2') >&3
+.test.env = MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$(( $${RANDOM:-0} % 255 + 1))}
+
+# $1 = test name, $2 = test target (human or tap)
+.test.run = $(call .test.print,$(.test.cmd.$1),$(.test.name.$1)) && $(call 
.test-$2-$(.test.driver.$1),$(.test.cmd.$1),$(.test.name.$1))
+
+define .test.human_k
+@exec 3>&1; rc=0; $(foreach TEST, $1, $(call .test.run,$(TEST),human) 
|| rc=$$?;) \\
+  exit $$rc
+endef
+define .test.human_no_k
+$(foreach TEST, $1, @exec 3>&1; $(call .test.run,$(TEST),human)
+)
+endef
+.test.human = \\
+$(if $(findstring k, $(MAKEFLAGS)), $(.test.human_k), 
$(.test.human_no_k))
+
+define .test.tap
+@exec 3>&1; { $(foreach TEST, $1, $(call .test.run,$(TEST),tap); ) } \\
+  | ./scripts/tap-merge.pl | tee "$@" \\
+  | ./scripts/tap-driver.pl $(if $(V),, --show-failures-only)
+endef
+''')
+
+suites = defaultdict(Suite)
+i = 0
+for test in json.load(sys.stdin):
+env = ' '.join(('%s=%s' % (shlex.quote(k), shlex.quote(v))
+for k, v in test['env'].items()))
+executable = os.path.relpath(test['cmd'][0])
+if test['workdir'] is not None:
+test['cmd'][0] = os.path.relpath(test['cmd'][0], test['workdir'])
+else:
+test['cmd'][0] = executable
+cmd = '$(.test.env) %s %s' % (env, ' '.join((shlex.quote(x) for x in 
test['cmd'])))
+if test['workdir'] is not None:
+cmd = '(cd %s && %s)' % (shlex.quote(test['workdir']), cmd)
+driver = test['protocol'] if 'protocol' in test else 'exitcode'
+
+i += 1
+print('.test.name.%d := %s' % (i, test['name']))
+print('.test.driver.%d := %s' % (i, driver))
+print('.test.cmd.%d := %s' % (i, cmd))
+
+test_suites = test['suite'] or ['default']
+is_slow = any(s.endswith('-slow') for s in test_suites)
+for s in test_suites:
+# The suite name in the introspection info is "PROJECT:SUITE"
+s = s.split(':')[1]
+if s.endswith('-slow'):
+s = s[:-5]
+if is_slow:
+suites[s].slow_tests.append(i)
+else:
+suites[s].tests.append(i)
+suites[s].executables.add(executable)
+
+print('.PHONY: check check-report.tap')
+print('check:')
+print('check-report.tap:')
+print('\t@cat $^ | scripts/tap-

[PULL v8 029/152] contrib/libvhost-user: convert to Meson

2020-08-21 Thread Paolo Bonzini
Since libqemuutil.a has been converted to Meson, the conversion is
straightforward.

Signed-off-by: Paolo Bonzini 
---
 Makefile| 16 +---
 Makefile.objs   |  1 -
 contrib/libvhost-user/Makefile.objs |  1 -
 contrib/libvhost-user/meson.build   |  3 +++
 meson.build |  8 
 5 files changed, 16 insertions(+), 13 deletions(-)
 delete mode 100644 contrib/libvhost-user/Makefile.objs
 create mode 100644 contrib/libvhost-user/meson.build

diff --git a/Makefile b/Makefile
index 5731d739f1..78d3a78e97 100644
--- a/Makefile
+++ b/Makefile
@@ -318,7 +318,6 @@ dummy := $(call unnest-vars,, \
 ivshmem-server-obj-y \
 virtiofsd-obj-y \
 rdmacm-mux-obj-y \
-libvhost-user-obj-y \
 vhost-user-scsi-obj-y \
 vhost-user-blk-obj-y \
 vhost-user-input-obj-y \
@@ -443,11 +442,6 @@ $(BUILD_DIR)/version.o: $(SRC_PATH)/version.rc 
config-host.h
 
 Makefile: $(version-obj-y)
 
-##
-# Build libraries
-
-libvhost-user.a: $(libvhost-user-obj-y)
-
 ##
 
 COMMON_LDADDS = libqemuutil.a
@@ -546,9 +540,9 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) 
$(COMMON_LDADDS)
 ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
$(call LINK, $^)
 endif
-vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) libvhost-user.a 
$(COMMON_LDADDS)
+vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
-vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) libvhost-user.a 
$(COMMON_LDADDS)
+vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
@@ -557,16 +551,16 @@ rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
 
 # relies on Linux-specific syscalls
 ifeq ($(CONFIG_LINUX)$(CONFIG_SECCOMP)$(CONFIG_LIBCAP_NG),yyy)
-virtiofsd$(EXESUF): $(virtiofsd-obj-y) libvhost-user.a $(COMMON_LDADDS)
+virtiofsd$(EXESUF): $(virtiofsd-obj-y) contrib/libvhost-user/libvhost-user.a 
$(COMMON_LDADDS)
$(call LINK, $^)
 endif
 
-vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) libvhost-user.a 
$(COMMON_LDADDS)
+vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
 
 ifdef CONFIG_VHOST_USER_INPUT
 ifdef CONFIG_LINUX
-vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) libvhost-user.a 
$(COMMON_LDADDS)
+vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
 
 # build by default, do not install
diff --git a/Makefile.objs b/Makefile.objs
index e5c9077517..9489864967 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -106,7 +106,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-libvhost-user-obj-y = contrib/libvhost-user/
 vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
 vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
 vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
diff --git a/contrib/libvhost-user/Makefile.objs 
b/contrib/libvhost-user/Makefile.objs
deleted file mode 100644
index ef3778edd4..00
--- a/contrib/libvhost-user/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-libvhost-user-obj-y += libvhost-user.o libvhost-user-glib.o
diff --git a/contrib/libvhost-user/meson.build 
b/contrib/libvhost-user/meson.build
new file mode 100644
index 00..e68dd1a581
--- /dev/null
+++ b/contrib/libvhost-user/meson.build
@@ -0,0 +1,3 @@
+libvhost_user = static_library('vhost-user',
+   files('libvhost-user.c', 
'libvhost-user-glib.c'),
+   build_by_default: false)
diff --git a/meson.build b/meson.build
index feb3c8a30c..00f17ef36f 100644
--- a/meson.build
+++ b/meson.build
@@ -243,6 +243,14 @@ libqemuutil = static_library('qemuutil',
 qemuutil = declare_dependency(link_with: libqemuutil,
   sources: genh + version_res)
 
+# Other build targets
+
+if have_tools
+  if 'CONFIG_VHOST_USER' in config_host
+subdir('contrib/libvhost-user')
+  endif
+endif
+
 summary_info = {}
 summary_info += {'Install prefix':config_host['prefix']}
 summary_info += {'BIOS directory':config_host['qemu_datadir']}
-- 
2.26.2





[PULL v8 034/152] contrib/rdmacm-mux: convert to Meson

2020-08-21 Thread Paolo Bonzini
We can use config-host.mak to decide whether the tool has to be built,
apart from that the conversion is straightforward.

Signed-off-by: Paolo Bonzini 
---
 Makefile | 5 -
 Makefile.objs| 1 -
 contrib/rdmacm-mux/Makefile.objs | 3 ---
 contrib/rdmacm-mux/meson.build   | 9 +
 meson.build  | 2 ++
 5 files changed, 11 insertions(+), 9 deletions(-)
 delete mode 100644 contrib/rdmacm-mux/Makefile.objs
 create mode 100644 contrib/rdmacm-mux/meson.build

diff --git a/Makefile b/Makefile
index 1f5a710f77..ad3d92 100644
--- a/Makefile
+++ b/Makefile
@@ -311,7 +311,6 @@ dummy := $(call unnest-vars,, \
 elf2dmp-obj-y \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
-rdmacm-mux-obj-y \
 vhost-user-input-obj-y \
 vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
@@ -533,10 +532,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) 
$(COMMON_LDADDS)
$(call LINK, $^)
 endif
 
-rdmacm-mux$(EXESUF): LIBS += "-libumad"
-rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
-   $(call LINK, $^)
-
 vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
 
diff --git a/Makefile.objs b/Makefile.objs
index ab798b08a7..0f80b63554 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -106,7 +106,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
 
diff --git a/contrib/rdmacm-mux/Makefile.objs b/contrib/rdmacm-mux/Makefile.objs
deleted file mode 100644
index 3df744af89..00
--- a/contrib/rdmacm-mux/Makefile.objs
+++ /dev/null
@@ -1,3 +0,0 @@
-ifdef CONFIG_PVRDMA
-rdmacm-mux-obj-y = main.o
-endif
diff --git a/contrib/rdmacm-mux/meson.build b/contrib/rdmacm-mux/meson.build
new file mode 100644
index 00..6cc5016747
--- /dev/null
+++ b/contrib/rdmacm-mux/meson.build
@@ -0,0 +1,9 @@
+if 'CONFIG_PVRDMA' in config_host
+  # if not found, CONFIG_PVRDMA should not be set
+  # FIXME: broken on big endian architectures
+  libumad = cc.find_library('ibumad', required: true)
+  executable('rdmacm-mux', files('main.c'),
+ dependencies: [glib, libumad],
+ build_by_default: false,
+ install: false)
+endif
diff --git a/meson.build b/meson.build
index 4c067d9fe6..c60ec5825b 100644
--- a/meson.build
+++ b/meson.build
@@ -260,6 +260,8 @@ qemuutil = declare_dependency(link_with: libqemuutil,
 # Other build targets
 
 if have_tools
+  subdir('contrib/rdmacm-mux')
+
   if 'CONFIG_VHOST_USER' in config_host
 subdir('contrib/libvhost-user')
 subdir('contrib/vhost-user-blk')
-- 
2.26.2





[PULL v8 012/152] configure: do not include $(...) variables in config-host.mak

2020-08-21 Thread Paolo Bonzini
This ensures that Meson will be able to reuse the results of
the tests that are performed in the configure script.

Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Paolo Bonzini 
---
 configure | 32 
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/configure b/configure
index 625c14c500..606c327cca 100755
--- a/configure
+++ b/configure
@@ -610,8 +610,8 @@ QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv 
-std=gnu99 $QEMU_CFLAGS"
 QEMU_CFLAGS="-Wall -Wundef -Wwrite-strings -Wmissing-prototypes $QEMU_CFLAGS"
 QEMU_CFLAGS="-Wstrict-prototypes -Wredundant-decls $QEMU_CFLAGS"
 QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
$QEMU_CFLAGS"
-QEMU_INCLUDES="-iquote . -iquote \$(SRC_PATH) -iquote \$(SRC_PATH)/accel/tcg 
-iquote \$(SRC_PATH)/include"
-QEMU_INCLUDES="$QEMU_INCLUDES -iquote \$(SRC_PATH)/disas/libvixl"
+QEMU_INCLUDES="-iquote . -iquote ${source_path} -iquote 
${source_path}/accel/tcg -iquote ${source_path}/include"
+QEMU_INCLUDES="$QEMU_INCLUDES -iquote ${source_path}/disas/libvixl"
 if test "$debug_info" = "yes"; then
 CFLAGS="-g $CFLAGS"
 fi
@@ -911,7 +911,7 @@ Linux)
   linux="yes"
   linux_user="yes"
   kvm="yes"
-  QEMU_INCLUDES="-isystem \$(SRC_PATH)/linux-headers -isystem 
$PWD/linux-headers $QEMU_INCLUDES"
+  QEMU_INCLUDES="-isystem ${source_path}/linux-headers -I$PWD/linux-headers 
$QEMU_INCLUDES"
   supported_os="yes"
   libudev="yes"
 ;;
@@ -4397,8 +4397,8 @@ EOF
   if [ "$pwd_is_source_path" != "y" ] ; then
   symlink "$source_path/dtc/Makefile" "dtc/Makefile"
   fi
-  fdt_cflags="-I\$(SRC_PATH)/dtc/libfdt"
-  fdt_ldflags="-L\$(BUILD_DIR)/dtc/libfdt"
+  fdt_cflags="-I${source_path}/dtc/libfdt"
+  fdt_ldflags="-L$PWD/dtc/libfdt"
   fdt_libs="$fdt_libs"
   elif test "$fdt" = "yes" ; then
   # Not a git build & no libfdt found, prompt for system install
@@ -5385,13 +5385,13 @@ case "$capstone" in
   git_submodules="${git_submodules} capstone"
 fi
 mkdir -p capstone
-QEMU_CFLAGS="$QEMU_CFLAGS -I\$(SRC_PATH)/capstone/include"
+QEMU_CFLAGS="$QEMU_CFLAGS -I${source_path}/capstone/include"
 if test "$mingw32" = "yes"; then
   LIBCAPSTONE=capstone.lib
 else
   LIBCAPSTONE=libcapstone.a
 fi
-libs_cpu="-L\$(BUILD_DIR)/capstone -lcapstone $libs_cpu"
+libs_cpu="-L$PWD/capstone -lcapstone $libs_cpu"
 ;;
 
   system)
@@ -6414,8 +6414,8 @@ case "$slirp" in
   git_submodules="${git_submodules} slirp"
 fi
 mkdir -p slirp
-slirp_cflags="-I\$(SRC_PATH)/slirp/src -I\$(BUILD_DIR)/slirp/src"
-slirp_libs="-L\$(BUILD_DIR)/slirp -lslirp"
+slirp_cflags="-I${source_path}/slirp/src -I$PWD/slirp/src"
+slirp_libs="-L$PWD/slirp -lslirp"
 if test "$mingw32" = "yes" ; then
   slirp_libs="$slirp_libs -lws2_32 -liphlpapi"
 fi
@@ -7909,19 +7909,19 @@ if test "$secret_keyring" = "yes" ; then
 fi
 
 if test "$tcg_interpreter" = "yes"; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/tci $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/tci $QEMU_INCLUDES"
 elif test "$ARCH" = "sparc64" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/sparc $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/sparc $QEMU_INCLUDES"
 elif test "$ARCH" = "s390x" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/s390 $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/s390 $QEMU_INCLUDES"
 elif test "$ARCH" = "x86_64" || test "$ARCH" = "x32" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/i386 $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/i386 $QEMU_INCLUDES"
 elif test "$ARCH" = "ppc64" ; then
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/ppc $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/ppc $QEMU_INCLUDES"
 elif test "$ARCH" = "riscv32" || test "$ARCH" = "riscv64" ; then
-  QEMU_INCLUDES="-I\$(SRC_PATH)/tcg/riscv $QEMU_INCLUDES"
+  QEMU_INCLUDES="-I${source_path}/tcg/riscv $QEMU_INCLUDES"
 else
-  QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/\$(ARCH) $QEMU_INCLUDES"
+  QEMU_INCLUDES="-iquote ${source_path}/tcg/${ARCH} $QEMU_INCLUDES"
 fi
 
 echo "HELPERS=$helpers" >> $config_host_mak
-- 
2.26.2





[PULL v8 024/152] meson: add sparse support

2020-08-21 Thread Paolo Bonzini
Do not use cgcc; instead, extract compilation commands from 
compile_commands.json
and invoke sparse directly.

Signed-off-by: Paolo Bonzini 
---
 Makefile|  1 +
 configure   |  8 ++--
 meson.build |  7 +++
 scripts/check_sparse.py | 25 +
 4 files changed, 35 insertions(+), 6 deletions(-)
 create mode 100644 scripts/check_sparse.py

diff --git a/Makefile b/Makefile
index ba8413d809..81c9642a30 100644
--- a/Makefile
+++ b/Makefile
@@ -1261,6 +1261,7 @@ endif
$(call print-help,install,Install QEMU, documentation and tools)
$(call print-help,ctags/TAGS,Generate tags file for editors)
$(call print-help,cscope,Generate cscope index)
+   $(call print-help,sparse,Run sparse on the QEMU source)
@echo  ''
@$(if $(TARGET_DIRS), \
echo 'Architecture specific targets:'; \
diff --git a/configure b/configure
index b2c498a0bf..e23b2f616f 100755
--- a/configure
+++ b/configure
@@ -3054,7 +3054,7 @@ fi
 ##
 # Sparse probe
 if test "$sparse" != "no" ; then
-  if has cgcc; then
+  if has sparse; then
 sparse=yes
   else
 if test "$sparse" = "yes" ; then
@@ -7855,11 +7855,7 @@ echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
 echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak
 echo "QEMU_INCLUDES=$QEMU_INCLUDES" >> $config_host_mak
 if test "$sparse" = "yes" ; then
-  echo "CC   := REAL_CC=\"\$(CC)\" cgcc"   >> $config_host_mak
-  echo "CPP  := REAL_CC=\"\$(CPP)\" cgcc"  >> $config_host_mak
-  echo "CXX  := REAL_CC=\"\$(CXX)\" cgcc"  >> $config_host_mak
-  echo "HOST_CC  := REAL_CC=\"\$(HOST_CC)\" cgcc"  >> $config_host_mak
-  echo "QEMU_CFLAGS  += -Wbitwise -Wno-transparent-union -Wno-old-initializer 
-Wno-non-pointer-null" >> $config_host_mak
+  echo "SPARSE_CFLAGS = -Wbitwise -Wno-transparent-union -Wno-old-initializer 
-Wno-non-pointer-null" >> $config_host_mak
 fi
 echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak
 echo "LDFLAGS_NOPIE=$LDFLAGS_NOPIE" >> $config_host_mak
diff --git a/meson.build b/meson.build
index 1a56ac8b26..86219e500a 100644
--- a/meson.build
+++ b/meson.build
@@ -25,6 +25,13 @@ if host_machine.system() == 'darwin'
   add_languages('objc', required: false, native: false)
 endif
 
+if 'SPARSE_CFLAGS' in config_host
+  run_target('sparse',
+ command: [find_program('scripts/check_sparse.py'),
+   config_host['SPARSE_CFLAGS'].split(),
+   'compile_commands.json'])
+endif
+
 configure_file(input: files('scripts/ninjatool.py'),
output: 'ninjatool',
configuration: config_host)
diff --git a/scripts/check_sparse.py b/scripts/check_sparse.py
new file mode 100644
index 00..0de7aa55d9
--- /dev/null
+++ b/scripts/check_sparse.py
@@ -0,0 +1,25 @@
+#! /usr/bin/env python3
+
+# Invoke sparse based on the contents of compile_commands.json
+
+import json
+import subprocess
+import sys
+import shlex
+
+def extract_cflags(shcmd):
+cflags = shlex.split(shcmd)
+return [x for x in cflags
+if x.startswith('-D') or x.startswith('-I') or x.startswith('-W')
+   or x.startswith('-std=')]
+
+cflags = sys.argv[1:-1]
+with open(sys.argv[-1], 'r') as fd:
+compile_commands = json.load(fd)
+
+for cmd in compile_commands:
+cmd = ['sparse'] + cflags + extract_cflags(cmd['command']) + [cmd['file']]
+print(' '.join((shlex.quote(x) for x in cmd)))
+r = subprocess.run(cmd)
+if r.returncode != 0:
+sys.exit(r.returncode)
-- 
2.26.2





[PULL v8 036/152] contrib/vhost-user-gpu: convert to meson

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile | 18 +-
 Makefile.objs|  1 -
 contrib/vhost-user-gpu/Makefile.objs | 10 --
 contrib/vhost-user-gpu/meson.build   | 13 +
 meson.build  | 15 +++
 rules.mak|  7 ---
 6 files changed, 29 insertions(+), 35 deletions(-)
 delete mode 100644 contrib/vhost-user-gpu/Makefile.objs
 create mode 100644 contrib/vhost-user-gpu/meson.build

diff --git a/Makefile b/Makefile
index c0cb10cd77..ed6494b771 100644
--- a/Makefile
+++ b/Makefile
@@ -195,16 +195,10 @@ $(call set-vpath, $(SRC_PATH))
 
 LIBS+=-lz $(LIBS_TOOLS)
 
-vhost-user-json-y =
 HELPERS-y = $(HELPERS)
 
 HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) += 
qemu-bridge-helper$(EXESUF)
 
-ifeq ($(CONFIG_LINUX)$(CONFIG_VIRGL)$(CONFIG_GBM)$(CONFIG_TOOLS),)
-HELPERS-y += vhost-user-gpu$(EXESUF)
-vhost-user-json-y += contrib/vhost-user-gpu/50-qemu-gpu.json
-endif
-
 # Sphinx does not allow building manuals into the same directory as
 # the source files, so if we're doing an in-tree QEMU build we must
 # build the manuals into a subdirectory (and then install them from
@@ -311,7 +305,6 @@ dummy := $(call unnest-vars,, \
 elf2dmp-obj-y \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
-vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
 block-obj-y \
 block-obj-m \
@@ -325,7 +318,7 @@ dummy := $(call unnest-vars,, \
 
 include $(SRC_PATH)/tests/Makefile.include
 
-all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) $(HELPERS-y) recurse-all 
modules $(vhost-user-json-y)
+all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) $(HELPERS-y) recurse-all 
modules
 
 qemu-version.h: FORCE
$(call quiet-command, \
@@ -531,9 +524,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) 
$(COMMON_LDADDS)
$(call LINK, $^)
 endif
 
-vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
-   $(call LINK, $^)
-
 module_block.h: $(SRC_PATH)/scripts/modules/module_block.py config-host.mak
$(call quiet-command,$(PYTHON) $< $@ \
$(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \
@@ -731,12 +721,6 @@ endif
 ifneq ($(HELPERS-y),)
$(call install-prog,$(HELPERS-y),$(DESTDIR)$(libexecdir))
 endif
-ifneq ($(vhost-user-json-y),)
-   $(INSTALL_DIR) "$(DESTDIR)$(qemu_datadir)/vhost-user/"
-   for x in $(vhost-user-json-y); do \
-   $(INSTALL_DATA) $$x "$(DESTDIR)$(qemu_datadir)/vhost-user/"; \
-   done
-endif
 ifdef CONFIG_TRACE_SYSTEMTAP
$(INSTALL_PROG) "scripts/qemu-trace-stap" $(DESTDIR)$(bindir)
 endif
diff --git a/Makefile.objs b/Makefile.objs
index 0922c3ed00..2f2d4b2066 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -106,6 +106,5 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
 
 ##
diff --git a/contrib/vhost-user-gpu/Makefile.objs 
b/contrib/vhost-user-gpu/Makefile.objs
deleted file mode 100644
index 09296091be..00
--- a/contrib/vhost-user-gpu/Makefile.objs
+++ /dev/null
@@ -1,10 +0,0 @@
-vhost-user-gpu-obj-y = vhost-user-gpu.o virgl.o vugbm.o
-
-vhost-user-gpu.o-cflags := $(PIXMAN_CFLAGS) $(GBM_CFLAGS)
-vhost-user-gpu.o-libs := $(PIXMAN_LIBS)
-
-virgl.o-cflags := $(VIRGL_CFLAGS) $(GBM_CFLAGS)
-virgl.o-libs := $(VIRGL_LIBS)
-
-vugbm.o-cflags := $(GBM_CFLAGS)
-vugbm.o-libs := $(GBM_LIBS)
diff --git a/contrib/vhost-user-gpu/meson.build 
b/contrib/vhost-user-gpu/meson.build
new file mode 100644
index 00..6c1459f54a
--- /dev/null
+++ b/contrib/vhost-user-gpu/meson.build
@@ -0,0 +1,13 @@
+if 'CONFIG_TOOLS' in config_host and 'CONFIG_VIRGL' in config_host \
+and 'CONFIG_GBM' in config_host and 'CONFIG_LINUX' in config_host
+  executable('vhost-user-gpu', files('vhost-user-gpu.c', 'virgl.c', 'vugbm.c'),
+ link_with: libvhost_user,
+ dependencies: [qemuutil, pixman, gbm, virgl],
+ install: true,
+ install_dir: get_option('libexecdir'))
+
+  configure_file(input: '50-qemu-gpu.json.in',
+ output: '50-qemu-gpu.json',
+ configuration: config_host,
+ install_dir: config_host['qemu_datadir'] / 'vhost-user')
+endif
diff --git a/meson.build b/meson.build
index 371d571495..71ef66e1d6 100644
--- a/meson.build
+++ b/meson.build
@@ -83,6 +83,8 @@ if 'CONFIG_GNUTLS' in config_host
   gnutls = declare_dependency(compile_args: 
config_host['GNUTLS_CFLAGS'].split(),
   link_args: config_host['GNUTLS_LIBS'].split()

[PULL v8 023/152] meson: use coverage option

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile   | 14 --
 Makefile.target| 16 
 configure  |  9 ++---
 docs/devel/testing.rst |  7 +++
 meson.build|  2 +-
 5 files changed, 6 insertions(+), 42 deletions(-)

diff --git a/Makefile b/Makefile
index b726e7d8d2..ba8413d809 100644
--- a/Makefile
+++ b/Makefile
@@ -777,14 +777,6 @@ module_block.h: 
$(SRC_PATH)/scripts/modules/module_block.py config-host.mak
$(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \
"GEN","$@")
 
-ifdef CONFIG_GCOV
-.PHONY: clean-coverage
-clean-coverage:
-   $(call quiet-command, \
-   find . \( -name '*.gcda' -o -name '*.gcov' \) -type f -exec rm 
{} +, \
-   "CLEAN", "coverage files")
-endif
-
 clean: recurse-clean ninja-clean clean-ctlist
-test -f ninjatool && ./ninjatool $(if $(V),-v,) -t clean
 # avoid old build problems by removing potentially incorrect old files
@@ -1291,9 +1283,6 @@ endif
echo '')
@echo  'Cleaning targets:'
$(call print-help,clean,Remove most generated files but keep the config)
-ifdef CONFIG_GCOV
-   $(call print-help,clean-coverage,Remove coverage files)
-endif
$(call print-help,distclean,Remove all generated files)
$(call print-help,dist,Build a distributable tarball)
@echo  ''
@@ -1304,9 +1293,6 @@ endif
@echo  ''
@echo  'Documentation targets:'
$(call print-help,html info pdf txt,Build documentation in specified 
format)
-ifdef CONFIG_GCOV
-   $(call print-help,coverage-report,Create code coverage report)
-endif
@echo  ''
 ifdef CONFIG_WIN32
@echo  'Windows targets:'
diff --git a/Makefile.target b/Makefile.target
index ffa2657269..d61a6a978b 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -269,19 +269,3 @@ endif
 
 generated-files-y += config-target.h
 Makefile: $(generated-files-y)
-
-# Reports/Analysis
-#
-# The target specific coverage report only cares about target specific
-# blobs and not the shared code.
-#
-
-%/coverage-report.html:
-   @mkdir -p $*
-   $(call quiet-command,\
-   gcovr -r $(SRC_PATH) --object-directory $(CURDIR) \
-   -p --html --html-details -o $@, \
-   "GEN", "coverage-report.html")
-
-.PHONY: coverage-report
-coverage-report: $(CURDIR)/reports/coverage/coverage-report.html
diff --git a/configure b/configure
index 7c2a46ade0..b2c498a0bf 100755
--- a/configure
+++ b/configure
@@ -466,7 +466,6 @@ tcg_interpreter="no"
 bigendian="no"
 mingw32="no"
 gcov="no"
-gcov_tool="gcov"
 EXESUF=""
 DSOSUF=".so"
 LDFLAGS_SHARED="-shared"
@@ -1054,8 +1053,6 @@ for opt do
   ;;
   --meson=*) meson="$optarg"
   ;;
-  --gcov=*) gcov_tool="$optarg"
-  ;;
   --smbd=*) smbd="$optarg"
   ;;
   --extra-cflags=*)
@@ -1862,7 +1859,6 @@ Advanced options (experts only):
   --with-coroutine=BACKEND coroutine backend. Supported options:
ucontext, sigaltstack, windows
   --enable-gcovenable test coverage analysis with gcov
-  --gcov=GCOV  use specified gcov [$gcov_tool]
   --disable-blobs  disable installing provided firmware blobs
   --with-vss-sdk=SDK-path  enable Windows VSS support in QEMU Guest Agent
   --with-win-sdk=SDK-path  path to Windows Platform SDK (to build VSS .tlb)
@@ -6596,8 +6592,7 @@ fi
 write_c_skeleton
 
 if test "$gcov" = "yes" ; then
-  QEMU_CFLAGS="-fprofile-arcs -ftest-coverage -g $QEMU_CFLAGS"
-  QEMU_LDFLAGS="-fprofile-arcs -ftest-coverage $QEMU_LDFLAGS"
+  :
 elif test "$fortify_source" = "yes" ; then
   QEMU_CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 $QEMU_CFLAGS"
   debug=no
@@ -7882,7 +7877,6 @@ echo "TASN1_CFLAGS=$tasn1_cflags" >> $config_host_mak
 echo "POD2MAN=$POD2MAN" >> $config_host_mak
 if test "$gcov" = "yes" ; then
   echo "CONFIG_GCOV=y" >> $config_host_mak
-  echo "GCOV=$gcov_tool" >> $config_host_mak
 fi
 
 if test "$libudev" != "no"; then
@@ -8515,6 +8509,7 @@ NINJA=$PWD/ninjatool $meson setup \
 -Dwerror=$(if test "$werror" = yes; then echo true; else echo false; 
fi) \
 -Dstrip=$(if test "$strip_opt" = yes; then echo true; else echo false; 
fi) \
 -Db_pie=$(if test "$pie" = yes; then echo true; else echo false; fi) \
+-Db_coverage=$(if test "$gcov" = yes; then echo true; else echo false; 
fi) \
 $cross_arg \
 "$PWD" "$source_path"
 
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index c1ff24370b..a4264691be 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -164,13 +164,12 @@ instrumenting the tested code. To use it, configure QEMU 
with
 ``--enable-gcov`` option and build. Then run ``make check`` as usual.
 
 If you want to gather coverage information on a single test the ``make
-clean-coverage`` target can be used to delete any existing coverage
+clean-gcda`` target can be u

[PULL v8 019/152] build-sys hack: link with whole .fa archives

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

In order to link the *-obj-y files into tests, we will make static
libraries of them in Meson, and then link them as whole archives
into the tests.  To separate regular static libraries from link-whole
libraries, give them a different file extension.

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 rules.mak | 18 +-
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/rules.mak b/rules.mak
index e79a4005a7..5ab42def82 100644
--- a/rules.mak
+++ b/rules.mak
@@ -39,6 +39,13 @@ find-symbols = $(if $1, $(sort $(shell $(NM) -P -g $1 | $2)))
 defined-symbols = $(call find-symbols,$1,awk '$$2!="U"{print $$1}')
 undefined-symbols = $(call find-symbols,$1,awk '$$2=="U"{print $$1}')
 
+WL := -Wl,
+ifdef CONFIG_DARWIN
+whole-archive = $(WL)-force_load,$1
+else
+whole-archive = $(WL)--whole-archive $1 $(WL)--no-whole-archive
+endif
+
 # All the .mo objects in -m variables are also added into corresponding -y
 # variable in unnest-vars, but filtered out here, when LINK is called.
 #
@@ -54,11 +61,12 @@ undefined-symbols = $(call find-symbols,$1,awk 
'$$2=="U"{print $$1}')
 # This is necessary because the exectuable itself may not use the function, in
 # which case the function would not be linked in. Then the DSO loading will
 # fail because of the missing symbol.
-process-archive-undefs = $(filter-out %.a %.mo,$1) \
+process-archive-undefs = $(filter-out %.a %.fa %.mo,$1) \
 $(addprefix $(WL_U), \
- $(filter $(call defined-symbols,$(filter %.a, $1)), \
+ $(filter $(call defined-symbols,$(filter %.a %.fa, $1)), \
   $(call undefined-symbols,$(filter %.mo,$1 \
-$(filter %.a,$1)
+   $(foreach l,$(filter %.fa,$1),$(call whole-archive,$l)) \
+   $(filter %.a,$1)
 
 extract-libs = $(strip $(foreach o,$(filter-out %.mo,$1),$($o-libs)))
 expand-objs = $(strip $(sort $(filter %.o,$1)) \
@@ -122,7 +130,7 @@ LD_REL := $(CC) -nostdlib $(LD_REL_FLAGS)
 modules:
 
 %$(EXESUF): %.o
-   $(call LINK,$(filter %.o %.a %.mo, $^))
+   $(call LINK,$(filter %.o %.a %.mo %.fa, $^))
 
 %.a:
$(call quiet-command,rm -f $@ && $(AR) rcs $@ $^,"AR","$(TARGET_DIR)$@")
@@ -378,7 +386,7 @@ define unnest-vars
 $(error $o added in $v but $o-objs is not set)))
 $(shell mkdir -p ./ $(sort $(dir $($v
 # Include all the .d files
-$(eval -include $(patsubst %.o,%.d,$(patsubst %.mo,%.d,$($v
+$(eval -include $(patsubst %.o,%.d,$(patsubst %.mo,%.d,$(filter 
%.o,$($v)
 $(eval $v := $(filter-out %/,$($v
 endef
 
-- 
2.26.2





[PULL v8 030/152] tools/virtiofsd: convert to Meson

2020-08-21 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini 
---
 Makefile  | 12 
 Makefile.objs |  1 -
 configure |  1 +
 meson.build   | 11 +++
 tools/meson.build | 10 ++
 tools/virtiofsd/Makefile.objs | 12 
 tools/virtiofsd/meson.build   | 19 +++
 tools/virtiofsd/passthrough_ll.c  |  2 +-
 .../{seccomp.c => passthrough_seccomp.c}  |  2 +-
 .../{seccomp.h => passthrough_seccomp.h}  |  0
 10 files changed, 43 insertions(+), 27 deletions(-)
 create mode 100644 tools/meson.build
 delete mode 100644 tools/virtiofsd/Makefile.objs
 create mode 100644 tools/virtiofsd/meson.build
 rename tools/virtiofsd/{seccomp.c => passthrough_seccomp.c} (99%)
 rename tools/virtiofsd/{seccomp.h => passthrough_seccomp.h} (100%)

diff --git a/Makefile b/Makefile
index 78d3a78e97..e591d1b2b4 100644
--- a/Makefile
+++ b/Makefile
@@ -205,11 +205,6 @@ HELPERS-y += vhost-user-gpu$(EXESUF)
 vhost-user-json-y += contrib/vhost-user-gpu/50-qemu-gpu.json
 endif
 
-ifeq 
($(CONFIG_SOFTMMU)$(CONFIG_LINUX)$(CONFIG_SECCOMP)$(CONFIG_LIBCAP_NG),)
-HELPERS-y += virtiofsd$(EXESUF)
-vhost-user-json-y += tools/virtiofsd/50-qemu-virtiofsd.json
-endif
-
 # Sphinx does not allow building manuals into the same directory as
 # the source files, so if we're doing an in-tree QEMU build we must
 # build the manuals into a subdirectory (and then install them from
@@ -316,7 +311,6 @@ dummy := $(call unnest-vars,, \
 elf2dmp-obj-y \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
-virtiofsd-obj-y \
 rdmacm-mux-obj-y \
 vhost-user-scsi-obj-y \
 vhost-user-blk-obj-y \
@@ -549,12 +543,6 @@ rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
$(call LINK, $^)
 
-# relies on Linux-specific syscalls
-ifeq ($(CONFIG_LINUX)$(CONFIG_SECCOMP)$(CONFIG_LIBCAP_NG),yyy)
-virtiofsd$(EXESUF): $(virtiofsd-obj-y) contrib/libvhost-user/libvhost-user.a 
$(COMMON_LDADDS)
-   $(call LINK, $^)
-endif
-
 vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
 
diff --git a/Makefile.objs b/Makefile.objs
index 9489864967..fee0f71372 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -113,6 +113,5 @@ vhost-user-blk-obj-y = contrib/vhost-user-blk/
 rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
-virtiofsd-obj-y = tools/virtiofsd/
 
 ##
diff --git a/configure b/configure
index 05c7cd24aa..40c033a48c 100755
--- a/configure
+++ b/configure
@@ -7043,6 +7043,7 @@ if test "$gprof" = "yes" ; then
 fi
 if test "$cap_ng" = "yes" ; then
   echo "CONFIG_LIBCAP_NG=y" >> $config_host_mak
+  echo "LIBCAP_NG_LIBS=$cap_libs" >> $config_host_mak
 fi
 echo "CONFIG_AUDIO_DRIVERS=$audio_drv_list" >> $config_host_mak
 for drv in $audio_drv_list; do
diff --git a/meson.build b/meson.build
index 00f17ef36f..00970e6e19 100644
--- a/meson.build
+++ b/meson.build
@@ -83,6 +83,15 @@ if 'CONFIG_GNUTLS' in config_host
   gnutls = declare_dependency(compile_args: 
config_host['GNUTLS_CFLAGS'].split(),
   link_args: config_host['GNUTLS_LIBS'].split())
 endif
+seccomp = not_found
+if 'CONFIG_SECCOMP' in config_host
+  seccomp = declare_dependency(compile_args: 
config_host['SECCOMP_CFLAGS'].split(),
+   link_args: config_host['SECCOMP_LIBS'].split())
+endif
+libcap_ng = not_found
+if 'CONFIG_LIBCAP_NG' in config_host
+  libcap_ng = declare_dependency(link_args: 
config_host['LIBCAP_NG_LIBS'].split())
+endif
 
 target_dirs = config_host['TARGET_DIRS'].split()
 have_user = false
@@ -251,6 +260,8 @@ if have_tools
   endif
 endif
 
+subdir('tools')
+
 summary_info = {}
 summary_info += {'Install prefix':config_host['prefix']}
 summary_info += {'BIOS directory':config_host['qemu_datadir']}
diff --git a/tools/meson.build b/tools/meson.build
new file mode 100644
index 00..513bd2ff4f
--- /dev/null
+++ b/tools/meson.build
@@ -0,0 +1,10 @@
+have_virtiofsd = (have_system and
+have_tools and
+'CONFIG_LINUX' in config_host and 
+'CONFIG_SECCOMP' in config_host and
+'CONFIG_LIBCAP_NG' in config_host and
+'CONFIG_VHOST_USER' in config_host)
+
+if have_virtiofsd
+  subdir('virtiofsd')
+endif
diff --git a/tools/virtiofsd/Makefile.objs b/tools/virtiofsd/Makefile.objs
deleted file mode 100644
index 076f667e46..00
--- a/tools/virtiofsd/Makefile.objs
+++ /dev/null
@@ -1,12 +0,0 @@
-virtiofsd-obj-y = buffer.o \
-  fuse_opt.o \
-  fuse_log.o \
-  fuse_lowlevel

[PULL v8 027/152] meson: add remaining generated tcg trace helpers

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile  | 51 ---
 trace/meson.build | 14 +
 2 files changed, 14 insertions(+), 51 deletions(-)

diff --git a/Makefile b/Makefile
index e5d217d4d2..5731d739f1 100644
--- a/Makefile
+++ b/Makefile
@@ -137,61 +137,10 @@ FULL_VERSION := $(if $(QEMU_PKGVERSION),$(VERSION) 
($(QEMU_PKGVERSION)),$(VERSIO
 
 generated-files-y = qemu-version.h config-host.h qemu-options.def
 
-generated-files-y += trace/generated-tcg-tracers.h
-
-generated-files-y += trace/generated-helpers-wrappers.h
-generated-files-y += trace/generated-helpers.h
-generated-files-y += trace/generated-helpers.c
-
 generated-files-y += module_block.h
 
 generated-files-y += .git-submodule-status
 
-tracetool-y = $(SRC_PATH)/scripts/tracetool.py
-tracetool-y += $(shell find $(SRC_PATH)/scripts/tracetool -name "*.py")
-
-trace/generated-helpers-wrappers.h: 
trace/generated-helpers-wrappers.h-timestamp
-   @cmp $< $@ >/dev/null 2>&1 || cp $< $@
-trace/generated-helpers-wrappers.h-timestamp: $(SRC_PATH)/trace-events 
$(BUILD_DIR)/config-host.mak $(tracetool-y)
-   @mkdir -p $(dir $@)
-   $(call quiet-command,$(TRACETOOL) \
-   --group=root \
-   --format=tcg-helper-wrapper-h \
-   --backend=$(TRACE_BACKENDS) \
-   $< > $@,"GEN","$(patsubst %-timestamp,%,$@)")
-
-trace/generated-helpers.h: trace/generated-helpers.h-timestamp
-   @cmp $< $@ >/dev/null 2>&1 || cp $< $@
-trace/generated-helpers.h-timestamp: $(SRC_PATH)/trace-events 
$(BUILD_DIR)/config-host.mak $(tracetool-y)
-   @mkdir -p $(dir $@)
-   $(call quiet-command,$(TRACETOOL) \
-   --group=root \
-   --format=tcg-helper-h \
-   --backend=$(TRACE_BACKENDS) \
-   $< > $@,"GEN","$(patsubst %-timestamp,%,$@)")
-
-trace/generated-helpers.c: trace/generated-helpers.c-timestamp
-   @cmp $< $@ >/dev/null 2>&1 || cp $< $@
-trace/generated-helpers.c-timestamp: $(SRC_PATH)/trace-events 
$(BUILD_DIR)/config-host.mak $(tracetool-y)
-   @mkdir -p $(dir $@)
-   $(call quiet-command,$(TRACETOOL) \
-   --group=root \
-   --format=tcg-helper-c \
-   --backend=$(TRACE_BACKENDS) \
-   $< > $@,"GEN","$(patsubst %-timestamp,%,$@)")
-
-trace/generated-helpers.o: trace/generated-helpers.c
-
-trace/generated-tcg-tracers.h: trace/generated-tcg-tracers.h-timestamp
-   @cmp $< $@ >/dev/null 2>&1 || cp $< $@
-trace/generated-tcg-tracers.h-timestamp: $(SRC_PATH)/trace-events 
$(BUILD_DIR)/config-host.mak $(tracetool-y)
-   @mkdir -p $(dir $@)
-   $(call quiet-command,$(TRACETOOL) \
-   --group=root \
-   --format=tcg-h \
-   --backend=$(TRACE_BACKENDS) \
-   $< > $@,"GEN","$(patsubst %-timestamp,%,$@)")
-
 KEYCODEMAP_GEN = $(SRC_PATH)/ui/keycodemapdb/tools/keymap-gen
 KEYCODEMAP_CSV = $(SRC_PATH)/ui/keycodemapdb/data/keymaps.csv
 
diff --git a/trace/meson.build b/trace/meson.build
index f0a8d1c2e2..cab36a248b 100644
--- a/trace/meson.build
+++ b/trace/meson.build
@@ -55,6 +55,20 @@ custom_target('trace-events-all',
   install: true,
   install_dir: config_host['qemu_datadir'])
 
+foreach d : [
+  ['generated-tcg-tracers.h', 'tcg-h'],
+  ['generated-helpers.c', 'tcg-helper-c'],
+  ['generated-helpers.h', 'tcg-helper-h'],
+  ['generated-helpers-wrappers.h', 'tcg-helper-wrapper-h'],
+]
+  custom_target(d[0],
+output: d[0],
+input: meson.source_root() / 'trace-events',
+command: [ tracetool, '--group=root', 
'--format=@0@'.format(d[1]), '@INPUT@' ],
+build_by_default: true, # to be removed when added to a target
+capture: true)
+endforeach
+
 if 'CONFIG_TRACE_UST' in config_host
   trace_ust_all_h = custom_target('trace-ust-all.h',
   output: 'trace-ust-all.h',
-- 
2.26.2





[PULL v8 032/152] vhost-user-scsi: add compatibility for libiscsi 1.9.0

2020-08-21 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini 
---
 contrib/vhost-user-scsi/vhost-user-scsi.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/contrib/vhost-user-scsi/vhost-user-scsi.c 
b/contrib/vhost-user-scsi/vhost-user-scsi.c
index 7a1db164c8..3c912384e9 100644
--- a/contrib/vhost-user-scsi/vhost-user-scsi.c
+++ b/contrib/vhost-user-scsi/vhost-user-scsi.c
@@ -12,7 +12,9 @@
 
 #include "qemu/osdep.h"
 #include 
+#define inline __attribute__((gnu_inline))  /* required for libiscsi v1.9.0 */
 #include 
+#undef inline
 #include "contrib/libvhost-user/libvhost-user-glib.h"
 #include "standard-headers/linux/virtio_scsi.h"
 
-- 
2.26.2





[PULL v8 039/152] meson: add macos dependencies

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

There is no probing in configure, so no need to pass them as
variables to meson. Do a regular meson dependency() instead.

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 meson.build | 9 +
 1 file changed, 9 insertions(+)

diff --git a/meson.build b/meson.build
index 59ac2f05d9..d9758629cc 100644
--- a/meson.build
+++ b/meson.build
@@ -50,6 +50,10 @@ m = cc.find_library('m', required: false)
 util = cc.find_library('util', required: false)
 socket = []
 version_res = []
+coref = []
+iokit = []
+cocoa = []
+hvf = []
 if targetos == 'windows'
   socket = cc.find_library('ws2_32')
 
@@ -57,6 +61,11 @@ if targetos == 'windows'
   version_res = win.compile_resources('version.rc',
   depend_files: 
files('pc-bios/qemu-nsis.ico'),
   include_directories: 
include_directories('.'))
+elif targetos == 'darwin'
+  coref = dependency('appleframeworks', modules: 'CoreFoundation')
+  iokit = dependency('appleframeworks', modules: 'IOKit')
+  cocoa = dependency('appleframeworks', modules: 'Cocoa')
+  hvf = dependency('appleframeworks', modules: 'Hypervisor')
 endif
 glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(),
   link_args: config_host['GLIB_LIBS'].split())
-- 
2.26.2





[PULL v8 044/152] meson: convert dummy Windows qga/qemu-ga target

2020-08-21 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini 
---
 Makefile  | 5 -
 qga/meson.build   | 4 
 qga/vss-win32/meson.build | 2 ++
 3 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index e93ebf6299..8d3c8d8fbd 100644
--- a/Makefile
+++ b/Makefile
@@ -450,11 +450,6 @@ qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx 
$(SRC_PATH)/scripts/hxtool
 qemu-keymap$(EXESUF): LIBS += $(XKBCOMMON_LIBS)
 qemu-keymap$(EXESUF): QEMU_CFLAGS += $(XKBCOMMON_CFLAGS)
 
-ifneq ($(EXESUF),)
-.PHONY: qga/qemu-ga
-qga/qemu-ga: qga/qemu-ga$(EXESUF) $(QGA_VSS_PROVIDER) $(QEMU_GA_MSI)
-endif
-
 module_block.h: $(SRC_PATH)/scripts/modules/module_block.py config-host.mak
$(call quiet-command,$(PYTHON) $< $@ \
$(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \
diff --git a/qga/meson.build b/qga/meson.build
index 33f6db2865..2b91261427 100644
--- a/qga/meson.build
+++ b/qga/meson.build
@@ -44,6 +44,7 @@ qga = executable('qemu-ga', qga_ss.sources(),
  link_args: config_host['LIBS_QGA'].split(),
  dependencies: [qemuutil, libudev],
  install: true)
+all_qga = [qga]
 
 if targetos == 'windows'
   if 'CONFIG_QGA_VSS' in config_host
@@ -72,6 +73,9 @@ if targetos == 'windows'
   config_host['QEMU_GA_MSI_WITH_VSS'].split(),
   
config_host['QEMU_GA_MSI_MINGW_DLL_PATH'].split(),
 ])
+all_qga += [qga_msi]
 alias_target('msi', qga_msi)
   endif
 endif
+
+alias_target('qemu-ga', all_qga)
diff --git a/qga/vss-win32/meson.build b/qga/vss-win32/meson.build
index 1f39e05335..780c461432 100644
--- a/qga/vss-win32/meson.build
+++ b/qga/vss-win32/meson.build
@@ -14,6 +14,8 @@ if add_languages('cpp', required: false)
cc.find_library('shlwapi'),
cc.find_library('uuid'),
cc.find_library('intl')])
+
+  all_qga += qga_vss
 endif
 
 # rules to build qga-vss.tlb
-- 
2.26.2





[PULL v8 041/152] meson: convert qemu-ga

2020-08-21 Thread Paolo Bonzini
Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile  | 53 +--
 Makefile.objs |  1 -
 configure | 14 +++
 meson.build   |  8 ++
 qga/Makefile.objs |  9 ---
 qga/installer/qemu-ga.wxs |  2 +-
 qga/meson.build   | 46 +
 tests/Makefile.include|  4 +--
 tests/test-qga.c  |  8 +++---
 9 files changed, 71 insertions(+), 74 deletions(-)
 create mode 100644 qga/meson.build

diff --git a/Makefile b/Makefile
index 91602ad51e..83e2c86725 100644
--- a/Makefile
+++ b/Makefile
@@ -301,7 +301,6 @@ endif
 dummy := $(call unnest-vars,, \
 authz-obj-y \
 chardev-obj-y \
-qga-obj-y \
 qga-vss-dll-obj-y \
 block-obj-y \
 block-obj-m \
@@ -449,52 +448,15 @@ endif
 qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > 
$@,"GEN","$@")
 
-qemu-ga$(EXESUF): LIBS = $(LIBS_QGA)
-qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated
-
 qemu-keymap$(EXESUF): LIBS += $(XKBCOMMON_LIBS)
 qemu-keymap$(EXESUF): QEMU_CFLAGS += $(XKBCOMMON_CFLAGS)
 
-qapi-py = $(SRC_PATH)/scripts/qapi/__init__.py \
-$(SRC_PATH)/scripts/qapi/commands.py \
-$(SRC_PATH)/scripts/qapi/common.py \
-$(SRC_PATH)/scripts/qapi/doc.py \
-$(SRC_PATH)/scripts/qapi/error.py \
-$(SRC_PATH)/scripts/qapi/events.py \
-$(SRC_PATH)/scripts/qapi/expr.py \
-$(SRC_PATH)/scripts/qapi/gen.py \
-$(SRC_PATH)/scripts/qapi/introspect.py \
-$(SRC_PATH)/scripts/qapi/parser.py \
-$(SRC_PATH)/scripts/qapi/schema.py \
-$(SRC_PATH)/scripts/qapi/source.py \
-$(SRC_PATH)/scripts/qapi/types.py \
-$(SRC_PATH)/scripts/qapi/visit.py \
-$(SRC_PATH)/scripts/qapi-gen.py
-
-qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h \
-qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h \
-qga/qapi-generated/qga-qapi-commands.h qga/qapi-generated/qga-qapi-commands.c \
-qga/qapi-generated/qga-qapi-init-commands.h 
qga/qapi-generated/qga-qapi-init-commands.c \
-qga/qapi-generated/qga-qapi-doc.texi: \
-qga/qapi-generated/qapi-gen-timestamp ;
-qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json 
$(qapi-py)
-   $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \
-   -o qga/qapi-generated -p "qga-" $<, \
-   "GEN","$(@:%-timestamp=%)")
-   @>$@
-
-QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h 
qga-qapi-commands.h qga-qapi-init-commands.h)
-$(qga-obj-y): $(QGALIB_GEN)
-
-qemu-ga$(EXESUF): $(qga-obj-y) $(COMMON_LDADDS)
-   $(call LINK, $^)
-
 ifdef QEMU_GA_MSI_ENABLED
 QEMU_GA_MSI=qemu-ga-$(ARCH).msi
 
 msi: $(QEMU_GA_MSI)
 
-$(QEMU_GA_MSI): qemu-ga.exe $(QGA_VSS_PROVIDER)
+$(QEMU_GA_MSI): qga/qemu-ga.exe $(QGA_VSS_PROVIDER)
 
 $(QEMU_GA_MSI): config-host.mak
 
@@ -507,8 +469,8 @@ msi:
 endif
 
 ifneq ($(EXESUF),)
-.PHONY: qemu-ga
-qemu-ga: qemu-ga$(EXESUF) $(QGA_VSS_PROVIDER) $(QEMU_GA_MSI)
+.PHONY: qga/qemu-ga
+qga/qemu-ga: qga/qemu-ga$(EXESUF) $(QGA_VSS_PROVIDER) $(QEMU_GA_MSI)
 endif
 
 module_block.h: $(SRC_PATH)/scripts/modules/module_block.py config-host.mak
@@ -533,9 +495,6 @@ clean: recurse-clean ninja-clean clean-ctlist
rm -f qemu-img-cmds.h
rm -f ui/shader/*-vert.h ui/shader/*-frag.h
rm -f $(foreach f,$(generated-files-y),$(f) $(f)-timestamp)
-   rm -f qapi-gen-timestamp
-   rm -f storage-daemon/qapi/qapi-gen-timestamp
-   rm -rf qga/qapi-generated
rm -f config-all-devices.mak
rm -f $(SUBDIR_DEVICES_MAK)
 
@@ -657,7 +616,7 @@ endif
 ifdef CONFIG_TRACE_SYSTEMTAP
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/tools/qemu-trace-stap.1 
"$(DESTDIR)$(mandir)/man1"
 endif
-ifneq (,$(findstring qemu-ga,$(TOOLS)))
+ifeq ($(CONFIG_GUEST_AGENT),y)
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/interop/qemu-ga.8 
"$(DESTDIR)$(mandir)/man8"
$(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)/interop"
$(INSTALL_DATA) docs/interop/qemu-ga-ref.html 
"$(DESTDIR)$(qemu_docdir)/interop"
@@ -678,7 +637,7 @@ install-datadir:
 
 install-localstatedir:
 ifdef CONFIG_POSIX
-ifneq (,$(findstring qemu-ga,$(TOOLS)))
+ifeq ($(CONFIG_GUEST_AGENT),y)
$(INSTALL_DIR) "$(DESTDIR)$(qemu_localstatedir)"/run
 endif
 endif
@@ -870,7 +829,7 @@ $(MANUAL_BUILDDIR)/index.html: 
$(SRC_PATH)/docs/index.html.in qemu-version.h
 docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
@cp -p $< $@
 
-docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi-doc.texi
+docs/interop/qemu-ga-qapi.texi: qga/qga-qapi-doc.texi
@cp -p $< $@
 
 html: docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html sphinxdocs
diff --git a/Makefile.objs b/Makefile.objs
index 1486254a2c..259f9936ac 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -98,5 +98,4 @@ version-obj-$(CONFIG_WIN32) 

[PULL v8 038/152] contrib/elf2dmp: convert to meson

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile  | 4 
 Makefile.objs | 6 --
 configure | 3 ---
 contrib/elf2dmp/Makefile.objs | 4 
 contrib/elf2dmp/meson.build   | 5 +
 meson.build   | 6 ++
 6 files changed, 11 insertions(+), 17 deletions(-)
 delete mode 100644 contrib/elf2dmp/Makefile.objs
 create mode 100644 contrib/elf2dmp/meson.build

diff --git a/Makefile b/Makefile
index e16cbe6191..91602ad51e 100644
--- a/Makefile
+++ b/Makefile
@@ -302,7 +302,6 @@ dummy := $(call unnest-vars,, \
 authz-obj-y \
 chardev-obj-y \
 qga-obj-y \
-elf2dmp-obj-y \
 qga-vss-dll-obj-y \
 block-obj-y \
 block-obj-m \
@@ -512,9 +511,6 @@ ifneq ($(EXESUF),)
 qemu-ga: qemu-ga$(EXESUF) $(QGA_VSS_PROVIDER) $(QEMU_GA_MSI)
 endif
 
-elf2dmp$(EXESUF): $(elf2dmp-obj-y)
-   $(call LINK, $^)
-
 module_block.h: $(SRC_PATH)/scripts/modules/module_block.py config-host.mak
$(call quiet-command,$(PYTHON) $< $@ \
$(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \
diff --git a/Makefile.objs b/Makefile.objs
index 336a684ff3..1486254a2c 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -100,9 +100,3 @@ version-obj-$(CONFIG_WIN32) += $(BUILD_DIR)/version.o
 # extracted into a QAPI schema module, or perhaps a separate schema.
 qga-obj-y = qga/
 qga-vss-dll-obj-y = qga/
-
-##
-# contrib
-elf2dmp-obj-y = contrib/elf2dmp/
-
-##
diff --git a/configure b/configure
index 1d00af339c..dc27877932 100755
--- a/configure
+++ b/configure
@@ -6725,9 +6725,6 @@ if test "$want_tools" = "yes" ; then
   if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then
 tools="qemu-nbd\$(EXESUF) qemu-storage-daemon\$(EXESUF) $tools"
   fi
-  if [ "$curl" = "yes" ]; then
-  tools="elf2dmp\$(EXESUF) $tools"
-  fi
 fi
 if test "$softmmu" = yes ; then
   if test "$linux" = yes; then
diff --git a/contrib/elf2dmp/Makefile.objs b/contrib/elf2dmp/Makefile.objs
deleted file mode 100644
index 1505716916..00
--- a/contrib/elf2dmp/Makefile.objs
+++ /dev/null
@@ -1,4 +0,0 @@
-elf2dmp-obj-y = main.o addrspace.o download.o pdb.o qemu_elf.o
-
-download.o-cflags := $(CURL_CFLAGS)
-download.o-libs   := $(CURL_LIBS)
diff --git a/contrib/elf2dmp/meson.build b/contrib/elf2dmp/meson.build
new file mode 100644
index 00..b3de173316
--- /dev/null
+++ b/contrib/elf2dmp/meson.build
@@ -0,0 +1,5 @@
+if 'CONFIG_CURL' in config_host
+  executable('elf2dmp', files('main.c', 'addrspace.c', 'download.c', 'pdb.c', 
'qemu_elf.c'),
+ dependencies: [glib, curl],
+ install: true)
+endif
diff --git a/meson.build b/meson.build
index c1078b6759..59ac2f05d9 100644
--- a/meson.build
+++ b/meson.build
@@ -110,6 +110,11 @@ if 'CONFIG_VIRGL' in config_host
   virgl = declare_dependency(compile_args: config_host['VIRGL_CFLAGS'].split(),
  link_args: config_host['VIRGL_LIBS'].split())
 endif
+curl = not_found
+if 'CONFIG_CURL' in config_host
+  curl = declare_dependency(compile_args: config_host['CURL_CFLAGS'].split(),
+link_args: config_host['CURL_LIBS'].split())
+endif
 
 target_dirs = config_host['TARGET_DIRS'].split()
 have_user = false
@@ -274,6 +279,7 @@ qemuutil = declare_dependency(link_with: libqemuutil,
 
 if have_tools
   subdir('contrib/rdmacm-mux')
+  subdir('contrib/elf2dmp')
 
   if 'CONFIG_VHOST_USER' in config_host
 subdir('contrib/libvhost-user')
-- 
2.26.2





[PULL v8 017/152] configure: integrate Meson in the build system

2020-08-21 Thread Paolo Bonzini
The Meson build system is integrated in the existing configure/make steps
by invoking Meson from the configure script and converting Meson's build.ninja
rules to an included Makefile.

build.ninja already provides tags/ctags/cscope rules, so they are removed.

Signed-off-by: Paolo Bonzini 
---
 Makefile |   50 ++-
 configure|   52 ++-
 meson.build  |   25 ++
 scripts/ninjatool.py | 1002 ++
 4 files changed, 1107 insertions(+), 22 deletions(-)
 create mode 100644 meson.build
 create mode 100755 scripts/ninjatool.py

diff --git a/Makefile b/Makefile
index ec12101a84..b726e7d8d2 100644
--- a/Makefile
+++ b/Makefile
@@ -49,6 +49,25 @@ git-submodule-update:
 endif
 endif
 
+export NINJA=./ninjatool
+
+# Running meson regenerates both build.ninja and ninjatool, and that is
+# enough to prime the rest of the build.
+ninjatool: build.ninja
+
+# Only needed in case Makefile.ninja does not exist.
+.PHONY: ninja-clean ninja-distclean clean-ctlist
+clean-ctlist:
+ninja-clean::
+ninja-distclean::
+build.ninja: config-host.mak
+
+Makefile.ninja: build.ninja ninjatool
+   ./ninjatool -t ninja2make --omit clean dist uninstall < $< > $@
+-include Makefile.ninja
+
+${ninja-targets-c_COMPILER} ${ninja-targets-cpp_COMPILER}: .var.command += -MP
+
 .git-submodule-status: git-submodule-update config-host.mak
 
 # Check that we're not trying to do an out-of-tree build from
@@ -70,7 +89,11 @@ CONFIG_ALL=y
 
 config-host.mak: $(SRC_PATH)/configure $(SRC_PATH)/pc-bios $(SRC_PATH)/VERSION
@echo $@ is out-of-date, running configure
-   @./config.status
+   @if test -f meson-private/coredata.dat; then \
+ ./config.status --skip-meson; \
+   else \
+ ./config.status; \
+   fi
 
 # Force configure to re-run if the API symbols are updated
 ifeq ($(CONFIG_PLUGIN),y)
@@ -762,7 +785,8 @@ clean-coverage:
"CLEAN", "coverage files")
 endif
 
-clean: recurse-clean
+clean: recurse-clean ninja-clean clean-ctlist
+   -test -f ninjatool && ./ninjatool $(if $(V),-v,) -t clean
 # avoid old build problems by removing potentially incorrect old files
rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h 
gen-op-arm.h
rm -f qemu-options.def
@@ -799,7 +823,8 @@ rm -rf $(MANUAL_BUILDDIR)/$1/_static
 rm -f $(MANUAL_BUILDDIR)/$1/objects.inv $(MANUAL_BUILDDIR)/$1/searchindex.js 
$(MANUAL_BUILDDIR)/$1/*.html
 endef
 
-distclean: clean
+distclean: clean ninja-distclean
+   -test -f ninjatool && ./ninjatool $(if $(V),-v,) -t clean -g
rm -f config-host.mak config-host.h* $(DOCS)
rm -f tests/tcg/config-*.mak
rm -f config-all-devices.mak config-all-disas.mak config.status
@@ -807,6 +832,8 @@ distclean: clean
rm -f po/*.mo tests/qemu-iotests/common.env
rm -f roms/seabios/config.mak roms/vgabios/config.mak
rm -f qemu-plugins-ld.symbols qemu-plugins-ld64.symbols
+   rm -rf meson-private meson-logs meson-info compile_commands.json
+   rm -f Makefile.ninja ninjatool ninjatool.stamp
rm -f config.log
rm -f linux-headers/asm
rm -f docs/version.texi
@@ -933,6 +960,8 @@ ICON_SIZES=16x16 24x24 32x32 48x48 64x64 128x128 256x256 
512x512
 install-includedir:
$(INSTALL_DIR) "$(DESTDIR)$(includedir)"
 
+# Needed by "meson install"
+export DESTDIR
 install: all $(if $(BUILD_DOCS),install-doc) \
install-datadir install-localstatedir install-includedir \
$(if $(INSTALL_BLOBS),$(edk2-decompressed)) \
@@ -1006,21 +1035,6 @@ endif
done
$(INSTALL_DATA) $(BUILD_DIR)/trace-events-all 
"$(DESTDIR)$(qemu_datadir)/trace-events-all"
 
-.PHONY: ctags
-ctags:
-   rm -f tags
-   find "$(SRC_PATH)" -name '*.[hc]' -exec ctags --append {} +
-
-.PHONY: TAGS
-TAGS:
-   rm -f TAGS
-   find "$(SRC_PATH)" -name '*.[hc]' -exec etags --append {} +
-
-cscope:
-   rm -f "$(SRC_PATH)"/cscope.*
-   find "$(SRC_PATH)/" -name "*.[chsS]" -print | sed 's,^\./,,' > 
"$(SRC_PATH)/cscope.files"
-   cscope -b -i"$(SRC_PATH)/cscope.files"
-
 # opengl shader programs
 ui/shader/%-vert.h: $(SRC_PATH)/ui/shader/%.vert 
$(SRC_PATH)/scripts/shaderinclude.pl
@mkdir -p $(dir $@)
diff --git a/configure b/configure
index cc04a560a0..ce32c568f8 100755
--- a/configure
+++ b/configure
@@ -548,6 +548,8 @@ fuzzing="no"
 rng_none="no"
 secret_keyring=""
 libdaxctl=""
+meson=""
+skip_meson=no
 
 supported_cpu="no"
 supported_os="no"
@@ -1044,6 +1046,10 @@ for opt do
   ;;
   --sphinx-build=*) sphinx_build="$optarg"
   ;;
+  --skip-meson) skip_meson=yes
+  ;;
+  --meson=*) meson="$optarg"
+  ;;
   --gcov=*) gcov_tool="$optarg"
   ;;
   --smbd=*) smbd="$optarg"
@@ -1809,6 +1815,7 @@ Advanced options (experts only):
   --install=INSTALLuse specified install [$install]
   --python=PYTHON  use specified python [$python]
   --sphinx-build=SPHINXuse specified sphinx-build [$sphinx_build]
+  --meson

[PULL v8 043/152] meson: add msi generation

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile  | 18 --
 qga/meson.build   | 25 +
 qga/vss-win32/meson.build |  3 ++-
 3 files changed, 27 insertions(+), 19 deletions(-)

diff --git a/Makefile b/Makefile
index 0fa9d63bb4..e93ebf6299 100644
--- a/Makefile
+++ b/Makefile
@@ -450,23 +450,6 @@ qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx 
$(SRC_PATH)/scripts/hxtool
 qemu-keymap$(EXESUF): LIBS += $(XKBCOMMON_LIBS)
 qemu-keymap$(EXESUF): QEMU_CFLAGS += $(XKBCOMMON_CFLAGS)
 
-ifdef QEMU_GA_MSI_ENABLED
-QEMU_GA_MSI=qemu-ga-$(ARCH).msi
-
-msi: $(QEMU_GA_MSI)
-
-$(QEMU_GA_MSI): qga/qemu-ga.exe $(QGA_VSS_PROVIDER)
-
-$(QEMU_GA_MSI): config-host.mak
-
-$(QEMU_GA_MSI):  $(SRC_PATH)/qga/installer/qemu-ga.wxs
-   $(call quiet-command,QEMU_GA_VERSION="$(QEMU_GA_VERSION)" 
QEMU_GA_MANUFACTURER="$(QEMU_GA_MANUFACTURER)" 
QEMU_GA_DISTRO="$(QEMU_GA_DISTRO)" BUILD_DIR="$(BUILD_DIR)" \
-   wixl -o $@ $(QEMU_GA_MSI_ARCH) $(QEMU_GA_MSI_WITH_VSS) 
$(QEMU_GA_MSI_MINGW_DLL_PATH) $<,"WIXL","$@")
-else
-msi:
-   @echo "MSI build not configured or dependency resolution failed 
(reconfigure with --enable-guest-agent-msi option)"
-endif
-
 ifneq ($(EXESUF),)
 .PHONY: qga/qemu-ga
 qga/qemu-ga: qga/qemu-ga$(EXESUF) $(QGA_VSS_PROVIDER) $(QEMU_GA_MSI)
@@ -482,7 +465,6 @@ clean: recurse-clean ninja-clean clean-ctlist
 # avoid old build problems by removing potentially incorrect old files
rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h 
gen-op-arm.h
rm -f qemu-options.def
-   rm -f *.msi
find . \( -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name 
'*.[oda]' \) -type f \
! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \
! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
diff --git a/qga/meson.build b/qga/meson.build
index 9ddb260cff..33f6db2865 100644
--- a/qga/meson.build
+++ b/qga/meson.build
@@ -48,5 +48,30 @@ qga = executable('qemu-ga', qga_ss.sources(),
 if targetos == 'windows'
   if 'CONFIG_QGA_VSS' in config_host
 subdir('vss-win32')
+  else
+gen_tlb = []
+  endif
+
+  wixl = find_program('wixl', required: false)
+  if wixl.found()
+deps = [gen_tlb, qga]
+if 'CONFIG_QGA_VSS' in config_host and 'QEMU_GA_MSI_WITH_VSS' in 
config_host
+  deps += qga_vss
+endif
+qga_msi = custom_target('QGA MSI',
+input: files('installer/qemu-ga.wxs'),
+output: 
'qemu-ga-@0@.msi'.format(config_host['ARCH']),
+depends: deps,
+command: [
+  'QEMU_GA_VERSION=' + 
config_host['QEMU_GA_VERSION'],
+  'QEMU_GA_MANUFACTURER=' + 
config_host['QEMU_GA_MANUFACTURER'],
+  'QEMU_GA_DISTRO=' + 
config_host['QEMU_GA_DISTRO'],
+  'BUILD_DIR=' + meson.build_root(),
+  wixl, '-o', '@OUTPUT0@', '@INPUT0@',
+  config_host['QEMU_GA_MSI_ARCH'].split(),
+  config_host['QEMU_GA_MSI_WITH_VSS'].split(),
+  
config_host['QEMU_GA_MSI_MINGW_DLL_PATH'].split(),
+])
+alias_target('msi', qga_msi)
   endif
 endif
diff --git a/qga/vss-win32/meson.build b/qga/vss-win32/meson.build
index 42c8d31a3d..1f39e05335 100644
--- a/qga/vss-win32/meson.build
+++ b/qga/vss-win32/meson.build
@@ -2,7 +2,8 @@ if add_languages('cpp', required: false)
   glib_static = dependency('glib-2.0', static: true)
   link_args = cc.get_supported_link_arguments(['-fstack-protector-all', 
'-fstack-protector-strong',
'-Wl,--add-stdcall-alias', 
'-Wl,--enable-stdcall-fixup'])
-  shared_module('qga-vss', ['requester.cpp', 'provider.cpp', 'install.cpp'],
+
+  qga_vss = shared_module('qga-vss', ['requester.cpp', 'provider.cpp', 
'install.cpp'],
 name_prefix: '',
 cpp_args: ['-Wno-unknown-pragmas', 
'-Wno-delete-non-virtual-dtor', '-Wno-non-virtual-dtor'],
 link_args: link_args,
-- 
2.26.2





[PULL v8 047/152] meson: add qemu-edid

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Paolo Bonzini 
---
 Makefile| 2 --
 configure   | 2 +-
 meson.build | 4 
 3 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index b0a261bdc7..da1e556efe 100644
--- a/Makefile
+++ b/Makefile
@@ -429,8 +429,6 @@ qemu-nbd$(EXESUF): qemu-nbd.o $(authz-obj-y) $(block-obj-y) 
$(crypto-obj-y) $(io
 qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) 
$(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
 qemu-storage-daemon$(EXESUF): qemu-storage-daemon.o $(authz-obj-y) 
$(block-obj-y) $(crypto-obj-y) $(chardev-obj-y) $(io-obj-y) $(qom-obj-y) 
$(storage-daemon-obj-y) $(COMMON_LDADDS)
 
-qemu-edid$(EXESUF): qemu-edid.o hw/display/edid-generate.o $(COMMON_LDADDS)
-
 fsdev/virtfs-proxy-helper$(EXESUF): fsdev/virtfs-proxy-helper.o 
fsdev/9p-marshal.o fsdev/9p-iov-marshal.o $(COMMON_LDADDS)
 
 scsi/qemu-pr-helper$(EXESUF): scsi/qemu-pr-helper.o scsi/utils.o 
$(authz-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
diff --git a/configure b/configure
index 62f1bcf9ba..518491d234 100755
--- a/configure
+++ b/configure
@@ -6713,7 +6713,7 @@ fi
 
 tools=""
 if test "$want_tools" = "yes" ; then
-  tools="qemu-img\$(EXESUF) qemu-io\$(EXESUF) qemu-edid\$(EXESUF) $tools"
+  tools="qemu-img\$(EXESUF) qemu-io\$(EXESUF) $tools"
   if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then
 tools="qemu-nbd\$(EXESUF) qemu-storage-daemon\$(EXESUF) $tools"
   fi
diff --git a/meson.build b/meson.build
index 9ec7dd9ddc..f49f5e08d6 100644
--- a/meson.build
+++ b/meson.build
@@ -315,6 +315,10 @@ if have_tools
dependencies: [qemuutil, xkbcommon], install: true)
   endif
 
+  executable('qemu-edid', files('qemu-edid.c', 'hw/display/edid-generate.c'),
+ dependencies: qemuutil,
+ install: true)
+
   if 'CONFIG_VHOST_USER' in config_host
 subdir('contrib/libvhost-user')
 subdir('contrib/vhost-user-blk')
-- 
2.26.2





[PULL v8 051/152] meson: generate shader headers

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile  | 19 ---
 meson.build   |  1 +
 ui/meson.build|  2 ++
 ui/shader/meson.build | 15 +++
 4 files changed, 18 insertions(+), 19 deletions(-)
 create mode 100644 ui/shader/meson.build

diff --git a/Makefile b/Makefile
index 0b86e3a981..e7394e748d 100644
--- a/Makefile
+++ b/Makefile
@@ -393,7 +393,6 @@ clean: recurse-clean ninja-clean clean-ctlist
rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) TAGS cscope.* *.pod *~ 
*/*~
rm -f fsdev/*.pod scsi/*.pod
rm -f qemu-img-cmds.h
-   rm -f ui/shader/*-vert.h ui/shader/*-frag.h
rm -f $(foreach f,$(generated-files-y),$(f) $(f)-timestamp)
rm -f config-all-devices.mak
rm -f $(SUBDIR_DEVICES_MAK)
@@ -618,24 +617,6 @@ endif
$(MAKE) $(SUBDIR_MAKEFLAGS) TARGET_DIR=$$d/ -C $$d $@ || exit 1 ; \
 done
 
-# opengl shader programs
-ui/shader/%-vert.h: $(SRC_PATH)/ui/shader/%.vert 
$(SRC_PATH)/scripts/shaderinclude.pl
-   @mkdir -p $(dir $@)
-   $(call quiet-command,\
-   perl $(SRC_PATH)/scripts/shaderinclude.pl $< > $@,\
-   "VERT","$@")
-
-ui/shader/%-frag.h: $(SRC_PATH)/ui/shader/%.frag 
$(SRC_PATH)/scripts/shaderinclude.pl
-   @mkdir -p $(dir $@)
-   $(call quiet-command,\
-   perl $(SRC_PATH)/scripts/shaderinclude.pl $< > $@,\
-   "FRAG","$@")
-
-ui/shader.o: $(SRC_PATH)/ui/shader.c \
-   ui/shader/texture-blit-vert.h \
-   ui/shader/texture-blit-flip-vert.h \
-   ui/shader/texture-blit-frag.h
-
 # documentation
 MAKEINFO=makeinfo
 MAKEINFOINCLUDES= -I docs -I $(

[PULL v8 035/152] contrib/vhost-user-input: convert to meson

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile   | 11 ---
 Makefile.objs  |  1 -
 contrib/vhost-user-input/Makefile.objs |  1 -
 contrib/vhost-user-input/meson.build   |  5 +
 meson.build|  1 +
 5 files changed, 6 insertions(+), 13 deletions(-)
 delete mode 100644 contrib/vhost-user-input/Makefile.objs
 create mode 100644 contrib/vhost-user-input/meson.build

diff --git a/Makefile b/Makefile
index ad3d92..c0cb10cd77 100644
--- a/Makefile
+++ b/Makefile
@@ -311,7 +311,6 @@ dummy := $(call unnest-vars,, \
 elf2dmp-obj-y \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
-vhost-user-input-obj-y \
 vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
 block-obj-y \
@@ -535,16 +534,6 @@ endif
 vhost-user-gpu$(EXESUF): $(vhost-user-gpu-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
 
-ifdef CONFIG_VHOST_USER_INPUT
-ifdef CONFIG_LINUX
-vhost-user-input$(EXESUF): $(vhost-user-input-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
-   $(call LINK, $^)
-
-# build by default, do not install
-all: vhost-user-input$(EXESUF)
-endif
-endif
-
 module_block.h: $(SRC_PATH)/scripts/modules/module_block.py config-host.mak
$(call quiet-command,$(PYTHON) $< $@ \
$(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \
diff --git a/Makefile.objs b/Makefile.objs
index 0f80b63554..0922c3ed00 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -106,7 +106,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
 
 ##
diff --git a/contrib/vhost-user-input/Makefile.objs 
b/contrib/vhost-user-input/Makefile.objs
deleted file mode 100644
index b1fad90d51..00
--- a/contrib/vhost-user-input/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-vhost-user-input-obj-y = main.o
diff --git a/contrib/vhost-user-input/meson.build 
b/contrib/vhost-user-input/meson.build
new file mode 100644
index 00..1eeb1329d9
--- /dev/null
+++ b/contrib/vhost-user-input/meson.build
@@ -0,0 +1,5 @@
+executable('vhost-user-input', files('main.c'),
+   link_with: libvhost_user,
+   dependencies: qemuutil,
+   build_by_default: targetos == 'linux',
+   install: false)
diff --git a/meson.build b/meson.build
index c60ec5825b..371d571495 100644
--- a/meson.build
+++ b/meson.build
@@ -265,6 +265,7 @@ if have_tools
   if 'CONFIG_VHOST_USER' in config_host
 subdir('contrib/libvhost-user')
 subdir('contrib/vhost-user-blk')
+subdir('contrib/vhost-user-input')
 subdir('contrib/vhost-user-scsi')
   endif
 endif
-- 
2.26.2





[PULL v8 049/152] meson: keymap-gen

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile   | 38 --
 meson.build|  1 +
 ui/meson.build | 34 ++
 3 files changed, 35 insertions(+), 38 deletions(-)
 create mode 100644 ui/meson.build

diff --git a/Makefile b/Makefile
index 39633ec7db..c07b6bb5b8 100644
--- a/Makefile
+++ b/Makefile
@@ -141,44 +141,6 @@ generated-files-y += module_block.h
 
 generated-files-y += .git-submodule-status
 
-KEYCODEMAP_GEN = $(SRC_PATH)/ui/keycodemapdb/tools/keymap-gen
-KEYCODEMAP_CSV = $(SRC_PATH)/ui/keycodemapdb/data/keymaps.csv
-
-KEYCODEMAP_FILES = \
-ui/input-keymap-atset1-to-qcode.c.inc \
-ui/input-keymap-linux-to-qcode.c.inc \
-ui/input-keymap-qcode-to-atset1.c.inc \
-ui/input-keymap-qcode-to-atset2.c.inc \
-ui/input-keymap-qcode-to-atset3.c.inc \
-ui/input-keymap-qcode-to-linux.c.inc \
-ui/input-keymap-qcode-to-qnum.c.inc \
-ui/input-keymap-qcode-to-sun.c.inc \
-ui/input-keymap-qnum-to-qcode.c.inc \
-ui/input-keymap-usb-to-qcode.c.inc \
-ui/input-keymap-win32-to-qcode.c.inc \
-ui/input-keymap-x11-to-qcode.c.inc \
-ui/input-keymap-xorgevdev-to-qcode.c.inc \
-ui/input-keymap-xorgkbd-to-qcode.c.inc \
-ui/input-keymap-xorgxquartz-to-qcode.c.inc \
-ui/input-keymap-xorgxwin-to-qcode.c.inc \
-ui/input-keymap-osx-to-qcode.c.inc \
-$(NULL)
-
-generated-files-$(CONFIG_SOFTMMU) += $(KEYCODEMAP_FILES)
-
-ui/input-keymap-%.c.inc: $(KEYCODEMAP_GEN) $(KEYCODEMAP_CSV) 
$(SRC_PATH)/ui/Makefile.objs
-   $(call quiet-command,\
-   stem=$* && src=$${stem%-to-*} dst=$${stem#*-to-} && \
-   test -e $(KEYCODEMAP_GEN) && \
-   $(PYTHON) $(KEYCODEMAP_GEN) \
- --lang glib2 \
- --varname qemu_input_map_$${src}_to_$${dst} \
- code-map $(KEYCODEMAP_CSV) $${src} $${dst} \
-   > $@ || rm -f $@, "GEN", "$@")
-
-$(KEYCODEMAP_GEN): .git-submodule-status
-$(KEYCODEMAP_CSV): .git-submodule-status
-
 edk2-decompressed = $(basename $(wildcard pc-bios/edk2-*.fd.bz2))
 pc-bios/edk2-%.fd: pc-bios/edk2-%.fd.bz2
$(call quiet-command,bzip2 -d -c $< > $@,"BUNZIP2",$<)
diff --git a/meson.build b/meson.build
index 3e9e0a9b33..b4a2f9db3a 100644
--- a/meson.build
+++ b/meson.build
@@ -292,6 +292,7 @@ subdir('trace')
 subdir('util')
 subdir('crypto')
 subdir('storage-daemon')
+subdir('ui')
 
 # Build targets from sourcesets
 
diff --git a/ui/meson.build b/ui/meson.build
new file mode 100644
index 00..a6aa7f2b36
--- /dev/null
+++ b/ui/meson.build
@@ -0,0 +1,34 @@
+keymaps = [
+  ['atset1', 'qcode'],
+  ['linux', 'qcode'],
+  ['qcode', 'atset1'],
+  ['qcode', 'atset2'],
+  ['qcode', 'atset3'],
+  ['qcode', 'linux'],
+  ['qcode', 'qnum'],
+  ['qcode', 'sun'],
+  ['qnum', 'qcode'],
+  ['usb', 'qcode'],
+  ['win32', 'qcode'],
+  ['x11', 'qcode'],
+  ['xorgevdev', 'qcode'],
+  ['xorgkbd', 'qcode'],
+  ['xorgxquartz', 'qcode'],
+  ['xorgxwin', 'qcode'],
+  ['osx', 'qcode'],
+]
+
+if have_system
+  foreach e : keymaps
+output = 'input-keymap-@0@-to-@1...@.c.inc'.format(e[0], e[1])
+genh += custom_target(output,
+  output: output,
+  capture: true,
+  build_by_default: true, # to be removed when added to a 
target
+  input: files('keycodemapdb/data/keymaps.csv'),
+  command: [python.full_path(), 
files('keycodemapdb/tools/keymap-gen'),
+'--lang', 'glib2',
+'--varname', 
'qemu_input_map_@0@_to_@1@'.format(e[0], e[1]),
+'code-map', '@INPUT0@', e[0], e[1]])
+  endforeach
+endif
-- 
2.26.2





[PULL v8 052/152] meson: generate hxtool files

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Signed-off-by: Paolo Bonzini 
---
 Makefile|  8 +---
 Makefile.target |  6 --
 meson.build | 22 ++
 scripts/hxtool  |  2 +-
 4 files changed, 24 insertions(+), 14 deletions(-)
 mode change 100644 => 100755 scripts/hxtool

diff --git a/Makefile b/Makefile
index e7394e748d..13df463e33 100644
--- a/Makefile
+++ b/Makefile
@@ -121,7 +121,7 @@ include $(SRC_PATH)/rules.mak
 # lor is defined in rules.mak
 CONFIG_BLOCK := $(call lor,$(CONFIG_SOFTMMU),$(CONFIG_TOOLS))
 
-generated-files-y = config-host.h qemu-options.def
+generated-files-y = config-host.h
 
 generated-files-y += module_block.h
 
@@ -263,8 +263,6 @@ all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) 
$(HELPERS-y) recurse-all mo
 
 config-host.h: config-host.h-timestamp
 config-host.h-timestamp: config-host.mak
-qemu-options.def: $(SRC_PATH)/qemu-options.hx $(SRC_PATH)/scripts/hxtool
-   $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > 
$@,"GEN","$@")
 
 TARGET_DIRS_RULES := $(foreach t, all fuzz clean install, $(addsuffix /$(t), 
$(TARGET_DIRS)))
 
@@ -371,9 +369,6 @@ ifdef CONFIG_MPATH
 scsi/qemu-pr-helper$(EXESUF): LIBS += -ludev -lmultipath -lmpathpersist
 endif
 
-qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
-   $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > 
$@,"GEN","$@")
-
 module_block.h: $(SRC_PATH)/scripts/modules/module_block.py config-host.mak
$(call quiet-command,$(PYTHON) $< $@ \
$(addprefix $(SRC_PATH)/,$(patsubst %.mo,%.c,$(block-obj-m))), \
@@ -392,7 +387,6 @@ clean: recurse-clean ninja-clean clean-ctlist
rm -f $(edk2-decompressed)
rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) TAGS cscope.* *.pod *~ 
*/*~
rm -f fsdev/*.pod scsi/*.pod
-   rm -f qemu-img-cmds.h
rm -f $(foreach f,$(generated-files-y),$(f) $(f)-timestamp)
rm -f config-all-devices.mak
rm -f $(SUBDIR_DEVICES_MAK)
diff --git a/Makefile.target b/Makefile.target
index 0b323641bb..257afc2723 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -225,12 +225,6 @@ endif
 gdbstub-xml.c: $(TARGET_XML_FILES) $(SRC_PATH)/scripts/feature_to_c.sh
$(call quiet-command,rm -f $@ && $(SHELL) 
$(SRC_PATH)/scripts/feature_to_c.sh $@ 
$(TARGET_XML_FILES),"GEN","$(TARGET_DIR)$@")
 
-hmp-commands.h: $(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/scripts/hxtool
-   $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > 
$@,"GEN","$(TARGET_DIR)$@")
-
-hmp-commands-info.h: $(SRC_PATH)/hmp-commands-info.hx 
$(SRC_PATH)/scripts/hxtool
-   $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > 
$@,"GEN","$(TARGET_DIR)$@")
-
 clean: clean-target
rm -f *.a *~ $(PROGS)
rm -f $(shell find . -name '*.[od]')
diff --git a/meson.build b/meson.build
index 2fb209b228..222b0d0e77 100644
--- a/meson.build
+++ b/meson.build
@@ -159,6 +159,7 @@ have_block = have_system or have_tools
 # Generators
 
 genh = []
+hxtool = find_program('scripts/hxtool')
 shaderinclude = find_program('scripts/shaderinclude.pl')
 qapi_gen = find_program('scripts/qapi-gen.py')
 qapi_gen_depends = [ meson.source_root() / 'scripts/qapi/__init__.py',
@@ -196,6 +197,27 @@ qemu_version = custom_target('qemu-version.h',
  build_always_stale: true)
 genh += qemu_version
 
+hxdep = []
+hx_headers = [
+  ['qemu-options.hx', 'qemu-options.def'],
+  ['qemu-img-cmds.hx', 'qemu-img-cmds.h'],
+]
+if have_system
+  hx_headers += [
+['hmp-commands.hx', 'hmp-commands.h'],
+['hmp-commands-info.hx', 'hmp-commands-info.h'],
+  ]
+endif
+foreach d : hx_headers
+  custom_target(d[1],
+input: files(d[0]),
+output: d[1],
+capture: true,
+build_by_default: true, # to be removed when added to a target
+command: [hxtool, '-h', '@INPUT0@'])
+endforeach
+genh += hxdep
+
 # Collect sourcesets.
 
 util_ss = ss.source_set()
diff --git a/scripts/hxtool b/scripts/hxtool
old mode 100644
new mode 100755
index 7b1452f3cf..80516b9437
--- a/scripts/hxtool
+++ b/scripts/hxtool
@@ -19,6 +19,6 @@ hxtoh()
 case "$1" in
 "-h") hxtoh ;;
 *) exit 1 ;;
-esac
+esac < "$2"
 
 exit 0
-- 
2.26.2





Re: [PATCH v5 14/15] block/nvme: Extract nvme_poll_queue()

2020-08-21 Thread Stefano Garzarella
On Thu, Aug 20, 2020 at 06:59:00PM +0200, Philippe Mathieu-Daudé wrote:
> As we want to do per-queue polling, extract the nvme_poll_queue()
> method which operates on a single queue.
> 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  block/nvme.c | 44 +++-
>  1 file changed, 27 insertions(+), 17 deletions(-)

Reviewed-by: Stefano Garzarella 

> 
> diff --git a/block/nvme.c b/block/nvme.c
> index 1f67e888c84..a61e86a83eb 100644
> --- a/block/nvme.c
> +++ b/block/nvme.c
> @@ -590,31 +590,41 @@ out:
>  qemu_vfree(id);
>  }
>  
> +static bool nvme_poll_queue(NVMeQueuePair *q)
> +{
> +bool progress = false;
> +
> +const size_t cqe_offset = q->cq.head * NVME_CQ_ENTRY_BYTES;
> +NvmeCqe *cqe = (NvmeCqe *)&q->cq.queue[cqe_offset];
> +
> +/*
> + * Do an early check for completions. q->lock isn't needed because
> + * nvme_process_completion() only runs in the event loop thread and
> + * cannot race with itself.
> + */
> +if ((le16_to_cpu(cqe->status) & 0x1) == q->cq_phase) {
> +return false;
> +}
> +
> +qemu_mutex_lock(&q->lock);
> +while (nvme_process_completion(q)) {
> +/* Keep polling */
> +progress = true;
> +}
> +qemu_mutex_unlock(&q->lock);
> +
> +return progress;
> +}
> +
>  static bool nvme_poll_queues(BDRVNVMeState *s)
>  {
>  bool progress = false;
>  int i;
>  
>  for (i = 0; i < s->nr_queues; i++) {
> -NVMeQueuePair *q = s->queues[i];
> -const size_t cqe_offset = q->cq.head * NVME_CQ_ENTRY_BYTES;
> -NvmeCqe *cqe = (NvmeCqe *)&q->cq.queue[cqe_offset];
> -
> -/*
> - * Do an early check for completions. q->lock isn't needed because
> - * nvme_process_completion() only runs in the event loop thread and
> - * cannot race with itself.
> - */
> -if ((le16_to_cpu(cqe->status) & 0x1) == q->cq_phase) {
> -continue;
> -}
> -
> -qemu_mutex_lock(&q->lock);
> -while (nvme_process_completion(q)) {
> -/* Keep polling */
> +if (nvme_poll_queue(s->queues[i])) {
>  progress = true;
>  }
> -qemu_mutex_unlock(&q->lock);
>  }
>  return progress;
>  }
> -- 
> 2.26.2
> 
> 




[PULL v8 031/152] contrib/vhost-user-blk: convert to Meson

2020-08-21 Thread Paolo Bonzini
The GLib pkg-config information is extracted from config-host.mak and
used to link vhost-user-blk.

Signed-off-by: Paolo Bonzini 
---
 Makefile | 3 ---
 Makefile.objs| 1 -
 contrib/vhost-user-blk/Makefile.objs | 1 -
 contrib/vhost-user-blk/meson.build   | 6 ++
 meson.build  | 1 +
 5 files changed, 7 insertions(+), 5 deletions(-)
 delete mode 100644 contrib/vhost-user-blk/Makefile.objs
 create mode 100644 contrib/vhost-user-blk/meson.build

diff --git a/Makefile b/Makefile
index e591d1b2b4..e387b67a3b 100644
--- a/Makefile
+++ b/Makefile
@@ -313,7 +313,6 @@ dummy := $(call unnest-vars,, \
 ivshmem-server-obj-y \
 rdmacm-mux-obj-y \
 vhost-user-scsi-obj-y \
-vhost-user-blk-obj-y \
 vhost-user-input-obj-y \
 vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
@@ -536,8 +535,6 @@ ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) 
$(COMMON_LDADDS)
 endif
 vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
$(call LINK, $^)
-vhost-user-blk$(EXESUF): $(vhost-user-blk-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
-   $(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index fee0f71372..f69736c10c 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -109,7 +109,6 @@ ivshmem-server-obj-$(CONFIG_IVSHMEM) = 
contrib/ivshmem-server/
 vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
 vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
 vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
-vhost-user-blk-obj-y = contrib/vhost-user-blk/
 rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
diff --git a/contrib/vhost-user-blk/Makefile.objs 
b/contrib/vhost-user-blk/Makefile.objs
deleted file mode 100644
index 72e2cdc3ad..00
--- a/contrib/vhost-user-blk/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-vhost-user-blk-obj-y = vhost-user-blk.o
diff --git a/contrib/vhost-user-blk/meson.build 
b/contrib/vhost-user-blk/meson.build
new file mode 100644
index 00..5db8cc3fe2
--- /dev/null
+++ b/contrib/vhost-user-blk/meson.build
@@ -0,0 +1,6 @@
+# FIXME: broken on 32-bit architectures
+executable('vhost-user-blk', files('vhost-user-blk.c'),
+   link_with: libvhost_user,
+   dependencies: qemuutil,
+   build_by_default: false,
+   install: false)
diff --git a/meson.build b/meson.build
index 00970e6e19..4343f0be6c 100644
--- a/meson.build
+++ b/meson.build
@@ -257,6 +257,7 @@ qemuutil = declare_dependency(link_with: libqemuutil,
 if have_tools
   if 'CONFIG_VHOST_USER' in config_host
 subdir('contrib/libvhost-user')
+subdir('contrib/vhost-user-blk')
   endif
 endif
 
-- 
2.26.2





[PULL v8 056/152] meson: convert check-qapi-schema

2020-08-21 Thread Paolo Bonzini
Signed-off-by: Paolo Bonzini 
---
 tests/Makefile.include| 218 +---
 tests/meson.build |   2 +
 tests/qapi-schema/meson.build | 225 ++
 3 files changed, 229 insertions(+), 216 deletions(-)
 create mode 100644 tests/qapi-schema/meson.build

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8fb6baa0a1..02d31fbe0c 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -161,201 +161,6 @@ check-unit-y += tests/test-qapi-util$(EXESUF)
 
 check-block-$(call land,$(CONFIG_POSIX),$(CONFIG_SOFTMMU)) += 
tests/check-block.sh
 
-qapi-schema += alternate-any.json
-qapi-schema += alternate-array.json
-qapi-schema += alternate-base.json
-qapi-schema += alternate-branch-if-invalid.json
-qapi-schema += alternate-clash.json
-qapi-schema += alternate-conflict-dict.json
-qapi-schema += alternate-conflict-enum-bool.json
-qapi-schema += alternate-conflict-enum-int.json
-qapi-schema += alternate-conflict-string.json
-qapi-schema += alternate-conflict-bool-string.json
-qapi-schema += alternate-conflict-num-string.json
-qapi-schema += alternate-empty.json
-qapi-schema += alternate-invalid-dict.json
-qapi-schema += alternate-nested.json
-qapi-schema += alternate-unknown.json
-qapi-schema += args-alternate.json
-qapi-schema += args-any.json
-qapi-schema += args-array-empty.json
-qapi-schema += args-array-unknown.json
-qapi-schema += args-bad-boxed.json
-qapi-schema += args-boxed-anon.json
-qapi-schema += args-boxed-string.json
-qapi-schema += args-int.json
-qapi-schema += args-invalid.json
-qapi-schema += args-member-array-bad.json
-qapi-schema += args-member-case.json
-qapi-schema += args-member-unknown.json
-qapi-schema += args-name-clash.json
-qapi-schema += args-union.json
-qapi-schema += args-unknown.json
-qapi-schema += bad-base.json
-qapi-schema += bad-data.json
-qapi-schema += bad-ident.json
-qapi-schema += bad-if.json
-qapi-schema += bad-if-empty.json
-qapi-schema += bad-if-empty-list.json
-qapi-schema += bad-if-list.json
-qapi-schema += bad-type-bool.json
-qapi-schema += bad-type-dict.json
-qapi-schema += bad-type-int.json
-qapi-schema += base-cycle-direct.json
-qapi-schema += base-cycle-indirect.json
-qapi-schema += command-int.json
-qapi-schema += comments.json
-qapi-schema += doc-bad-alternate-member.json
-qapi-schema += doc-bad-boxed-command-arg.json
-qapi-schema += doc-bad-command-arg.json
-qapi-schema += doc-bad-enum-member.json
-qapi-schema += doc-bad-event-arg.json
-qapi-schema += doc-bad-feature.json
-qapi-schema += doc-bad-section.json
-qapi-schema += doc-bad-symbol.json
-qapi-schema += doc-bad-union-member.json
-qapi-schema += doc-before-include.json
-qapi-schema += doc-before-pragma.json
-qapi-schema += doc-duplicated-arg.json
-qapi-schema += doc-duplicated-return.json
-qapi-schema += doc-duplicated-since.json
-qapi-schema += doc-empty-arg.json
-qapi-schema += doc-empty-section.json
-qapi-schema += doc-empty-symbol.json
-qapi-schema += doc-good.json
-qapi-schema += doc-interleaved-section.json
-qapi-schema += doc-invalid-end.json
-qapi-schema += doc-invalid-end2.json
-qapi-schema += doc-invalid-return.json
-qapi-schema += doc-invalid-section.json
-qapi-schema += doc-invalid-start.json
-qapi-schema += doc-missing-colon.json
-qapi-schema += doc-missing-expr.json
-qapi-schema += doc-missing-space.json
-qapi-schema += doc-missing.json
-qapi-schema += doc-no-symbol.json
-qapi-schema += doc-undoc-feature.json
-qapi-schema += double-type.json
-qapi-schema += duplicate-key.json
-qapi-schema += empty.json
-qapi-schema += enum-bad-member.json
-qapi-schema += enum-bad-name.json
-qapi-schema += enum-bad-prefix.json
-qapi-schema += enum-clash-member.json
-qapi-schema += enum-dict-member-unknown.json
-qapi-schema += enum-if-invalid.json
-qapi-schema += enum-int-member.json
-qapi-schema += enum-member-case.json
-qapi-schema += enum-missing-data.json
-qapi-schema += enum-wrong-data.json
-qapi-schema += event-boxed-empty.json
-qapi-schema += event-case.json
-qapi-schema += event-member-invalid-dict.json
-qapi-schema += event-nest-struct.json
-qapi-schema += features-bad-type.json
-qapi-schema += features-deprecated-type.json
-qapi-schema += features-duplicate-name.json
-qapi-schema += features-if-invalid.json
-qapi-schema += features-missing-name.json
-qapi-schema += features-name-bad-type.json
-qapi-schema += features-no-list.json
-qapi-schema += features-unknown-key.json
-qapi-schema += flat-union-array-branch.json
-qapi-schema += flat-union-bad-base.json
-qapi-schema += flat-union-bad-discriminator.json
-qapi-schema += flat-union-base-any.json
-qapi-schema += flat-union-base-union.json
-qapi-schema += flat-union-clash-member.json
-qapi-schema += flat-union-discriminator-bad-name.json
-qapi-schema += flat-union-empty.json
-qapi-schema += flat-union-inline.json
-qapi-schema += flat-union-inline-invalid-dict.json
-qapi-schema += flat-union-int-branch.json
-qapi-schema += flat-union-invalid-branch-key.json
-qapi-schema +

[PULL v8 033/152] contrib/vhost-user-scsi: convert to Meson

2020-08-21 Thread Paolo Bonzini
The libiscsi pkg-config information is extracted from config-host.mak and
used to link vhost-user-blk.

Signed-off-by: Paolo Bonzini 
---
 Makefile  | 3 ---
 Makefile.objs | 3 ---
 contrib/vhost-user-scsi/Makefile.objs | 1 -
 contrib/vhost-user-scsi/meson.build   | 7 +++
 meson.build   | 6 ++
 5 files changed, 13 insertions(+), 7 deletions(-)
 delete mode 100644 contrib/vhost-user-scsi/Makefile.objs
 create mode 100644 contrib/vhost-user-scsi/meson.build

diff --git a/Makefile b/Makefile
index e387b67a3b..1f5a710f77 100644
--- a/Makefile
+++ b/Makefile
@@ -312,7 +312,6 @@ dummy := $(call unnest-vars,, \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
 rdmacm-mux-obj-y \
-vhost-user-scsi-obj-y \
 vhost-user-input-obj-y \
 vhost-user-gpu-obj-y \
 qga-vss-dll-obj-y \
@@ -533,8 +532,6 @@ ivshmem-client$(EXESUF): $(ivshmem-client-obj-y) 
$(COMMON_LDADDS)
 ivshmem-server$(EXESUF): $(ivshmem-server-obj-y) $(COMMON_LDADDS)
$(call LINK, $^)
 endif
-vhost-user-scsi$(EXESUF): $(vhost-user-scsi-obj-y) 
contrib/libvhost-user/libvhost-user.a $(COMMON_LDADDS)
-   $(call LINK, $^)
 
 rdmacm-mux$(EXESUF): LIBS += "-libumad"
 rdmacm-mux$(EXESUF): $(rdmacm-mux-obj-y) $(COMMON_LDADDS)
diff --git a/Makefile.objs b/Makefile.objs
index f69736c10c..ab798b08a7 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -106,9 +106,6 @@ qga-vss-dll-obj-y = qga/
 elf2dmp-obj-y = contrib/elf2dmp/
 ivshmem-client-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-client/
 ivshmem-server-obj-$(CONFIG_IVSHMEM) = contrib/ivshmem-server/
-vhost-user-scsi.o-cflags := $(LIBISCSI_CFLAGS)
-vhost-user-scsi.o-libs := $(LIBISCSI_LIBS)
-vhost-user-scsi-obj-y = contrib/vhost-user-scsi/
 rdmacm-mux-obj-y = contrib/rdmacm-mux/
 vhost-user-input-obj-y = contrib/vhost-user-input/
 vhost-user-gpu-obj-y = contrib/vhost-user-gpu/
diff --git a/contrib/vhost-user-scsi/Makefile.objs 
b/contrib/vhost-user-scsi/Makefile.objs
deleted file mode 100644
index e83a38a85b..00
--- a/contrib/vhost-user-scsi/Makefile.objs
+++ /dev/null
@@ -1 +0,0 @@
-vhost-user-scsi-obj-y = vhost-user-scsi.o
diff --git a/contrib/vhost-user-scsi/meson.build 
b/contrib/vhost-user-scsi/meson.build
new file mode 100644
index 00..257cbffc8e
--- /dev/null
+++ b/contrib/vhost-user-scsi/meson.build
@@ -0,0 +1,7 @@
+if 'CONFIG_LIBISCSI' in config_host
+  executable('vhost-user-scsi', files('vhost-user-scsi.c'),
+ link_with: libvhost_user,
+ dependencies: [qemuutil, libiscsi],
+ build_by_default: targetos == 'linux',
+ install: false)
+endif
diff --git a/meson.build b/meson.build
index 4343f0be6c..4c067d9fe6 100644
--- a/meson.build
+++ b/meson.build
@@ -92,6 +92,11 @@ libcap_ng = not_found
 if 'CONFIG_LIBCAP_NG' in config_host
   libcap_ng = declare_dependency(link_args: 
config_host['LIBCAP_NG_LIBS'].split())
 endif
+libiscsi = not_found
+if 'CONFIG_LIBISCSI' in config_host
+  libiscsi = declare_dependency(compile_args: 
config_host['LIBISCSI_CFLAGS'].split(),
+link_args: 
config_host['LIBISCSI_LIBS'].split())
+endif
 
 target_dirs = config_host['TARGET_DIRS'].split()
 have_user = false
@@ -258,6 +263,7 @@ if have_tools
   if 'CONFIG_VHOST_USER' in config_host
 subdir('contrib/libvhost-user')
 subdir('contrib/vhost-user-blk')
+subdir('contrib/vhost-user-scsi')
   endif
 endif
 
-- 
2.26.2





Re: [PATCH] configure: add support for psuedo-"in source tree" builds

2020-08-21 Thread Kevin Wolf
Am 21.08.2020 um 12:14 hat Daniel P. Berrangé geschrieben:
> On Fri, Aug 21, 2020 at 11:58:21AM +0200, Kevin Wolf wrote:
> > Am 20.08.2020 um 19:42 hat Eric Blake geschrieben:
> > > On 8/20/20 11:55 AM, Daniel P. Berrangé wrote:
> > > > Meson requires the build dir to be separate from the source tree. Many
> > > > people are used to just running "./configure && make" though and the
> > > > meson conversion breaks that.
> > > > 
> > > > This introduces some backcompat support to make it appear as if an
> > > > "in source tree" build is being done, but with the the results in the
> > > > "build/" directory. This allows "./configure && make" to work as it
> > > > did historically, albeit with the output binaries staying under build/.
> > > > 
> > > > Signed-off-by: Daniel P. Berrangé 
> > > > ---
> > > 
> > > In addition to reviews you already have,
> > > 
> > > 
> > > > I've not tested it beyond that. Note it blows away the "build/"
> > > > dir each time ./configure is run so it is pristine each time.
> > > 
> > > I definitely like the idea of only blowing away what we created - but if 
> > > we
> > > created build, then recreating it for each new configure run is nice.
> > 
> > I think I actually wouldn't automatically remove anything on configure.
> > It can be surprising behaviour for configure to delete directories, and
> > the old setup didn't do an automatic "make clean" either. By having a
> > separate build directory, manually emptying as needed has already become
> > easier.
> 
> The issue is that previously you could do
> 
>   ./configure
>   make
>   ./configure
>   make
> 
> This isn't possible with the new system because meson will refuse
> to use the "build/" directory if it already contains a previous
> configured build.

Oh. So now you always have to create a new build directory if you want
to change configure options?

> Doing "rm -rf build" in configure lets the above sequence work.
> 
> I can remove the "rm -rf biuld" in configure if we are happy
> to require
> 
>   ./configure
>   make
>   make distclean
>   ./configure
>   make
> 
> because the "GNUmakefile" wires up "distclean" to purge the
> build/ directory.

Hm, I see. Then maybe better keep the 'rm'.

> > > > We could optionally symlink binaries from build/ into $PWD
> > > > if poeople think that is important, eg by changing GNUmakefile
> > > > to have:
> > > > 
> > > > recurse: all
> > > > for bin in `find build -maxdepth 1 -type f -executable | grep 
> > > > -v -E '(ninjatool|config.status)'`; \
> > > 
> > > Using -maxdepth gets rid of the need to pre-create empty directories for
> > > nested binaries, but also loses out on binaries such as
> > > x86_64-softmmu/qemu-system-x86_64.  Oh, it looks like meson creates
> > > qemu-system-x86_64 as a binary in the top level, then a symlink in its old
> > > location.  Populating symlinks to ALL old locations is thus trickier than
> > > what you are proposing here, so it is fine to save that for a followup 
> > > patch
> > > (let's get the bare minimum in first, so that at least ./configure && make
> > > works, before we worry about back-compat symlinks).
> > 
> > Having the system emulator symlinks in the top level would be a change,
> > but even more convenient than the original places. I'd vote for adding
> > the auto-symlinking at least for the tools; if the top-level symlinks
> > for system emulators get also symlinked by this, that's fine, too.
> > 
> > I was actually surprised that Dan reports "make check" from the source
> > tree to be working without the symlinks. Some code must be cleverer than
> > I thought!
> 
> That's because "make check" is not actually running from the source
> tree. When you run "make check" in the source tree, what's acutally
> happening is
> 
>   cd build
>   make check
> 
> so it is actually running from build-dir context.

Yup, I wasn't aware that it would do this.

Kevin




[PULL v8 053/152] meson: handle edk2 bios and descriptors

2020-08-21 Thread Paolo Bonzini
From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Paolo Bonzini 
---
 Makefile| 29 -
 meson.build |  1 +
 pc-bios/descriptors/meson.build | 14 ++
 pc-bios/meson.build | 27 +++
 4 files changed, 42 insertions(+), 29 deletions(-)
 create mode 100644 pc-bios/descriptors/meson.build
 create mode 100644 pc-bios/meson.build

diff --git a/Makefile b/Makefile
index 13df463e33..bc7647d37b 100644
--- a/Makefile
+++ b/Makefile
@@ -127,10 +127,6 @@ generated-files-y += module_block.h
 
 generated-files-y += .git-submodule-status
 
-edk2-decompressed = $(basename $(wildcard pc-bios/edk2-*.fd.bz2))
-pc-bios/edk2-%.fd: pc-bios/edk2-%.fd.bz2
-   $(call quiet-command,bzip2 -d -c $< > $@,"BUNZIP2",$<)
-
 # Don't try to regenerate Makefile or configure
 # We don't generate any of them
 Makefile: ;
@@ -274,9 +270,6 @@ $(SOFTMMU_ALL_RULES): $(chardev-obj-y)
 $(SOFTMMU_ALL_RULES): $(crypto-obj-y)
 $(SOFTMMU_ALL_RULES): $(io-obj-y)
 $(SOFTMMU_ALL_RULES): config-all-devices.mak
-ifdef DECOMPRESS_EDK2_BLOBS
-$(SOFTMMU_ALL_RULES): $(edk2-decompressed)
-endif
 
 SOFTMMU_FUZZ_RULES=$(filter %-softmmu/fuzz, $(TARGET_DIRS_RULES))
 $(SOFTMMU_FUZZ_RULES): $(authz-obj-y)
@@ -384,7 +377,6 @@ clean: recurse-clean ninja-clean clean-ctlist
! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll 
\
-exec rm {} +
-   rm -f $(edk2-decompressed)
rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) TAGS cscope.* *.pod *~ 
*/*~
rm -f fsdev/*.pod scsi/*.pod
rm -f $(foreach f,$(generated-files-y),$(f) $(f)-timestamp)
@@ -461,13 +453,8 @@ edk2-licenses.txt \
 hppa-firmware.img \
 opensbi-riscv32-sifive_u-fw_jump.bin opensbi-riscv32-virt-fw_jump.bin \
 opensbi-riscv64-sifive_u-fw_jump.bin opensbi-riscv64-virt-fw_jump.bin
-
-
-DESCS=50-edk2-i386-secure.json 50-edk2-x86_64-secure.json \
-60-edk2-aarch64.json 60-edk2-arm.json 60-edk2-i386.json 60-edk2-x86_64.json
 else
 BLOBS=
-DESCS=
 endif
 
 # Note that we manually filter-out the non-Sphinx documentation which
@@ -544,7 +531,6 @@ install-includedir:
 export DESTDIR
 install: all $(if $(BUILD_DOCS),install-doc) \
install-datadir install-localstatedir install-includedir \
-   $(if $(INSTALL_BLOBS),$(edk2-decompressed)) \
recurse-install
 ifneq ($(TOOLS),)
$(call install-prog,$(TOOLS),$(DESTDIR)$(bindir))
@@ -567,21 +553,6 @@ ifneq ($(BLOBS),)
set -e; for x in $(BLOBS); do \
$(INSTALL_DATA) $(SRC_PATH)/pc-bios/$$x 
"$(DESTDIR)$(qemu_datadir)"; \
done
-endif
-ifdef INSTALL_BLOBS
-   set -e; for x in $(edk2-decompressed); do \
-   $(INSTALL_DATA) $$x "$(DESTDIR)$(qemu_datadir)"; \
-   done
-endif
-ifneq ($(DESCS),)
-   $(INSTALL_DIR) "$(DESTDIR)$(qemu_datadir)/firmware"
-   set -e; tmpf=$$(mktemp); trap 'rm -f -- "$$tmpf"' EXIT; \
-   for x in $(DESCS); do \
-   sed -e 's,@DATADIR@,$(qemu_datadir),' \
-   "$(SRC_PATH)/pc-bios/descriptors/$$x" > "$$tmpf"; \
-   $(INSTALL_DATA) "$$tmpf" \
-   "$(DESTDIR)$(qemu_datadir)/firmware/$$x"; \
-   done
 endif
for s in $(ICON_SIZES); do \
mkdir -p "$(DESTDIR)$(qemu_icondir)/hicolor/$${s}/apps"; \
diff --git a/meson.build b/meson.build
index 222b0d0e77..528198b6bd 100644
--- a/meson.build
+++ b/meson.build
@@ -383,6 +383,7 @@ if have_tools
 endif
 
 subdir('tools')
+subdir('pc-bios')
 
 summary_info = {}
 summary_info += {'Install prefix':config_host['prefix']}
diff --git a/pc-bios/descriptors/meson.build b/pc-bios/descriptors/meson.build
new file mode 100644
index 00..7c715bace8
--- /dev/null
+++ b/pc-bios/descriptors/meson.build
@@ -0,0 +1,14 @@
+foreach f: [
+  '50-edk2-i386-secure.json',
+  '50-edk2-x86_64-secure.json',
+  '60-edk2-aarch64.json',
+  '60-edk2-arm.json',
+  '60-edk2-i386.json',
+  '60-edk2-x86_64.json'
+]
+  configure_file(input: files(f),
+ output: f,
+ configuration: {'DATADIR': config_host['qemu_datadir']},
+ install: install_blobs,
+ install_dir: config_host['qemu_datadir'] / 'firmware')
+endforeach
diff --git a/pc-bios/meson.build b/pc-bios/meson.build
new file mode 100644
index 00..6e3bfe3ca4
--- /dev/null
+++ b/pc-bios/meson.build
@@ -0,0 +1,27 @@
+bzip2 = find_program('bzip2')
+
+install_blobs = 'INSTALL_BLOBS' in config_host
+if 'DECOMPRESS_EDK2_BLOBS' in config_host
+  fds = [
+'edk2-aarch64-code.fd',
+'edk2-arm-code.fd',
+'edk2-arm-vars.fd',
+'edk2-i386-code.fd',
+'edk2-i386-secure-code.fd',
+'edk2-i386-vars.fd',
+'edk2-x86_64-code.fd',
+'edk2-x86_64-secure-code.fd',
+  ]
+
+  foreach f : fds
+custom_target(f,
+  

  1   2   3   4   5   6   >