Re: [PATCH v1 3/8] qcow2: add zstd cluster compression

[Denis Plotnikov]

On 27.02.2020 17:01, Eric Blake wrote:

On 2/27/20 1:29 AM, Denis Plotnikov wrote:

zstd significantly reduces cluster compression time.
It provides better compression performance maintaining
the same level of the compression ratio in comparison with
zlib, which, at the moment, is the only compression
method available.




+static ssize_t qcow2_zstd_compress(void *dest, size_t dest_size,
+   const void *src, size_t src_size)
+{
+    size_t ret;
+
+    /*
+ * steal ZSTD_LEN_BUF bytes in the very beginng of the buffer


beginning


+ * to store compressed chunk size
+ */
+    char *d_buf = ((char *) dest) + ZSTD_LEN_BUF;
+
+    /*
+ * sanity check that we can store the compressed data length,
+ * and there is some space left for the compressor buffer
+ */
+    if (dest_size <= ZSTD_LEN_BUF) {
+    return -ENOMEM;
+    }
+
+    dest_size -= ZSTD_LEN_BUF;
+
+    ret = ZSTD_compress(d_buf, dest_size, src, src_size, 5);
+
+    if (ZSTD_isError(ret)) {
+    if (ZSTD_getErrorCode(ret) == ZSTD_error_dstSize_tooSmall) {
+    return -ENOMEM;
+    } else {
+    return -EIO;
+    }
+    }
+
+    /* paraniod sanity check that we can store the commpressed size */


paranoid, compressed


+    if (ret > UINT_MAX) {
+    return -ENOMEM;
+    }


This is pointless.  Better is to ensure that we actually compressed 
data (the pigeonhole principle states that there are some inputs that 
MUST result in inflation, in order for most other inputs to result in 
compression).  But that check was satisfied by checking for 
ZSTD_error_dstSize_tooSmall, which is what happens for one of those 
uncompressible inputs.  Namely, zstd will never return a result larger 
than dest_size, and since dest_size is smaller than UINT_MAX on entry, 
this check is pointless.  But if you want something, I'd be okay with: 
assert(ret <= dest_size).
Taking into account that this is "just in case" and I'm trying to 
protect the first 4 bytes of the buffer from the overflow and
I can't imagine the situation when we deal with cluster sizes greater 
than UINT32_MAX but the input size is size_t which can be > UINT32_MAX 
on 64bit archs.

I'd rather stick with
    if (ret > UINT32_MAX) {
    return -ENOMEM;
    }
as Vladimir suggested.

I'm not sure that the assert is good here, since it stops the system 
operating and this isn't potential error but a limitation

Does it work for you?

Denis



+++ b/docs/interop/qcow2.txt
@@ -208,6 +208,7 @@ version 2.
    Available compression type values:
  0: zlib 
+    1: zstd 
      === Header padding ===
@@ -575,11 +576,28 @@ Compressed Clusters Descriptor (x = 62 - 
(cluster_bits - 8)):
  Another compressed cluster may map to the tail 
of the final

  sector used by this compressed cluster.
  +    The layout of the compressed data depends on 
the compression
+    type used for the image (see compressed cluster 
layout).

+
  If a cluster is unallocated, read requests shall read the data from 
the backing
  file (except if bit 0 in the Standard Cluster Descriptor is set). 
If there is
  no backing file or the backing file is smaller than the image, they 
shall read

  zeros for all parts that are not covered by the backing file.
  +=== Compressed Cluster Layout ===
+
+The compressed cluster data has a layout depending on the compression
+type used for the image, as follows:
+
+Compressed data layout for the available compression types:
+(x = data_space_length - 1)
+
+    0:  (default)  zlib :
+    Byte  0 -  x: the compressed data content
+  all the space provided used for 
compressed data

+    1:  zstd :
+    Byte  0 -  3: the length of compressed data in bytes
+  4 -  x: the compressed data content
    == Snapshots ==
  diff --git a/qapi/block-core.json b/qapi/block-core.json
index 873fbef3b5..4b6e576c44 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4401,11 +4401,12 @@
  # Compression type used in qcow2 image file
  #
  # @zlib:  zlib compression, see 
+# @zstd:  zstd compression, see 
  #
  # Since: 5.0
  ##
  { 'enum': 'Qcow2CompressionType',
-  'data': [ 'zlib' ] }
+  'data': [ 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } 
] }


The spec and UI changes are okay.

Re: Strange data corruption issue with gluster (libgfapi) and ZFS

[Stefan Ring]

On Fri, Feb 28, 2020 at 12:10 PM Kevin Wolf  wrote:
>
> This sounds almost like two other bugs we got fixed recently (in the
> QEMU file-posix driver and in the XFS kernel driver) where two write
> extending the file size were in flight in parallel, but if the shorter
> one completed last, instead extending the file, it would end up
> truncating it.

Hmm, XFS is used on the gluster server, with a CentOS 7 kernel. I can
switch to ext4 and check if it changes anything.

At this point I actually wanted to take this investigation away from
qemu, because it’s not involved anymore in my reproducer.

[PATCH v2 01/18] vl: Add missing "hw/boards.h" include

[Philippe Mathieu-Daudé]

vl.c calls machine_usb() declared in "hw/boards.h". Include it.

This fixes (when modifying unrelated headers):

  vl.c:1283:10: error: implicit declaration of function 'machine_usb' is 
invalid in C99 [-Werror,-Wimplicit-function-declaration]
  if (!machine_usb(current_machine)) {
   ^
  vl.c:1283:10: error: this function declaration is not a prototype 
[-Werror,-Wstrict-prototypes]
  vl.c:1283:22: error: use of undeclared identifier 'current_machine'
  if (!machine_usb(current_machine)) {
   ^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 softmmu/vl.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 16ff5a16a3..20b7201451 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -25,6 +25,7 @@
 #include "qemu/osdep.h"
 #include "qemu-common.h"
 #include "qemu/units.h"
+#include "hw/boards.h"
 #include "hw/qdev-properties.h"
 #include "qapi/error.h"
 #include "qemu-version.h"
-- 
2.21.1

[PATCH v2 02/18] hw/southbridge/ich9: Removed unused headers

[Philippe Mathieu-Daudé]

The ICH9 chipset is not X86/PC specific.

These files don't use anything declared by the "hw/i386/pc.h"
or "hw/i386/ioapic.h" headers. Remove them.

Reviewed-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/i386/ich9.h | 1 -
 hw/acpi/ich9.c | 1 -
 hw/isa/lpc_ich9.c  | 1 -
 3 files changed, 3 deletions(-)

diff --git a/include/hw/i386/ich9.h b/include/hw/i386/ich9.h
index 72e803f6e2..a98d10b252 100644
--- a/include/hw/i386/ich9.h
+++ b/include/hw/i386/ich9.h
@@ -5,7 +5,6 @@
 #include "hw/sysbus.h"
 #include "hw/i386/pc.h"
 #include "hw/isa/apm.h"
-#include "hw/i386/ioapic.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pcie_host.h"
 #include "hw/pci/pci_bridge.h"
diff --git a/hw/acpi/ich9.c b/hw/acpi/ich9.c
index 2034dd749e..fdd0a6c79e 100644
--- a/hw/acpi/ich9.c
+++ b/hw/acpi/ich9.c
@@ -27,7 +27,6 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "qapi/visitor.h"
-#include "hw/i386/pc.h"
 #include "hw/pci/pci.h"
 #include "migration/vmstate.h"
 #include "qemu/timer.h"
diff --git a/hw/isa/lpc_ich9.c b/hw/isa/lpc_ich9.c
index f85b484eac..cb79616ced 100644
--- a/hw/isa/lpc_ich9.c
+++ b/hw/isa/lpc_ich9.c
@@ -37,7 +37,6 @@
 #include "migration/vmstate.h"
 #include "hw/irq.h"
 #include "hw/isa/apm.h"
-#include "hw/i386/ioapic.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_bridge.h"
 #include "hw/i386/ich9.h"
-- 
2.21.1

[PATCH v2 00/18] hw: Clean up hw/i386 headers (and few alpha/hppa)

[Philippe Mathieu-Daudé]

[Rebased since v1]:
https://www.mail-archive.com/qemu-block@nongnu.org/msg57485.html

This is a follow-up of Markus's cleanup series:
Tame a few "touch this, recompile the world"
https://www.mail-archive.com/qemu-devel@nongnu.org/msg635748.html

This part is mostly restricted to X86, but since some file from the
Alpha/PA-RISC machines include "hw/i386/pc.h" I had to fix them
too.

Eventually I'll succeed at removing hw/i386/ dependency on non-X86
platforms (Quest I started 2 years ago...).

Regards,

Phil.

Philippe Mathieu-Daudé (18):
  vl: Add missing "hw/boards.h" include
  hw/southbridge/ich9: Removed unused headers
  hw/i386/ioapic_internal: Remove unused "hw/i386/ioapic.h" header
  hw/timer: Remove unused "ui/console.h" header
  hw/usb/dev-storage: Remove unused "ui/console.h" header
  hw/i386/intel_iommu: Remove unused includes
  hw/alpha/alpha_sys: Remove unused "hw/ide.h" header
  hw/alpha/dp264: Include "net/net.h"
  hw/hppa/machine: Include "net/net.h"
  hw/acpi/cpu_hotplug: Include "hw/pci/pci.h"
  hw/timer/hpet: Include "exec/address-spaces.h"
  hw/pci-host/q35: Include "qemu/range.h"
  hw/i2c/smbus_ich9: Include "qemu/range.h"
  hw/pci-host/piix: Include "qemu/range.h"
  hw/acpi: Include "hw/mem/nvdimm.h"
  hw/i386: Include "hw/mem/nvdimm.h"
  hw/pci-host/q35: Remove unused includes
  hw/i386/pc: Clean up includes

 hw/alpha/alpha_sys.h  |  1 -
 include/hw/i386/ich9.h|  1 -
 include/hw/i386/intel_iommu.h |  4 
 include/hw/i386/ioapic_internal.h |  1 -
 include/hw/i386/pc.h  | 11 +++
 include/hw/pci-host/q35.h |  8 +---
 hw/acpi/cpu_hotplug.c |  1 +
 hw/acpi/ich9.c|  2 +-
 hw/acpi/piix4.c   |  1 +
 hw/alpha/dp264.c  |  1 +
 hw/hppa/machine.c |  1 +
 hw/i2c/smbus_ich9.c   |  1 +
 hw/i386/acpi-build.c  |  1 +
 hw/i386/pc.c  |  1 +
 hw/i386/pc_piix.c |  1 +
 hw/i386/pc_q35.c  |  1 +
 hw/isa/lpc_ich9.c |  1 -
 hw/pci-host/i440fx.c  |  1 +
 hw/pci-host/q35.c |  1 +
 hw/rtc/twl92230.c |  1 -
 hw/timer/hpet.c   |  2 +-
 hw/usb/dev-storage.c  |  1 -
 softmmu/vl.c  |  1 +
 23 files changed, 18 insertions(+), 27 deletions(-)

-- 
2.21.1

Re: [edk2-devel] A problem with live migration of UEFI virtual machines

[Laszlo Ersek]

On 02/28/20 05:04, Andrew Fish wrote:

> Maybe I was overcomplicating this. Given your explanation I think the part 
> I'm missing is OVMF is implying FLASH layout, in this split model, based on 
> the size of the OVMF_CODE.fd and OVMF_VARS.fd.  Given that if OVMF_CODE.fd 
> gets bigger the variable address changes from a QEMU point of view. So 
> basically it is the QEMU  API that is making assumptions about the relative 
> layout of the FD in the split model that makes a migration to larger ROM not 
> work.

No, QEMU does not make any assumptions here. QEMU simply grabs both
pflash chips (the order is not random, it can be specified on the
command line -- in fact the QEMU user is expected to specify in the
right order), and then QEMU maps them in decreasing address order from
4GB in guest-phys address space.

If we enlarge OVMF_CODE.fd, then the base address of the varstore
(PcdOvmfFlashNvStorageVariableBase) will sink. That's not a problem per
se, because QEMU doesn't know about PcdOvmfFlashNvStorageVariableBase at
all. QEMU will simply map the varstore, automatically, where the
enlarged OVMF_CODE.fd will look for it.

> Basically the -pflash API does not support changing the size of the ROM 
> without moving NVRAM given the way it is currently defined.

Let me put it like this: the NVRAM gets moved by virtue of how OVMF is
built, and by how QEMU maps the pflash chips into guest-phys address
space. They are in sync, automatically.

The problem is when the NVRAM is internally restructured, or resized --
the new OVMF_CODE.fd binary will reflect this with changed PCDs, and
look for "stuff" at those addresses. But if you still try to use an old
(differently sized, or differently structured) varstore file, while QEMU
will happily map it, parts of the NVRAM will just not end up in places
where OVMF_CODE.fd expects them.

> 
> Given the above it seems like the 2 options are:
> 1) Pad OVMF_CODE.fd to be very large so there is room to grow.

There's already room to grow, *inside* OVMF_CODE.fd. As I've shown
elsewhere in this thread, even the 2MB build has approx. 457 KB free in
the DXEFV volume, even without link-time optimization and without
DEBUG/ASSERT stripping, if you don't enable additional features.

> 2) Add some feature to QUEM that allows the variable store address to not be 
> based on OVMF_CODE.fd size. 

Yes, this has been proposed over time.

It wouldn't help with the case when you change the internal structure of
the NVRAM, and try to run an incompatible OVMF_CODE.fd against that.

> I did see this [1] and combined with your email I either understand, or I'm 
> still confused? :)
> 
> I'm not saying we need to change anything, I'm just trying to make sure I 
> understand how OVMF and QEMU are tied to together. 

I think the most interesting function for you could be
pc_system_flash_map(), in "hw/i386/pc_sysfw.c", in the QEMU source.

> 
> [1] https://www.redhat.com/archives/libvir-list/2019-January/msg01031.html


Thanks
Laszlo

[PATCH v2 03/18] hw/i386/ioapic_internal: Remove unused "hw/i386/ioapic.h" header

[Philippe Mathieu-Daudé]

The "ioapic_internal.h" does not use anything from
"hw/i386/ioapic.h", remove it.

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/i386/ioapic_internal.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/include/hw/i386/ioapic_internal.h 
b/include/hw/i386/ioapic_internal.h
index d46c87c510..fe06938bda 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -23,7 +23,6 @@
 #define QEMU_IOAPIC_INTERNAL_H
 
 #include "exec/memory.h"
-#include "hw/i386/ioapic.h"
 #include "hw/sysbus.h"
 #include "qemu/notify.h"
 
-- 
2.21.1

[PATCH v2 04/18] hw/timer: Remove unused "ui/console.h" header

[Philippe Mathieu-Daudé]

The timer models don't need anything from "ui/console.h".
Remove it.

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/rtc/twl92230.c | 1 -
 hw/timer/hpet.c   | 1 -
 2 files changed, 2 deletions(-)

diff --git a/hw/rtc/twl92230.c b/hw/rtc/twl92230.c
index 63bd13d2ca..d0011be89e 100644
--- a/hw/rtc/twl92230.c
+++ b/hw/rtc/twl92230.c
@@ -27,7 +27,6 @@
 #include "migration/qemu-file-types.h"
 #include "migration/vmstate.h"
 #include "sysemu/sysemu.h"
-#include "ui/console.h"
 #include "qemu/bcd.h"
 #include "qemu/module.h"
 
diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c
index 8dbcbdca16..4f30dd50a4 100644
--- a/hw/timer/hpet.c
+++ b/hw/timer/hpet.c
@@ -27,7 +27,6 @@
 #include "qemu/osdep.h"
 #include "hw/i386/pc.h"
 #include "hw/irq.h"
-#include "ui/console.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "qemu/timer.h"
-- 
2.21.1

[PATCH v2 05/18] hw/usb/dev-storage: Remove unused "ui/console.h" header

[Philippe Mathieu-Daudé]

The USB models related to storage don't need anything from
"ui/console.h". Remove it.

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/usb/dev-storage.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c
index 90da008df1..4883c1d89e 100644
--- a/hw/usb/dev-storage.c
+++ b/hw/usb/dev-storage.c
@@ -17,7 +17,6 @@
 #include "desc.h"
 #include "hw/qdev-properties.h"
 #include "hw/scsi/scsi.h"
-#include "ui/console.h"
 #include "migration/vmstate.h"
 #include "monitor/monitor.h"
 #include "sysemu/sysemu.h"
-- 
2.21.1

[PATCH v2 06/18] hw/i386/intel_iommu: Remove unused includes

[Philippe Mathieu-Daudé]

intel_iommu.h does not use any of these includes, remove them.

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/i386/intel_iommu.h | 4 
 1 file changed, 4 deletions(-)

diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 66b931e526..a1c4afcda5 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -22,11 +22,7 @@
 #ifndef INTEL_IOMMU_H
 #define INTEL_IOMMU_H
 
-#include "sysemu/dma.h"
 #include "hw/i386/x86-iommu.h"
-#include "hw/i386/ioapic.h"
-#include "hw/pci/msi.h"
-#include "hw/sysbus.h"
 #include "qemu/iova-tree.h"
 
 #define TYPE_INTEL_IOMMU_DEVICE "intel-iommu"
-- 
2.21.1

[PATCH v2 07/18] hw/alpha/alpha_sys: Remove unused "hw/ide.h" header

[Philippe Mathieu-Daudé]

alpha_sys.h does not use anything from the "hw/ide.h" header.
Remove it.

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/alpha/alpha_sys.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/hw/alpha/alpha_sys.h b/hw/alpha/alpha_sys.h
index bc0a286226..e2c02e2bbe 100644
--- a/hw/alpha/alpha_sys.h
+++ b/hw/alpha/alpha_sys.h
@@ -6,7 +6,6 @@
 #include "target/alpha/cpu-qom.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_host.h"
-#include "hw/ide.h"
 #include "hw/boards.h"
 #include "hw/intc/i8259.h"
 
-- 
2.21.1

[PATCH v2 09/18] hw/hppa/machine: Include "net/net.h"

[Philippe Mathieu-Daudé]

hw/hppa/machine.c uses NICInfo variables which are declared in
"net/net.h". Include it.

This fixes (when modifying unrelated headers):

  hw/hppa/machine.c:126:21: error: use of undeclared identifier 'nb_nics'
  for (i = 0; i < nb_nics; i++) {
  ^
  hw/hppa/machine.c:127:30: error: use of undeclared identifier 'nd_table'
  pci_nic_init_nofail(&nd_table[i], pci_bus, "e1000", NULL);
   ^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/hppa/machine.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 67181e75ba..d1e59a97f6 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -22,6 +22,7 @@
 #include "qapi/error.h"
 #include "net/net.h"
 #include "qemu/log.h"
+#include "net/net.h"
 
 #define MAX_IDE_BUS 2
 
-- 
2.21.1

[PATCH v2 13/18] hw/i2c/smbus_ich9: Include "qemu/range.h"

[Philippe Mathieu-Daudé]

hw/i2c/smbus_ich9.c calls range_covers_byte(). Include "qemu/range.h"
which declares it.

This fixes (when modifying unrelated headers):

  hw/i2c/smbus_ich9.c:66:9: error: implicit declaration of function 
'range_covers_byte' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  if (range_covers_byte(address, len, ICH9_SMB_HOSTC)) {
  ^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i2c/smbus_ich9.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/i2c/smbus_ich9.c b/hw/i2c/smbus_ich9.c
index fd50fb851a..48f1ff4191 100644
--- a/hw/i2c/smbus_ich9.c
+++ b/hw/i2c/smbus_ich9.c
@@ -21,6 +21,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/range.h"
 #include "hw/i2c/pm_smbus.h"
 #include "hw/pci/pci.h"
 #include "migration/vmstate.h"
-- 
2.21.1

[PATCH v2 10/18] hw/acpi/cpu_hotplug: Include "hw/pci/pci.h"

[Philippe Mathieu-Daudé]

hw/acpi/cpu_hotplug.c calls pci_address_space_io(). Include
"hw/pci/pci.h" which declares it.

This fixes (when modifying unrelated headers):

  hw/acpi/cpu_hotplug.c:103:28: error: implicit declaration of function 
'pci_address_space_io' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
  MemoryRegion *parent = pci_address_space_io(PCI_DEVICE(gpe_cpu->device));
 ^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/acpi/cpu_hotplug.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/acpi/cpu_hotplug.c b/hw/acpi/cpu_hotplug.c
index 9c3bcc84de..3e687d227a 100644
--- a/hw/acpi/cpu_hotplug.c
+++ b/hw/acpi/cpu_hotplug.c
@@ -14,6 +14,7 @@
 #include "qapi/error.h"
 #include "hw/core/cpu.h"
 #include "hw/i386/pc.h"
+#include "hw/pci/pci.h"
 #include "qemu/error-report.h"
 
 #define CPU_EJECT_METHOD "CPEJ"
-- 
2.21.1

[PATCH v2 08/18] hw/alpha/dp264: Include "net/net.h"

[Philippe Mathieu-Daudé]

hw/alpha/dp264.c uses NICInfo variables which are declared in
"net/net.h". Include it.

This fixes (when modifying unrelated headers):

  hw/alpha/dp264.c:89:21: error: use of undeclared identifier 'nb_nics'
  for (i = 0; i < nb_nics; i++) {
  ^
  hw/alpha/dp264.c:90:30: error: use of undeclared identifier 'nd_table'
  pci_nic_init_nofail(&nd_table[i], pci_bus, "e1000", NULL);
   ^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/alpha/dp264.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/alpha/dp264.c b/hw/alpha/dp264.c
index 8d71a30617..e264c783c1 100644
--- a/hw/alpha/dp264.c
+++ b/hw/alpha/dp264.c
@@ -21,6 +21,7 @@
 #include "hw/dma/i8257.h"
 #include "net/net.h"
 #include "qemu/cutils.h"
+#include "net/net.h"
 
 #define MAX_IDE_BUS 2
 
-- 
2.21.1

[PATCH v2 11/18] hw/timer/hpet: Include "exec/address-spaces.h"

[Philippe Mathieu-Daudé]

hw/timer/hpet.c calls address_space_stl_le() declared in
"exec/address-spaces.h". Include it.

This fixes (when modifying unrelated headers):

  hw/timer/hpet.c:210:31: error: use of undeclared identifier 
'address_space_memory'
  address_space_stl_le(&address_space_memory, timer->fsb >> 32,
   ^~~~

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/timer/hpet.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c
index 4f30dd50a4..380acfa7c8 100644
--- a/hw/timer/hpet.c
+++ b/hw/timer/hpet.c
@@ -36,6 +36,7 @@
 #include "hw/rtc/mc146818rtc_regs.h"
 #include "migration/vmstate.h"
 #include "hw/timer/i8254.h"
+#include "exec/address-spaces.h"
 
 //#define HPET_DEBUG
 #ifdef HPET_DEBUG
-- 
2.21.1

Re: [PATCH v1 4/8] iotests: filter out compression_type

[Denis Plotnikov]

On 27.02.2020 17:03, Eric Blake wrote:

On 2/27/20 1:29 AM, Denis Plotnikov wrote:

After adding compression type feature to qcow2 format, qemu framework
commands reporting the image settingd, e.g. "qemu-img create", started


settings


reporting the compression type for the image which breaks the iotests
output matching.

To fix it, add compression_type=zlib to the list of filtered image 
parameters.


Signed-off-by: Denis Plotnikov 
---
  tests/qemu-iotests/common.filter | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)


This should be squashed in to the patch that caused the breakage (3/8, 
if I'm right).




diff --git a/tests/qemu-iotests/common.filter 
b/tests/qemu-iotests/common.filter

index 3f8ee3e5f7..c6962d199c 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -152,7 +152,8 @@ _filter_img_create()
  -e "s# refcount_bits=[0-9]\\+##g" \
  -e "s# key-secret=[a-zA-Z0-9]\\+##g" \
  -e "s# iter-time=[0-9]\\+##g" \
-    -e "s# force_size=\\(on\\|off\\)##g"
+    -e "s# force_size=\\(on\\|off\\)##g" \
+    -e "s# compression_type=zlib##g"


Do you really want to hard-code just zlib, or should this be more 
generic as compression_type=[a-zA-Z0-9]\\+ as done on other lines like 
key-secret?
When I did this I meant additional implicit check that the default 
compression type is zlib. But non of the other items in the filter don't 
do it. So I'll change it to be consistent. Thanks!


Denis

[PATCH v2 17/18] hw/pci-host/q35: Remove unused includes

[Philippe Mathieu-Daudé]

Only q35.c requires declarations from "hw/i386/pc.h", move it there.
Remove all the includes not used by "q35.h".

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/pci-host/q35.h | 7 ---
 hw/pci-host/q35.c | 1 +
 2 files changed, 1 insertion(+), 7 deletions(-)

diff --git a/include/hw/pci-host/q35.h b/include/hw/pci-host/q35.h
index 47086c645e..070305f83d 100644
--- a/include/hw/pci-host/q35.h
+++ b/include/hw/pci-host/q35.h
@@ -22,16 +22,9 @@
 #ifndef HW_Q35_H
 #define HW_Q35_H
 
-#include "hw/isa/isa.h"
-#include "hw/sysbus.h"
-#include "hw/i386/pc.h"
-#include "hw/isa/apm.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pcie_host.h"
-#include "hw/acpi/acpi.h"
-#include "hw/acpi/ich9.h"
 #include "hw/pci-host/pam.h"
-#include "hw/i386/intel_iommu.h"
 #include "qemu/units.h"
 #include "qemu/range.h"
 
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index a9b9ccc876..993f467668 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -29,6 +29,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "hw/i386/pc.h"
 #include "hw/pci-host/q35.h"
 #include "hw/qdev-properties.h"
 #include "migration/vmstate.h"
-- 
2.21.1

[PATCH v2 12/18] hw/pci-host/q35: Include "qemu/range.h"

[Philippe Mathieu-Daudé]

The MCHPCIState structure uses the Range type which is declared in
"qemu/range.h". Include it.

This fixes (when modifying unrelated headers):

  In file included from hw/pci-host/q35.c:32:
  include/hw/pci-host/q35.h:57:11: error: field has incomplete type 'Range' 
(aka 'struct Range')
  Range pci_hole;
^
  include/qemu/typedefs.h:116:16: note: forward declaration of 'struct Range'
  typedef struct Range Range;
 ^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/pci-host/q35.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/hw/pci-host/q35.h b/include/hw/pci-host/q35.h
index 976fbae599..47086c645e 100644
--- a/include/hw/pci-host/q35.h
+++ b/include/hw/pci-host/q35.h
@@ -33,6 +33,7 @@
 #include "hw/pci-host/pam.h"
 #include "hw/i386/intel_iommu.h"
 #include "qemu/units.h"
+#include "qemu/range.h"
 
 #define TYPE_Q35_HOST_DEVICE "q35-pcihost"
 #define Q35_HOST_DEVICE(obj) \
-- 
2.21.1

[PATCH v2 18/18] hw/i386/pc: Clean up includes

[Philippe Mathieu-Daudé]

Various headers are not required by hw/i386/pc.h:

 - "qemu/range.h"
 - "qemu/bitmap.h"
 - "qemu/module.h"
 - "exec/memory.h"
 - "hw/pci/pci.h"
 - "hw/mem/pc-dimm.h"
 - "hw/mem/nvdimm.h"
 - "net/net.h"

Remove them.

Add 3 headers that were missing:

 - "hw/hotplug.h"

   PCMachineState::acpi_dev is of type HotplugHandler

 - "qemu/notify.h"

   PCMachineState::machine_done is of type Notifier

 - "qapi/qapi-types-common.h"

   PCMachineState::vmport/smm is of type OnOffAuto

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/i386/pc.h | 11 +++
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index d5ac76d54e..6ab6eda046 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -1,20 +1,15 @@
 #ifndef HW_PC_H
 #define HW_PC_H
 
-#include "exec/memory.h"
+#include "qemu/notify.h"
+#include "qapi/qapi-types-common.h"
 #include "hw/boards.h"
 #include "hw/block/fdc.h"
 #include "hw/block/flash.h"
-#include "net/net.h"
 #include "hw/i386/x86.h"
 
-#include "qemu/range.h"
-#include "qemu/bitmap.h"
-#include "qemu/module.h"
-#include "hw/pci/pci.h"
-#include "hw/mem/pc-dimm.h"
-#include "hw/mem/nvdimm.h"
 #include "hw/acpi/acpi_dev_interface.h"
+#include "hw/hotplug.h"
 
 #define HPET_INTCAP "hpet-intcap"
 
-- 
2.21.1

[PATCH v2 14/18] hw/pci-host/piix: Include "qemu/range.h"

[Philippe Mathieu-Daudé]

hw/pci-host/piix.c calls various functions from the Range API.
Include "qemu/range.h" which declares them.

This fixes (when modifying unrelated headers):

  hw/pci-host/i440fx.c:54:11: error: field has incomplete type 'Range' (aka 
'struct Range')
  Range pci_hole;
   ^
  include/qemu/typedefs.h:116:16: note: forward declaration of 'struct Range'
  typedef struct Range Range;
 ^
  hw/pci-host/i440fx.c:126:9: error: implicit declaration of function 
'ranges_overlap' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  if (ranges_overlap(address, len, I440FX_PAM, I440FX_PAM_SIZE) ||
  ^
  hw/pci-host/i440fx.c:126:9: error: this function declaration is not a 
prototype [-Werror,-Wstrict-prototypes]
  hw/pci-host/i440fx.c:127:9: error: implicit declaration of function 
'range_covers_byte' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  range_covers_byte(address, len, I440FX_SMRAM)) {
  ^
  hw/pci-host/i440fx.c:127:9: error: this function declaration is not a 
prototype [-Werror,-Wstrict-prototypes]
  hw/pci-host/i440fx.c:189:13: error: implicit declaration of function 
'range_is_empty' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  val64 = range_is_empty(&s->pci_hole) ? 0 : range_lob(&s->pci_hole);
  ^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/pci-host/i440fx.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/pci-host/i440fx.c b/hw/pci-host/i440fx.c
index 11050a0f8b..d980c97049 100644
--- a/hw/pci-host/i440fx.c
+++ b/hw/pci-host/i440fx.c
@@ -23,6 +23,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/range.h"
 #include "hw/i386/pc.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_host.h"
-- 
2.21.1

[PATCH v2 15/18] hw/acpi: Include "hw/mem/nvdimm.h"

[Philippe Mathieu-Daudé]

Both ich9.c and piix4.c use methods/definitions declared in the
NVDIMM device header. Include it.

This fixes (when modifying unrelated headers):

  hw/acpi/ich9.c:507:46: error: use of undeclared identifier 'TYPE_NVDIMM'
if (object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM)) {
 ^
  hw/acpi/ich9.c:508:13: error: implicit declaration of function 
'nvdimm_acpi_plug_cb' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
nvdimm_acpi_plug_cb(hotplug_dev, dev);
^
  hw/acpi/piix4.c:403:46: error: use of undeclared identifier 'TYPE_NVDIMM'
if (object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM)) {
 ^
  hw/acpi/piix4.c:404:13: error: implicit declaration of function 
'nvdimm_acpi_plug_cb' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
nvdimm_acpi_plug_cb(hotplug_dev, dev);
^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/acpi/ich9.c  | 1 +
 hw/acpi/piix4.c | 1 +
 2 files changed, 2 insertions(+)

diff --git a/hw/acpi/ich9.c b/hw/acpi/ich9.c
index fdd0a6c79e..4e74284b65 100644
--- a/hw/acpi/ich9.c
+++ b/hw/acpi/ich9.c
@@ -39,6 +39,7 @@
 
 #include "hw/i386/ich9.h"
 #include "hw/mem/pc-dimm.h"
+#include "hw/mem/nvdimm.h"
 
 //#define DEBUG
 
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 6d621c31e7..b84dbba2c3 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -38,6 +38,7 @@
 #include "hw/acpi/cpu.h"
 #include "hw/hotplug.h"
 #include "hw/mem/pc-dimm.h"
+#include "hw/mem/nvdimm.h"
 #include "hw/acpi/memory_hotplug.h"
 #include "hw/acpi/acpi_dev_interface.h"
 #include "hw/xen/xen.h"
-- 
2.21.1

[PATCH v2 16/18] hw/i386: Include "hw/mem/nvdimm.h"

[Philippe Mathieu-Daudé]

All this files use methods/definitions declared in the NVDIMM
device header. Include it.

This fixes (when modifying unrelated headers):

  hw/i386/acpi-build.c:2733:9: error: implicit declaration of function 
'nvdimm_build_acpi' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
nvdimm_build_acpi(table_offsets, tables_blob, tables->linker,
^
  hw/i386/pc.c:1996:61: error: use of undeclared identifier 'TYPE_NVDIMM'
const bool is_nvdimm = object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM);
^
  hw/i386/pc.c:2032:55: error: use of undeclared identifier 'TYPE_NVDIMM'
bool is_nvdimm = object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM);
  ^
  hw/i386/pc.c:2040:9: error: implicit declaration of function 'nvdimm_plug' is 
invalid in C99 [-Werror,-Wimplicit-function-declaration]
nvdimm_plug(ms->nvdimms_state);
^
  hw/i386/pc.c:2040:9: error: this function declaration is not a prototype 
[-Werror,-Wstrict-prototypes]
nvdimm_plug(ms->nvdimms_state);
^
  hw/i386/pc.c:2065:42: error: use of undeclared identifier 'TYPE_NVDIMM'
if (object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM)) {
 ^
  hw/i386/pc_i440fx.c:307:9: error: implicit declaration of function 
'nvdimm_init_acpi_state' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
nvdimm_init_acpi_state(machine->nvdimms_state, system_io,
^
  hw/i386/pc_q35.c:332:9: error: implicit declaration of function 
'nvdimm_init_acpi_state' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
nvdimm_init_acpi_state(machine->nvdimms_state, system_io,
^

Acked-by: John Snow 
Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/acpi-build.c | 1 +
 hw/i386/pc.c | 1 +
 hw/i386/pc_piix.c| 1 +
 hw/i386/pc_q35.c | 1 +
 4 files changed, 4 insertions(+)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 9c4e46fa74..9335a82e9f 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -47,6 +47,7 @@
 #include "hw/rtc/mc146818rtc_regs.h"
 #include "migration/vmstate.h"
 #include "hw/mem/memory-device.h"
+#include "hw/mem/nvdimm.h"
 #include "sysemu/numa.h"
 #include "sysemu/reset.h"
 
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 6ab4acb0c6..362eb2a180 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -76,6 +76,7 @@
 #include "hw/boards.h"
 #include "acpi-build.h"
 #include "hw/mem/pc-dimm.h"
+#include "hw/mem/nvdimm.h"
 #include "qapi/error.h"
 #include "qapi/qapi-visit-common.h"
 #include "qapi/visitor.h"
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index fa12203079..82f1693901 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -60,6 +60,7 @@
 #include "migration/global_state.h"
 #include "migration/misc.h"
 #include "sysemu/numa.h"
+#include "hw/mem/nvdimm.h"
 
 #define MAX_IDE_BUS 2
 
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 84cf925cf4..d37c425e22 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -53,6 +53,7 @@
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "sysemu/numa.h"
+#include "hw/mem/nvdimm.h"
 
 /* ICH9 AHCI has 6 ports */
 #define MAX_SATA_PORTS 6
-- 
2.21.1

Re: [edk2-devel] A problem with live migration of UEFI virtual machines

[Laszlo Ersek]

On 02/28/20 12:47, Laszlo Ersek wrote:
> On 02/28/20 05:04, Andrew Fish wrote:

>> Given the above it seems like the 2 options are:
>> 1) Pad OVMF_CODE.fd to be very large so there is room to grow.
> 
> There's already room to grow, *inside* OVMF_CODE.fd. As I've shown
> elsewhere in this thread, even the 2MB build has approx. 457 KB free in
> the DXEFV volume, even without link-time optimization and without
> DEBUG/ASSERT stripping, if you don't enable additional features.

Typo; I meant FVMAIN_COMPACT, not DXEFV.

Laszlo

Re: [PATCH v1 3/8] qcow2: add zstd cluster compression

[Denis Plotnikov]

On 27.02.2020 17:18, Vladimir Sementsov-Ogievskiy wrote:

27.02.2020 17:11, Denis Plotnikov wrote:



On 27.02.2020 12:55, Vladimir Sementsov-Ogievskiy wrote:

27.02.2020 10:29, Denis Plotnikov wrote:

zstd significantly reduces cluster compression time.
It provides better compression performance maintaining
the same level of the compression ratio in comparison with
zlib, which, at the moment, is the only compression
method available.

The performance test results:
Test compresses and decompresses qemu qcow2 image with just
installed rhel-7.6 guest.
Image cluster size: 64K. Image on disk size: 2.2G

The test was conducted with brd disk to reduce the influence
of disk subsystem to the test results.
The results is given in seconds.

compress cmd:
   time ./qemu-img convert -O qcow2 -c -o compression_type=[zlib|zstd]
   src.img [zlib|zstd]_compressed.img
decompress cmd
   time ./qemu-img convert -O qcow2
   [zlib|zstd]_compressed.img uncompressed.img

    compression   decompression
  zlib   zstd   zlib zstd

real 65.5   16.3 (-75 %)    1.9  1.6 (-16 %)
user 65.0   15.8    5.3  2.5
sys   3.3    0.2    2.0  2.0

Both ZLIB and ZSTD gave the same compression ratio: 1.57
compressed image size in both cases: 1.4G

Signed-off-by: Denis Plotnikov 
---
  block/qcow2-threads.c  | 122 
+

  block/qcow2.c  |   7 +++
  configure  |  29 ++
  docs/interop/qcow2.txt |  18 ++
  qapi/block-core.json   |   3 +-
  5 files changed, 178 insertions(+), 1 deletion(-)

diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
index 1c128e9840..e942c4d7e5 100644
--- a/block/qcow2-threads.c
+++ b/block/qcow2-threads.c
@@ -28,6 +28,11 @@
  #define ZLIB_CONST
  #include 
  +#ifdef CONFIG_ZSTD
+#include 
+#include 
+#endif
+
  #include "qcow2.h"
  #include "block/thread-pool.h"
  #include "crypto.h"
@@ -164,6 +169,113 @@ static ssize_t qcow2_zlib_decompress(void 
*dest, size_t dest_size,

  return ret;
  }
  +#ifdef CONFIG_ZSTD
+
+#define ZSTD_LEN_BUF 4
+
+/*
+ * qcow2_zstd_compress()
+ *
+ * Compress @src_size bytes of data using zstd compression method
+ *
+ * @dest - destination buffer, @dest_size bytes
+ * @src - source buffer, @src_size bytes
+ *
+ * Returns: compressed size on success


This doesn't match qcow2_co_compress definition. You should return 0 
on success.
does it? I'd rather say it doesn't match to qcow2_co_compress 
description in the function comment, which we can change actually,

because qcow2_co_compress is used like:


Oh, yes, you are right. Then we should change the comment.



block/qcow2.c:

static coroutine_fn int
qcow2_co_pwritev_compressed_task(BlockDriverState *bs,
  uint64_t offset, uint64_t bytes,
  QEMUIOVector *qiov, size_t 
qiov_offset)

{
 ...
 out_buf = g_malloc(s->cluster_size);

 out_len = qcow2_co_compress(bs, out_buf, s->cluster_size - 1,
 buf, s->cluster_size);
 if (out_len == -ENOMEM) {
 /* could not compress: write normal cluster */
 ret = qcow2_co_pwritev_part(bs, offset, bytes, qiov, 
qiov_offset, 0);

 if (ret < 0) {
 goto fail;
 }
 goto success;
 } else if (out_len < 0) {
 ret = -EINVAL;
 goto fail;
 }

 qemu_co_mutex_lock(&s->lock);
 ret = qcow2_alloc_compressed_cluster_offset(bs, offset, out_len, 


&cluster_offset);
 ...
}



+ *  -ENOMEM destination buffer is not enough to store 
compressed data

+ *  -EIO    on any other error
+ */
+
+static ssize_t qcow2_zstd_compress(void *dest, size_t dest_size,
+   const void *src, size_t src_size)
+{
+    size_t ret;
+
+    /*
+ * steal ZSTD_LEN_BUF bytes in the very beginng of the buffer


beginning


+ * to store compressed chunk size
+ */
+    char *d_buf = ((char *) dest) + ZSTD_LEN_BUF;
+
+    /*
+ * sanity check that we can store the compressed data length,
+ * and there is some space left for the compressor buffer
+ */
+    if (dest_size <= ZSTD_LEN_BUF) {
+    return -ENOMEM;
+    }
+
+    dest_size -= ZSTD_LEN_BUF;
+
+    ret = ZSTD_compress(d_buf, dest_size, src, src_size, 5);
+
+    if (ZSTD_isError(ret)) {
+    if (ZSTD_getErrorCode(ret) == ZSTD_error_dstSize_tooSmall) {
+    return -ENOMEM;
+    } else {
+    return -EIO;
+    }
+    }
+
+    /* paraniod sanity check that we can store the commpressed 
size */

+    if (ret > UINT_MAX) {
+    return -ENOMEM;
+    }


I'd use UINT32_MAX, possibly even more paranoid)

ok




+
+    /* store the compressed chunk size in the very beginning of 
the buffer */

+    stl_be_p(dest, ret);
+
+    ret

[PATCH] vfio: avoid SET_ACTION_TRIGGER ioctls

[Roman Kapl]

For MSI-X interrupts, remember what the last used eventfd was (KVM
bypass vs QEMU) and only call vfio_set_irq_signaling if it has changed.

This not only helps with performance, but it seems that interrupts can
be lost during VFIO_IRQ_SET_ACTION_TRIGGER. With the 'x-no-kvm-msix'
switch and this patch, SET_ACTION_TRIGGER is not called during
mask/unmask. This really only affects guests that actively use MSI-X masking.

Signed-off-by: Roman Kapl 
---

This patch scratches my particular itch. I am able to get our guest (which masks
MSI on each interrupt) running, without getting randomly stuck on waiting for
interrupt. However, the solution is far from perfect (x-no-kvm-msix is required)
and pretty slow. I would be interested in hearing any ideas how to improve this.
Some ideas:

1) Fix the kernel so that SET_ACTION_TRIGGER does not loose interrupts (I think
the problem is there, but not 100% sure). I've tested on 5.3.0-40-generic
#32~18.04.1-Ubuntu SMP.

2) Add support for MASK/UNMASK for MSI-X in kernel and use that. But I don't
know how to do PBA in that case. Another IOCTL? We could look at the real PBA
array, if mapping is supported, but that seems hacky.

3) Twiddle the bits behing kernel's back, if it can be mapped?

Still, I think this patch does not hurt anything and could be applied if no-one
can think of a better way.

---

 hw/vfio/pci.c | 32 ++--
 hw/vfio/pci.h |  2 ++
 2 files changed, 24 insertions(+), 10 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index e6569a7968..5f7ce91519 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -390,12 +390,16 @@ static int vfio_enable_vectors(VFIOPCIDevice *vdev, bool 
msix)
  * MSI-X mask and pending bits are emulated, so we want to use the
  * KVM signaling path only when configured and unmasked.
  */
-if (vdev->msi_vectors[i].use) {
-if (vdev->msi_vectors[i].virq < 0 ||
-(msix && msix_is_masked(&vdev->pdev, i))) {
-fd = event_notifier_get_fd(&vdev->msi_vectors[i].interrupt);
+VFIOMSIVector *vector = &vdev->msi_vectors[i];
+if (vector->use) {
+if (vector->virq < 0 ||
+(msix && msix_is_masked(&vdev->pdev, i)))
+{
+vector->kvm_path_active = false;
+fd = event_notifier_get_fd(&vector->interrupt);
 } else {
-fd = 
event_notifier_get_fd(&vdev->msi_vectors[i].kvm_interrupt);
+vector->kvm_path_active = true;
+fd = event_notifier_get_fd(&vector->kvm_interrupt);
 }
 }
 
@@ -509,17 +513,23 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, 
unsigned int nr,
 } else {
 Error *err = NULL;
 int32_t fd;
+bool kvm_path;
 
 if (vector->virq >= 0) {
 fd = event_notifier_get_fd(&vector->kvm_interrupt);
+kvm_path = true;
 } else {
 fd = event_notifier_get_fd(&vector->interrupt);
+kvm_path = false;
 }
 
-if (vfio_set_irq_signaling(&vdev->vbasedev,
- VFIO_PCI_MSIX_IRQ_INDEX, nr,
- VFIO_IRQ_SET_ACTION_TRIGGER, fd, &err)) {
-error_reportf_err(err, VFIO_MSG_PREFIX, vdev->vbasedev.name);
+if (vector->kvm_path_active != kvm_path) {
+if (vfio_set_irq_signaling(&vdev->vbasedev,
+   VFIO_PCI_MSIX_IRQ_INDEX, nr,
+   VFIO_IRQ_SET_ACTION_TRIGGER, fd, &err)) 
{
+error_reportf_err(err, VFIO_MSG_PREFIX, vdev->vbasedev.name);
+}
+vector->kvm_path_active = kvm_path;
 }
 }
 
@@ -555,13 +565,15 @@ static void vfio_msix_vector_release(PCIDevice *pdev, 
unsigned int nr)
  * core will mask the interrupt and set pending bits, allowing it to
  * be re-asserted on unmask.  Nothing to do if already using QEMU mode.
  */
-if (vector->virq >= 0) {
+if (vector->virq >= 0 && vector->kvm_path_active) {
 int32_t fd = event_notifier_get_fd(&vector->interrupt);
 Error *err = NULL;
 
 if (vfio_set_irq_signaling(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_INDEX, 
nr,
VFIO_IRQ_SET_ACTION_TRIGGER, fd, &err)) {
 error_reportf_err(err, VFIO_MSG_PREFIX, vdev->vbasedev.name);
+} else {
+vector->kvm_path_active = false;
 }
 }
 }
diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h
index b329d50338..b01d2676cf 100644
--- a/hw/vfio/pci.h
+++ b/hw/vfio/pci.h
@@ -91,6 +91,8 @@ typedef struct VFIOMSIVector {
  */
 EventNotifier interrupt;
 EventNotifier kvm_interrupt;
+/* Set when the trigger action is set to the KVM bypass FD */
+bool kvm_path_active;
 struct VFIOPCIDevice *vdev; /* back pointer to device */
 int virq;
 bool use;
-- 
2.22.0

[PATCH] hw/ppc/pnv: Fix typo in comment

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/ppc/pnv_lpc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/ppc/pnv_lpc.c b/hw/ppc/pnv_lpc.c
index f150deca34..b5ffa48dac 100644
--- a/hw/ppc/pnv_lpc.c
+++ b/hw/ppc/pnv_lpc.c
@@ -829,7 +829,7 @@ ISABus *pnv_lpc_isa_create(PnvLpcController *lpc, bool 
use_cpld, Error **errp)
 bool hostboot_mode = !!pnv->fw_load_addr;
 
 /* let isa_bus_new() create its own bridge on SysBus otherwise
- * devices speficied on the command line won't find the bus and
+ * devices specified on the command line won't find the bus and
  * will fail to create.
  */
 isa_bus = isa_bus_new(NULL, &lpc->isa_mem, &lpc->isa_io, &local_err);
-- 
2.21.1

[PATCH] block: Remove trailing newline in format used by error_report API

[Philippe Mathieu-Daudé]

The error_report API doesn't want trailing newline characters.
Remove it, to avoid and error when moving the code around:

  ERROR: Error messages should not contain newlines

Signed-off-by: Philippe Mathieu-Daudé 
---
 block.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block.c b/block.c
index 1bdb9c679d..e466d15914 100644
--- a/block.c
+++ b/block.c
@@ -5994,7 +5994,7 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 /* Couldn't open BS, but we have a size, so it's nonfatal */
 warn_reportf_err(local_err,
 "Could not verify backing image. "
-"This may become an error in future versions.\n");
+"This may become an error in future versions.");
 local_err = NULL;
 } else if (!bs) {
 /* Couldn't open bs, do not have size */
-- 
2.21.1

Re: [PATCH] hw/ppc/pnv: Fix typo in comment

[Cédric Le Goater]

On 2/28/20 1:33 PM, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé 

Reviewed-by: Cédric Le Goater 

Thnaks,

C. 


> ---
>  hw/ppc/pnv_lpc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/pnv_lpc.c b/hw/ppc/pnv_lpc.c
> index f150deca34..b5ffa48dac 100644
> --- a/hw/ppc/pnv_lpc.c
> +++ b/hw/ppc/pnv_lpc.c
> @@ -829,7 +829,7 @@ ISABus *pnv_lpc_isa_create(PnvLpcController *lpc, bool 
> use_cpld, Error **errp)
>  bool hostboot_mode = !!pnv->fw_load_addr;
>  
>  /* let isa_bus_new() create its own bridge on SysBus otherwise
> - * devices speficied on the command line won't find the bus and
> + * devices specified on the command line won't find the bus and
>   * will fail to create.
>   */
>  isa_bus = isa_bus_new(NULL, &lpc->isa_mem, &lpc->isa_io, &local_err);
> 

[PATCH 0/2] block/qcow2: Fix bitmap reopen with 'auto-read-only' file child

[Peter Krempa]

See patch 2/2 for explanation. Also please excuse the lack of tests
caused by my ignorance of not figuring out where to put them.

Peter Krempa (2):
  block: Introduce 'bdrv_reopen_commit_post' step
  block/qcow2: Move bitmap reopen into bdrv_reopen_commit_post

 block.c   | 9 +
 block/qcow2.c | 7 ++-
 include/block/block_int.h | 1 +
 3 files changed, 16 insertions(+), 1 deletion(-)

-- 
2.24.1

[PATCH 1/2] block: Introduce 'bdrv_reopen_commit_post' step

[Peter Krempa]

Add another step in the reopen process where driver can execute code
after permission changes are comitted.

Signed-off-by: Peter Krempa 
---
 block.c   | 9 +
 include/block/block_int.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/block.c b/block.c
index 1bdb9c679d..a82e53d4cc 100644
--- a/block.c
+++ b/block.c
@@ -3694,6 +3694,15 @@ cleanup_perm:
 }
 }
 }
+
+if (ret == 0) {
+QTAILQ_FOREACH_REVERSE(bs_entry, bs_queue, entry) {
+BlockDriverState *bs = bs_entry->state.bs;
+
+if (bs->drv->bdrv_reopen_commit_post)
+bs->drv->bdrv_reopen_commit_post(&bs_entry->state);
+}
+}
 cleanup:
 QTAILQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
 if (ret) {
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 6f9fd5e20e..f422c0bff0 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -122,6 +122,7 @@ struct BlockDriver {
 int (*bdrv_reopen_prepare)(BDRVReopenState *reopen_state,
BlockReopenQueue *queue, Error **errp);
 void (*bdrv_reopen_commit)(BDRVReopenState *reopen_state);
+void (*bdrv_reopen_commit_post)(BDRVReopenState *reopen_state);
 void (*bdrv_reopen_abort)(BDRVReopenState *reopen_state);
 void (*bdrv_join_options)(QDict *options, QDict *old_options);

-- 
2.24.1

[PATCH 2/2] block/qcow2: Move bitmap reopen into bdrv_reopen_commit_post

[Peter Krempa]

The bitmap code requires writing the 'file' child when the qcow2 driver
is reopened in read-write mode.

If the 'file' child is being reopened due to a permissions change, the
modification is commited yet when qcow2_reopen_commit is called. This
means that any attempt to write the 'file' child will end with EBADFD
as the original fd was already closed.

Moving bitmap reopening to the new callback which is called after
permission modifications are commited fixes this as the file descriptor
will be replaced with the correct one.

The above problem manifests itself when reopening 'qcow2' format layer
which uses a 'file-posix' file child which was opened with the
'auto-read-only' property set.

Signed-off-by: Peter Krempa 
---
 block/qcow2.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 3c754f616b..3640e8c07d 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1884,6 +1884,11 @@ fail:
 static void qcow2_reopen_commit(BDRVReopenState *state)
 {
 qcow2_update_options_commit(state->bs, state->opaque);
+g_free(state->opaque);
+}
+
+static void qcow2_reopen_commit_post(BDRVReopenState *state)
+{
 if (state->flags & BDRV_O_RDWR) {
 Error *local_err = NULL;

@@ -1898,7 +1903,6 @@ static void qcow2_reopen_commit(BDRVReopenState *state)
   bdrv_get_node_name(state->bs));
 }
 }
-g_free(state->opaque);
 }

 static void qcow2_reopen_abort(BDRVReopenState *state)
@@ -5534,6 +5538,7 @@ BlockDriver bdrv_qcow2 = {
 .bdrv_close = qcow2_close,
 .bdrv_reopen_prepare  = qcow2_reopen_prepare,
 .bdrv_reopen_commit   = qcow2_reopen_commit,
+.bdrv_reopen_commit_post = qcow2_reopen_commit_post,
 .bdrv_reopen_abort= qcow2_reopen_abort,
 .bdrv_join_options= qcow2_join_options,
 .bdrv_child_perm  = bdrv_format_default_perms,
-- 
2.24.1

Re: [PATCH v4 0/2] virtiofsd: Fix xattr operations

[Vivek Goyal]

On Thu, Feb 27, 2020 at 02:59:25PM +0900, Misono Tomohiro wrote:
> Currently xattr operations on virtiofs does not work properly in some case:
>  - directory ... cannot set
>  - special files (pipe) ... cause hang
> 
> This fixes these problems and now xfstests generic/062 passes on virtiofs
> with -o xattr option (I tested with xfs).
> 
> v3 -> v4:
>  - No logic change
>  - Some code style fix/update comments and commit log as suggested by Vivek
>  - CC qemu-devel ML too
> 
> Previous versions can be found in virtiofs ML:
>  v3: https://www.redhat.com/archives/virtio-fs/2020-February/msg00032.html
> 
> Thanks,
> Misono Tomohiro (2):
>   virtiofsd: passthrough_ll: cleanup getxattr/listxattr
>   virtiofsd: Fix xattr operations

Thanks Misono for this work. I am fine with this patch series.

Acked-by: Vivek Goyal 

Thanks
Vivek

> 
>  tools/virtiofsd/fuse_virtio.c|  13 +++
>  tools/virtiofsd/passthrough_ll.c | 139 ---
>  tools/virtiofsd/seccomp.c|   6 ++
>  3 files changed, 89 insertions(+), 69 deletions(-)
> 
> -- 
> 2.21.1
> 

[PATCH 2/5] hw/arm/mainstone: Simplify since the machines are little-endian only

[Philippe Mathieu-Daudé]

We only build the little-endian softmmu configurations. Checking
for big endian is pointless, remove the unused code.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/arm/mainstone.c | 8 +---
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/hw/arm/mainstone.c b/hw/arm/mainstone.c
index 6e64dfab50..1042017086 100644
--- a/hw/arm/mainstone.c
+++ b/hw/arm/mainstone.c
@@ -119,7 +119,6 @@ static void mainstone_common_init(MemoryRegion 
*address_space_mem,
 DeviceState *mst_irq;
 DriveInfo *dinfo;
 int i;
-int be;
 MemoryRegion *rom = g_new(MemoryRegion, 1);
 
 /* Setup CPU & memory */
@@ -130,11 +129,6 @@ static void mainstone_common_init(MemoryRegion 
*address_space_mem,
 memory_region_set_readonly(rom, true);
 memory_region_add_subregion(address_space_mem, 0, rom);
 
-#ifdef TARGET_WORDS_BIGENDIAN
-be = 1;
-#else
-be = 0;
-#endif
 /* There are two 32MiB flash devices on the board */
 for (i = 0; i < 2; i ++) {
 dinfo = drive_get(IF_PFLASH, 0, i);
@@ -142,7 +136,7 @@ static void mainstone_common_init(MemoryRegion 
*address_space_mem,
i ? "mainstone.flash1" : "mainstone.flash0",
MAINSTONE_FLASH,
dinfo ? blk_by_legacy_dinfo(dinfo) : NULL,
-   sector_len, 4, 0, 0, 0, 0, be)) {
+   sector_len, 4, 0, 0, 0, 0, 0)) {
 error_report("Error registering flash memory");
 exit(1);
 }
-- 
2.21.1

[PATCH 0/5] hw/arm: Remove big-endian checks on machine code

[Philippe Mathieu-Daudé]

We only build the little-endian softmmu configurations.
Checking for big endian is pointless, remove the unused code.

Philippe Mathieu-Daudé (5):
  hw/arm/gumstix: Simplify since the machines are little-endian only
  hw/arm/mainstone: Simplify since the machines are little-endian only
  hw/arm/omap_sx1: Simplify since the machines are little-endian only
  hw/arm/z2: Simplify since the machines are little-endian only
  hw/arm/musicpal: Simplify since the machines are little-endian only

 hw/arm/gumstix.c   | 16 ++--
 hw/arm/mainstone.c |  8 +---
 hw/arm/musicpal.c  | 10 --
 hw/arm/omap_sx1.c  | 11 ++-
 hw/arm/z2.c|  8 +---
 5 files changed, 6 insertions(+), 47 deletions(-)

-- 
2.21.1

[PATCH 1/5] hw/arm/gumstix: Simplify since the machines are little-endian only

[Philippe Mathieu-Daudé]

From: Philippe Mathieu-Daudé 

As the Connex and Verdex machines only boot in little-endian,
we can simplify the code.

Reviewed-by: Peter Maydell 
Signed-off-by: Philippe Mathieu-Daudé 
---
v2: Do not use target_words_bigendian(), arm-softmmu is only
little endian (Peter Maydell)
---
 hw/arm/gumstix.c | 16 ++--
 1 file changed, 2 insertions(+), 14 deletions(-)

diff --git a/hw/arm/gumstix.c b/hw/arm/gumstix.c
index f26a0e8010..3a4bc332c4 100644
--- a/hw/arm/gumstix.c
+++ b/hw/arm/gumstix.c
@@ -51,7 +51,6 @@ static void connex_init(MachineState *machine)
 {
 PXA2xxState *cpu;
 DriveInfo *dinfo;
-int be;
 MemoryRegion *address_space_mem = get_system_memory();
 
 uint32_t connex_rom = 0x0100;
@@ -66,14 +65,9 @@ static void connex_init(MachineState *machine)
 exit(1);
 }
 
-#ifdef TARGET_WORDS_BIGENDIAN
-be = 1;
-#else
-be = 0;
-#endif
 if (!pflash_cfi01_register(0x, "connext.rom", connex_rom,
dinfo ? blk_by_legacy_dinfo(dinfo) : NULL,
-   sector_len, 2, 0, 0, 0, 0, be)) {
+   sector_len, 2, 0, 0, 0, 0, 0)) {
 error_report("Error registering flash memory");
 exit(1);
 }
@@ -87,7 +81,6 @@ static void verdex_init(MachineState *machine)
 {
 PXA2xxState *cpu;
 DriveInfo *dinfo;
-int be;
 MemoryRegion *address_space_mem = get_system_memory();
 
 uint32_t verdex_rom = 0x0200;
@@ -102,14 +95,9 @@ static void verdex_init(MachineState *machine)
 exit(1);
 }
 
-#ifdef TARGET_WORDS_BIGENDIAN
-be = 1;
-#else
-be = 0;
-#endif
 if (!pflash_cfi01_register(0x, "verdex.rom", verdex_rom,
dinfo ? blk_by_legacy_dinfo(dinfo) : NULL,
-   sector_len, 2, 0, 0, 0, 0, be)) {
+   sector_len, 2, 0, 0, 0, 0, 0)) {
 error_report("Error registering flash memory");
 exit(1);
 }
-- 
2.21.1

[PATCH 3/5] hw/arm/omap_sx1: Simplify since the machines are little-endian only

[Philippe Mathieu-Daudé]

We only build the little-endian softmmu configurations. Checking
for big endian is pointless, remove the unused code.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/arm/omap_sx1.c | 11 ++-
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/hw/arm/omap_sx1.c b/hw/arm/omap_sx1.c
index 2bebab4171..de5ff447dc 100644
--- a/hw/arm/omap_sx1.c
+++ b/hw/arm/omap_sx1.c
@@ -114,7 +114,6 @@ static void sx1_init(MachineState *machine, const int 
version)
 DriveInfo *dinfo;
 int fl_idx;
 uint32_t flash_size = flash0_size;
-int be;
 
 if (machine->ram_size != mc->default_ram_size) {
 char *sz = size_to_str(mc->default_ram_size);
@@ -154,17 +153,11 @@ static void sx1_init(MachineState *machine, const int 
version)
 OMAP_CS2_BASE, &cs[3]);
 
 fl_idx = 0;
-#ifdef TARGET_WORDS_BIGENDIAN
-be = 1;
-#else
-be = 0;
-#endif
-
 if ((dinfo = drive_get(IF_PFLASH, 0, fl_idx)) != NULL) {
 if (!pflash_cfi01_register(OMAP_CS0_BASE,
"omap_sx1.flash0-1", flash_size,
blk_by_legacy_dinfo(dinfo),
-   sector_size, 4, 0, 0, 0, 0, be)) {
+   sector_size, 4, 0, 0, 0, 0, 0)) {
 fprintf(stderr, "qemu: Error registering flash memory %d.\n",
fl_idx);
 }
@@ -187,7 +180,7 @@ static void sx1_init(MachineState *machine, const int 
version)
 if (!pflash_cfi01_register(OMAP_CS1_BASE,
"omap_sx1.flash1-1", flash1_size,
blk_by_legacy_dinfo(dinfo),
-   sector_size, 4, 0, 0, 0, 0, be)) {
+   sector_size, 4, 0, 0, 0, 0, 0)) {
 fprintf(stderr, "qemu: Error registering flash memory %d.\n",
fl_idx);
 }
-- 
2.21.1

[PATCH 4/5] hw/arm/z2: Simplify since the machines are little-endian only

[Philippe Mathieu-Daudé]

We only build the little-endian softmmu configurations. Checking
for big endian is pointless, remove the unused code.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/arm/z2.c | 8 +---
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/hw/arm/z2.c b/hw/arm/z2.c
index 4bb237f22d..a0f4095990 100644
--- a/hw/arm/z2.c
+++ b/hw/arm/z2.c
@@ -300,7 +300,6 @@ static void z2_init(MachineState *machine)
 uint32_t sector_len = 0x1;
 PXA2xxState *mpu;
 DriveInfo *dinfo;
-int be;
 void *z2_lcd;
 I2CBus *bus;
 DeviceState *wm;
@@ -308,15 +307,10 @@ static void z2_init(MachineState *machine)
 /* Setup CPU & memory */
 mpu = pxa270_init(address_space_mem, z2_binfo.ram_size, machine->cpu_type);
 
-#ifdef TARGET_WORDS_BIGENDIAN
-be = 1;
-#else
-be = 0;
-#endif
 dinfo = drive_get(IF_PFLASH, 0, 0);
 if (!pflash_cfi01_register(Z2_FLASH_BASE, "z2.flash0", Z2_FLASH_SIZE,
dinfo ? blk_by_legacy_dinfo(dinfo) : NULL,
-   sector_len, 4, 0, 0, 0, 0, be)) {
+   sector_len, 4, 0, 0, 0, 0, 0)) {
 error_report("Error registering flash memory");
 exit(1);
 }
-- 
2.21.1

[PATCH 5/5] hw/arm/musicpal: Simplify since the machines are little-endian only

[Philippe Mathieu-Daudé]

We only build the little-endian softmmu configurations. Checking
for big endian is pointless, remove the unused code.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/arm/musicpal.c | 10 --
 1 file changed, 10 deletions(-)

diff --git a/hw/arm/musicpal.c b/hw/arm/musicpal.c
index db8b03cb83..b2d0cfdac8 100644
--- a/hw/arm/musicpal.c
+++ b/hw/arm/musicpal.c
@@ -1645,22 +1645,12 @@ static void musicpal_init(MachineState *machine)
  * 0xFF80 (if there is 8 MB flash). So remap flash access if the
  * image is smaller than 32 MB.
  */
-#ifdef TARGET_WORDS_BIGENDIAN
-pflash_cfi02_register(0x1ULL - MP_FLASH_SIZE_MAX,
-  "musicpal.flash", flash_size,
-  blk, 0x1,
-  MP_FLASH_SIZE_MAX / flash_size,
-  2, 0x00BF, 0x236D, 0x, 0x,
-  0x, 0x2AAA, 1);
-#else
 pflash_cfi02_register(0x1ULL - MP_FLASH_SIZE_MAX,
   "musicpal.flash", flash_size,
   blk, 0x1,
   MP_FLASH_SIZE_MAX / flash_size,
   2, 0x00BF, 0x236D, 0x, 0x,
   0x, 0x2AAA, 0);
-#endif
-
 }
 sysbus_create_simple(TYPE_MV88W8618_FLASHCFG, MP_FLASHCFG_BASE, NULL);
 
-- 
2.21.1

Re: [PATCH v3 1/4] scripts/simplebench: add simplebench.py

[Aleksandar Markovic]

On Fri, Feb 28, 2020 at 8:19 AM Vladimir Sementsov-Ogievskiy
 wrote:
>
> Add simple benchmark table creator.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> ---
>  scripts/simplebench/simplebench.py | 128 +
>  1 file changed, 128 insertions(+)
>  create mode 100644 scripts/simplebench/simplebench.py
>
> diff --git a/scripts/simplebench/simplebench.py 
> b/scripts/simplebench/simplebench.py
> new file mode 100644
> index 00..59e7314ff6
> --- /dev/null
> +++ b/scripts/simplebench/simplebench.py
> @@ -0,0 +1,128 @@
> +#!/usr/bin/env python
> +#
> +# Simple benchmarking framework
> +#
> +# Copyright (c) 2019 Virtuozzo International GmbH.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see .
> +#
> +
> +
> +def bench_one(test_func, test_env, test_case, count=5, initial_run=True):
> +"""Benchmark one test-case
> +
> +test_func   -- benchmarking function with prototype
> +   test_func(env, case), which takes test_env and test_case
> +   arguments and returns {'seconds': int} (which is benchmark
> +   result) on success and {'error': str} on error. Returned
> +   dict may contain any other additional fields.
> +test_env-- test environment - opaque first argument for test_func
> +test_case   -- test case - opaque second argument for test_func
> +count   -- how many times to call test_func, to calculate average
> +initial_run -- do initial run of test_func, which don't get into result
> +
> +Returns dict with the following fields:
> +'runs': list of test_func results
> +'average':  average seconds per run (exists only if at least one run
> +succeeded)
> +'delta':maximum delta between test_func result and the average
> +(exists only if at least one run succeeded)
> +'n-failed': number of failed runs (exists only if at least one run
> +failed)
> +"""
> +if initial_run:
> +print('  #initial run:')
> +print('   ', test_func(test_env, test_case))
> +
> +runs = []
> +for i in range(count):
> +print('  #run {}'.format(i+1))
> +res = test_func(test_env, test_case)
> +print('   ', res)
> +runs.append(res)
> +
> +result = {'runs': runs}
> +
> +successed = [r for r in runs if ('seconds' in r)]
> +if successed:
> +avg = sum(r['seconds'] for r in successed) / len(successed)
> +result['average'] = avg
> +result['delta'] = max(abs(r['seconds'] - avg) for r in successed)
> +
> +if len(successed) < count:
> +result['n-failed'] = count - len(successed)
> +
> +return result
> +
> +
> +def ascii_one(result):
> +"""Return ASCII representation of bench_one() returned dict."""
> +if 'average' in result:
> +s = '{:.2f} +- {:.2f}'.format(result['average'], result['delta'])
> +if 'n-failed' in result:
> +s += '\n({} failed)'.format(result['n-failed'])
> +return s
> +else:
> +return 'FAILED'

I think it would be visually clearer if "+-" was printed without any
space between it and the following number, using something
like this:

s = ' {:.2f} +-{:.2f}'.format(result['average'], result['delta'])

The resulting table would look like:

--  -  -  -
backup-1   backup-2   mirror
ssd -> ssd   0.43 +-0.004.48 +-0.064.38 +-0.02
ssd -> hdd   10.60 +-0.08   10.69 +-0.18   10.57 +-0.05
ssd -> nbd   33.81 +-0.37   10.67 +-0.17   10.07 +-0.07
--  -  -  -

But, this is just cosmetics.

With or without the suggestion above:

Reviewed-by: Aleksandar Markovic 

> +
> +
> +def bench(test_func, test_envs, test_cases, *args, **vargs):
> +"""Fill benchmark table
> +
> +test_func -- benchmarking function, see bench_one for description
> +test_envs -- list of test environments, see bench_one
> +test_cases -- list of test cases, see bench_one
> +args, vargs -- additional arguments for bench_one
> +
> +Returns dict with the following fields:
> +'envs':  test_envs
> +'cases': test_cases
> +'tab':   filled 2D array, where cell [i][j] is bench_one result for
> + test_cases[i] for te

Re: [PATCH v3 2/4] scripts/simplebench: add qemu/bench_block_job.py

[Aleksandar Markovic]

On Fri, Feb 28, 2020 at 8:19 AM Vladimir Sementsov-Ogievskiy
 wrote:
>
> Add block-job benchmarking helper functions.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> ---

Reviewed-by: Aleksandar Markovic 

>  scripts/simplebench/bench_block_job.py | 119 +
>  1 file changed, 119 insertions(+)
>  create mode 100755 scripts/simplebench/bench_block_job.py
>
> diff --git a/scripts/simplebench/bench_block_job.py 
> b/scripts/simplebench/bench_block_job.py
> new file mode 100755
> index 00..9808d696cf
> --- /dev/null
> +++ b/scripts/simplebench/bench_block_job.py
> @@ -0,0 +1,119 @@
> +#!/usr/bin/env python
> +#
> +# Benchmark block jobs
> +#
> +# Copyright (c) 2019 Virtuozzo International GmbH.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see .
> +#
> +
> +
> +import sys
> +import os
> +import socket
> +import json
> +
> +sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 
> 'python'))
> +from qemu.machine import QEMUMachine
> +from qemu.qmp import QMPConnectError
> +
> +
> +def bench_block_job(cmd, cmd_args, qemu_args):
> +"""Benchmark block-job
> +
> +cmd   -- qmp command to run block-job (like blockdev-backup)
> +cmd_args  -- dict of qmp command arguments
> +qemu_args -- list of Qemu command line arguments, including path to Qemu
> + binary
> +
> +Returns {'seconds': int} on success and {'error': str} on failure, dict 
> may
> +contain addional 'vm-log' field. Return value is compatible with
> +simplebench lib.
> +"""
> +
> +vm = QEMUMachine(qemu_args[0], args=qemu_args[1:])
> +
> +try:
> +vm.launch()
> +except OSError as e:
> +return {'error': 'popen failed: ' + str(e)}
> +except (QMPConnectError, socket.timeout):
> +return {'error': 'qemu failed: ' + str(vm.get_log())}
> +
> +try:
> +res = vm.qmp(cmd, **cmd_args)
> +if res != {'return': {}}:
> +vm.shutdown()
> +return {'error': '"{}" command failed: {}'.format(cmd, str(res))}
> +
> +e = vm.event_wait('JOB_STATUS_CHANGE')
> +assert e['data']['status'] == 'created'
> +start_ms = e['timestamp']['seconds'] * 100 + \
> +e['timestamp']['microseconds']
> +
> +e = vm.events_wait((('BLOCK_JOB_READY', None),
> +('BLOCK_JOB_COMPLETED', None),
> +('BLOCK_JOB_FAILED', None)), timeout=True)
> +if e['event'] not in ('BLOCK_JOB_READY', 'BLOCK_JOB_COMPLETED'):
> +vm.shutdown()
> +return {'error': 'block-job failed: ' + str(e),
> +'vm-log': vm.get_log()}
> +end_ms = e['timestamp']['seconds'] * 100 + \
> +e['timestamp']['microseconds']
> +finally:
> +vm.shutdown()
> +
> +return {'seconds': (end_ms - start_ms) / 100.0}
> +
> +
> +# Bench backup or mirror
> +def bench_block_copy(qemu_binary, cmd, source, target):
> +"""Helper to run bench_block_job() for mirror or backup"""
> +assert cmd in ('blockdev-backup', 'blockdev-mirror')
> +
> +source['node-name'] = 'source'
> +target['node-name'] = 'target'
> +
> +return bench_block_job(cmd,
> +   {'job-id': 'job0', 'device': 'source',
> +'target': 'target', 'sync': 'full'},
> +   [qemu_binary,
> +'-blockdev', json.dumps(source),
> +'-blockdev', json.dumps(target)])
> +
> +
> +def drv_file(filename):
> +return {'driver': 'file', 'filename': filename,
> +'cache': {'direct': True}, 'aio': 'native'}
> +
> +
> +def drv_nbd(host, port):
> +return {'driver': 'nbd',
> +'server': {'type': 'inet', 'host': host, 'port': port}}
> +
> +
> +if __name__ == '__main__':
> +import sys
> +
> +if len(sys.argv) < 4:
> +print('USAGE: {}  '
> +  ' '
> +  ''.format(sys.argv[0]))
> +exit(1)
> +
> +res = bench_block_job(sys.argv[1], json.loads(sys.argv[2]), sys.argv[3:])
> +if 'seconds' in res:
> +print('{:.2f}'.format(res['seconds']))
> +else:
> +print(res)
> --
> 2.21.0
>
>

Re: [PATCH] block: Remove trailing newline in format used by error_report API

[Liam Merwick]

On 28/02/2020 12:36, Philippe Mathieu-Daudé wrote:

The error_report API doesn't want trailing newline characters.
Remove it, to avoid and error when moving the code around:



s/and/an/


   ERROR: Error messages should not contain newlines

Signed-off-by: Philippe Mathieu-Daudé 


Reviewed-by: Liam Merwick 



---
  block.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block.c b/block.c
index 1bdb9c679d..e466d15914 100644
--- a/block.c
+++ b/block.c
@@ -5994,7 +5994,7 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
  /* Couldn't open BS, but we have a size, so it's nonfatal */
  warn_reportf_err(local_err,
  "Could not verify backing image. "
-"This may become an error in future versions.\n");
+"This may become an error in future versions.");
  local_err = NULL;
  } else if (!bs) {
  /* Couldn't open bs, do not have size */

Re: [PATCH v2 05/18] hw/usb/dev-storage: Remove unused "ui/console.h" header

[Gerd Hoffmann]

On Fri, Feb 28, 2020 at 12:46:36PM +0100, Philippe Mathieu-Daudé wrote:
> The USB models related to storage don't need anything from
> "ui/console.h". Remove it.
> 
> Acked-by: John Snow 
> Signed-off-by: Philippe Mathieu-Daudé 

Acked-by: Gerd Hoffmann 

Re: [PATCH v3 3/4] scripts/simplebench: add example usage of simplebench

[Aleksandar Markovic]

On Fri, Feb 28, 2020 at 8:21 AM Vladimir Sementsov-Ogievskiy
 wrote:
>
> This example may be used as a template for custom benchmark.
> It illustrates three things to prepare:
>  - define bench_func
>  - define test environments (columns)
>  - define test cases (rows)
> And final call of simplebench API.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> ---

In future, it would be perhaps useful to add option to output results
in the format of csv (comma separated values) (or similar) - it would
simplify import to other data presentation tools. In any case:

Reviewed-by: Aleksandar Markovic 

>  scripts/simplebench/bench-example.py | 80 
>  1 file changed, 80 insertions(+)
>  create mode 100644 scripts/simplebench/bench-example.py
>
> diff --git a/scripts/simplebench/bench-example.py 
> b/scripts/simplebench/bench-example.py
> new file mode 100644
> index 00..c642a5b891
> --- /dev/null
> +++ b/scripts/simplebench/bench-example.py
> @@ -0,0 +1,80 @@
> +#!/usr/bin/env python3
> +#
> +# Benchmark example
> +#
> +# Copyright (c) 2019 Virtuozzo International GmbH.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see .
> +#
> +
> +import simplebench
> +from bench_block_job import bench_block_copy, drv_file, drv_nbd
> +
> +
> +def bench_func(env, case):
> +""" Handle one "cell" of benchmarking table. """
> +return bench_block_copy(env['qemu_binary'], env['cmd'],
> +case['source'], case['target'])
> +
> +
> +# You may set the following five variables to correct values, to turn this
> +# example to real benchmark.
> +ssd_source = '/path-to-raw-source-image-at-ssd'
> +ssd_target = '/path-to-raw-target-image-at-ssd'
> +hdd_target = '/path-to-raw-source-image-at-hdd'
> +nbd_ip = 'nbd-ip-addr'
> +nbd_port = 'nbd-port-number'
> +
> +# Test-cases are "rows" in benchmark resulting table, 'id' is a caption for
> +# the row, other fields are handled by bench_func.
> +test_cases = [
> +{
> +'id': 'ssd -> ssd',
> +'source': drv_file(ssd_source),
> +'target': drv_file(ssd_target)
> +},
> +{
> +'id': 'ssd -> hdd',
> +'source': drv_file(ssd_source),
> +'target': drv_file(hdd_target)
> +},
> +{
> +'id': 'ssd -> nbd',
> +'source': drv_file(ssd_source),
> +'target': drv_nbd(nbd_ip, nbd_port)
> +},
> +]
> +
> +# Test-envs are "columns" in benchmark resulting table, 'id is a caption for
> +# the column, other fields are handled by bench_func.
> +test_envs = [
> +{
> +'id': 'backup-1',
> +'cmd': 'blockdev-backup',
> +'qemu_binary': '/path-to-qemu-binary-1'
> +},
> +{
> +'id': 'backup-2',
> +'cmd': 'blockdev-backup',
> +'qemu_binary': '/path-to-qemu-binary-2'
> +},
> +{
> +'id': 'mirror',
> +'cmd': 'blockdev-mirror',
> +'qemu_binary': '/path-to-qemu-binary-1'
> +}
> +]
> +
> +result = simplebench.bench(bench_func, test_envs, test_cases, count=3)
> +print(simplebench.ascii(result))
> --
> 2.21.0
>
>

Re: [PATCH v3 4/4] MAINTAINERS: add simplebench

[Aleksandar Markovic]

On Fri, Feb 28, 2020 at 8:19 AM Vladimir Sementsov-Ogievskiy
 wrote:
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> ---

Reviewed-by: Aleksandar Markovic 

>  MAINTAINERS | 5 +
>  1 file changed, 5 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 5e5e3e52d6..16d069adc5 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2038,6 +2038,11 @@ F: python/qemu/*py
>  F: scripts/*.py
>  F: tests/*.py
>
> +Benchmark util
> +M: Vladimir Sementsov-Ogievskiy 
> +S: Maintained
> +F: scripts/simplebench/
> +
>  QAPI
>  M: Markus Armbruster 
>  M: Michael Roth 
> --
> 2.21.0
>
>

Re: [PATCH v3 1/4] scripts/simplebench: add simplebench.py

[Vladimir Sementsov-Ogievskiy]

28.02.2020 16:03, Aleksandar Markovic wrote:

On Fri, Feb 28, 2020 at 8:19 AM Vladimir Sementsov-Ogievskiy
 wrote:


Add simple benchmark table creator.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
  scripts/simplebench/simplebench.py | 128 +
  1 file changed, 128 insertions(+)
  create mode 100644 scripts/simplebench/simplebench.py

diff --git a/scripts/simplebench/simplebench.py 
b/scripts/simplebench/simplebench.py
new file mode 100644
index 00..59e7314ff6
--- /dev/null
+++ b/scripts/simplebench/simplebench.py
@@ -0,0 +1,128 @@
+#!/usr/bin/env python
+#
+# Simple benchmarking framework
+#
+# Copyright (c) 2019 Virtuozzo International GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+
+def bench_one(test_func, test_env, test_case, count=5, initial_run=True):
+"""Benchmark one test-case
+
+test_func   -- benchmarking function with prototype
+   test_func(env, case), which takes test_env and test_case
+   arguments and returns {'seconds': int} (which is benchmark
+   result) on success and {'error': str} on error. Returned
+   dict may contain any other additional fields.
+test_env-- test environment - opaque first argument for test_func
+test_case   -- test case - opaque second argument for test_func
+count   -- how many times to call test_func, to calculate average
+initial_run -- do initial run of test_func, which don't get into result
+
+Returns dict with the following fields:
+'runs': list of test_func results
+'average':  average seconds per run (exists only if at least one run
+succeeded)
+'delta':maximum delta between test_func result and the average
+(exists only if at least one run succeeded)
+'n-failed': number of failed runs (exists only if at least one run
+failed)
+"""
+if initial_run:
+print('  #initial run:')
+print('   ', test_func(test_env, test_case))
+
+runs = []
+for i in range(count):
+print('  #run {}'.format(i+1))
+res = test_func(test_env, test_case)
+print('   ', res)
+runs.append(res)
+
+result = {'runs': runs}
+
+successed = [r for r in runs if ('seconds' in r)]
+if successed:
+avg = sum(r['seconds'] for r in successed) / len(successed)
+result['average'] = avg
+result['delta'] = max(abs(r['seconds'] - avg) for r in successed)
+
+if len(successed) < count:
+result['n-failed'] = count - len(successed)
+
+return result
+
+
+def ascii_one(result):
+"""Return ASCII representation of bench_one() returned dict."""
+if 'average' in result:
+s = '{:.2f} +- {:.2f}'.format(result['average'], result['delta'])
+if 'n-failed' in result:
+s += '\n({} failed)'.format(result['n-failed'])
+return s
+else:
+return 'FAILED'


I think it would be visually clearer if "+-" was printed without any
space between it and the following number, using something
like this:

s = ' {:.2f} +-{:.2f}'.format(result['average'], result['delta'])

The resulting table would look like:

--  -  -  -
 backup-1   backup-2   mirror
ssd -> ssd   0.43 +-0.004.48 +-0.064.38 +-0.02
ssd -> hdd   10.60 +-0.08   10.69 +-0.18   10.57 +-0.05
ssd -> nbd   33.81 +-0.37   10.67 +-0.17   10.07 +-0.07
--  -  -  -

But, this is just cosmetics.

With or without the suggestion above:

Reviewed-by: Aleksandar Markovic 


Thanks for reviewing!

Agree with this change, but I don't think it worth to resend the series for 
this one space)
Hope it may be applied with pull request..




+
+
+def bench(test_func, test_envs, test_cases, *args, **vargs):
+"""Fill benchmark table
+
+test_func -- benchmarking function, see bench_one for description
+test_envs -- list of test environments, see bench_one
+test_cases -- list of test cases, see bench_one
+args, vargs -- additional arguments for bench_one
+
+Returns dict with the following fields:
+'envs':  test_envs
+'cases': test_cases
+'tab':   filled 2D array, where cell [i][j] is bench_one result for
+ test_cases[i] fo

Re: [PULL 0/5] MIPS queue for February 27th, 2020

[Peter Maydell]

On Thu, 27 Feb 2020 at 13:27, Aleksandar Markovic
 wrote:
>
> From: Aleksandar Markovic 
>
> The following changes since commit db736e0437aa6fd7c1b7e4599c17f9619ab6b837:
>
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into 
> staging (2020-02-25 13:31:16 +)
>
> are available in the git repository at:
>
>   https://github.com/AMarkovic/qemu tags/mips-queue-feb-27-2020
>
> for you to fetch changes up to 0484d9d4fbe9beacd3dd36fdf7fc3e14940bf078:
>
>   tests/acceptance: Count multiple Tux logos displayed on framebuffer 
> (2020-02-27 14:19:07 +0100)
>
> 
>
> MIPS queue for February 27th, 2020
>
>   - Adjusting maintainership of MIPS KVM CPUs
>   - Machine memory and interrupt-related minor improvements
>   - Acceptance test for SMP support on Malta with I6400 CPU
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
for any user-visible changes.

-- PMM

[Bug 1865048] Re: qemu-img --force-share does not disable file locking

[Max Reitz]

Hi Olaf,

Every “node” in the block graph corresponds to some driver.  A driver
can be a protocol or a format driver (or a filter driver, but that isn’t
important here).  In your example, there is only a single node, for a
protocol driver (namely “file”).  You need a format driver node on top
to interpret the image format.

If you use file.driver=file,file.filename=foo.qcow2,file.locking=off,
then that specifies the options driver, filename, and locking for a node
under another node’s “file” link.  So this has to create two nodes.  The
node on top (for which no options are specified) should default to being
a format node whose format is probed.

Of course you can also give options to the top (format) node, like e.g.
the driver.  (In fact, you should probably give the driver, because
format probing is considered dangerous.)

Then it would look like this:
driver=qcow2,file.driver=file,file.filename=foo.qcow2,file.locking=off

(Or, in JSON, but that only works with qemu’s -blockdev (but I think it’s 
better for visualizing the resulting block graph:
 {"node-name": "some-node-name",
  "driver": "qcow2",
  "file": {
  "driver": "file",
  "filename": "foo.qcow2",
  "locking": false
  } })


Hope that helps,

Max

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1865048

Title:
  qemu-img --force-share does not disable file locking

Status in QEMU:
  Invalid

Bug description:
  The new option "--force-share" for qemu-img does not disable file
  locking.

  I tried it with version qemu-img version 2.11.1(Debian 1:2.11+dfsg-
  1ubuntu7.21~cloud0) and I traced the source code of the current git
  trunk.

  Sample to demonstrate:

  # strace qemu-img info --force-share testfile.qcow2   2>&1 | grep F_RDLCK
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, 
l_len=1}) = 0

  I traced the passing of the --force-share option through the source
  code (I used commit 6c599282f8 as of Mon Feb 17 13:32:25 2020 +)

  qemu-img.c:img_info()
  force_share = true;
  qemu-img.c:collect_image_info_list(force_share)
  qemu-img.c:img_open(force_share)
  qemu-img.c:img_open_file(force_share)
  qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true);
  block/block-backend.c:blk_new_open(options)
  block.c:bdrv_open(options)
  block.c:bdrv_open_inheritoptions()
  block.c:bdrv_open_common(options)
  bs->force_share = qemu_opt_get_bool(opts, BDRV_OPT_FORCE_SHARE, 
false);
  block.c:bdrv_open_driver(bs)
  include/block/block_int.h:int (*bdrv_file_open)(BlockDriverState *bs, QDict 
*options, int flags,
  block/file-posix.c:.bdrv_file_open = raw_open,
  block/file-posix.c:raw_open_common(bs)
  locking = qapi_enum_parse(&OnOffAuto_lookup,
qemu_opt_get(opts, "locking"),
ON_OFF_AUTO_AUTO, &local_err);
  ignoring bs->force_share

  At the end, bs->force_share is ignored in determining the locking
  value.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1865048/+subscriptions

Re: [PATCH v4 00/10] vTPM for aarch64

[Stefan Berger]

On 2/27/20 3:07 AM, Auger Eric wrote:

Hi Stefan,
On 2/26/20 11:44 PM, Stefan Berger wrote:

On 2/26/20 3:59 PM, Eric Auger wrote:

This series adds the capability to instantiate an MMIO TPM TIS
in ARM virt. It is candidate to qemu 5.0.

I queued it now here:
https://github.com/stefanberger/qemu-tpm/commits/tpm-next

I will send the PR within a few days. Thanks!

Thank you. I will just ping Peter to make sure he has no comments on

[PATCH v4 06/10] hw/arm/virt: vTPM support



The little dent is now an arm boot failure:


https://travis-ci.org/stefanberger/qemu-tpm/jobs/655573347?utm_medium=notification&utm_source=email


Have a look at the raw log.


   Stefan

Re: [PATCH v4 00/10] vTPM for aarch64

[Auger Eric]

Hi Stefan,
On 2/28/20 3:37 PM, Stefan Berger wrote:
> On 2/27/20 3:07 AM, Auger Eric wrote:
>> Hi Stefan,
>> On 2/26/20 11:44 PM, Stefan Berger wrote:
>>> On 2/26/20 3:59 PM, Eric Auger wrote:
 This series adds the capability to instantiate an MMIO TPM TIS
 in ARM virt. It is candidate to qemu 5.0.
>>> I queued it now here:
>>> https://github.com/stefanberger/qemu-tpm/commits/tpm-next
>>>
>>> I will send the PR within a few days. Thanks!
>> Thank you. I will just ping Peter to make sure he has no comments on
>>
>> [PATCH v4 06/10] hw/arm/virt: vTPM support
> 
> 
> The little dent is now an arm boot failure:
> 
> 
> https://travis-ci.org/stefanberger/qemu-tpm/jobs/655573347?utm_medium=notification&utm_source=email
is this really related to the sysbus TPM-TIS addition? I have the
impression cubieboard acceptance tests (ARM 32b) are failing. I touched
ARM virt machine.

Thanks

Eric
> 
> 
> 
> Have a look at the raw log.
> 
> 
>    Stefan
> 
> 
> 

[Bug 1865160] [NEW] Unpredictable behaviour resulting in User process faults

[A. Farrell]

Public bug reported:

An example of the behaviour can be reproduced when using NPM, whereby
running the command multiple times will result in a variety of error
conditions causing the command to fail:

Example of failure:

Segmentation fault.] / rollbackFailedOptional: verb npm-session
1a805a5e0ff7b8f5

[ 3144.216869] User process fault: interruption code 0038 ilc:3 
[ 3144.216981] Failing address: 66616c736500 TEID: 66616c7365000800
[ 3144.217009] Fault in primary space mode while using user ASCE.
[ 3144.217055] AS:ed28c1c7 R3:0024 

Feb 28 14:32:08 qemus390x kernel: [ 3144.216869] User process fault: 
interruption code 0038 ilc:3 
Feb 28 14:32:08 qemus390x kernel: [ 3144.216981] Failing address: 
66616c736500 TEID: 66616c7365000800
Feb 28 14:32:08 qemus390x kernel: [ 3144.217009] Fault in primary space mode 
while using user ASCE.
Feb 28 14:32:08 qemus390x kernel: [ 3144.217055] AS:ed28c1c7 
R3:0024 
Feb 28 14:32:08 qemus390x kernel: [ 3144.217217] CPU: 2 PID: 1018 Comm: npm Not 
tainted 4.15.0-88-generic #88-Ubuntu
Feb 28 14:32:08 qemus390x kernel: [ 3144.217234] Hardware name: QEMU 2964 QEMU 
(KVM/Linux)
Feb 28 14:32:08 qemus390x kernel: [ 3144.217257] User PSW : 185db982 
c1d5a1a1
Feb 28 14:32:08 qemus390x kernel: [ 3144.217290]R:0 T:1 IO:1 EX:1 
Key:0 M:1 W:0 P:1 AS:0 CC:2 PM:0 RI:0 EA:3
Feb 28 14:32:08 qemus390x kernel: [ 3144.217322] User GPRS: 02aa03705200 
006a16d73ac1 003da4b829f1 
Feb 28 14:32:08 qemus390x kernel: [ 3144.217343]003da4b82a08 
003da4b82a08 02aa036a92ec 
Feb 28 14:32:08 qemus390x kernel: [ 3144.217364]003da4b829f1 
03ffdb8f7e50 003da4b82a08 03ffdb8f7d88
Feb 28 14:32:08 qemus390x kernel: [ 3144.217385]66616c736500 
02aa036a05b0 02aa015bcfb2 03ffdb8f7d88
Feb 28 14:32:08 qemus390x kernel: [ 3144.217512] User Code:#006a16d73b00: 
c0f400df  brcl15,006a16d73cbe
Feb 28 14:32:08 qemus390x kernel: [ 3144.217512]   >006a16d73b06: 
a729  lghi%r2,0
Feb 28 14:32:08 qemus390x kernel: [ 3144.217512]006a16d73b0a: 
07fe  bcr 15,%r14
Feb 28 14:32:08 qemus390x kernel: [ 3144.217512]006a16d73b0c: 
c02f01f3  llilf   %r2,499
Feb 28 14:32:08 qemus390x kernel: [ 3144.217512]006a16d73b12: 
e3d0dff8ff71  lay %r13,-8(%r13)
Feb 28 14:32:08 qemus390x kernel: [ 3144.217512]006a16d73b18: 
e320d024  stg %r2,0(%r13)
Feb 28 14:32:08 qemus390x kernel: [ 3144.217512]006a16d73b1e: 
c02802aa  iihf%r2,682
Feb 28 14:32:08 qemus390x kernel: [ 3144.217724] Last Breaking-Event-Address:
Feb 28 14:32:08 qemus390x kernel: [ 3144.217759]  [<02aa015bcfae>] 
0x2aa015bcfae


QEMU emulator version 4.2.0
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

QEMU Command:

sudo qemu-system-s390x -smp cpus=5 -machine s390-ccw-virtio -cpu
max,zpci=on -serial telnet::4441,server -display none -m 4096 -net nic
-net tap  -drive file=ubuntu.root,if=none,id=drive-virtio-
disk0,format=raw,cache=none -device virtio-blk-ccw,devno=fe.0.0003,drive
=drive-virtio-disk0,id=virtio-disk0,bootindex=100,scsi=off -drive
file=ubuntu.home,if=none,id=drive-virtio-disk1,format=raw,cache=none
-device virtio-blk-ccw,devno=fe.0.0002,drive=drive-virtio-disk1,id
=virtio-disk1,bootindex=1,scsi=off -drive file=ubuntu.swap,if=none,id
=drive-virtio-disk4,format=raw,cache=none -device virtio-blk-
ccw,devno=fe.0.0005,drive=drive-virtio-disk4,id=virtio-
disk4,bootindex=101,scsi=off


Ubuntu 18.04.4 LTS qemus390x ttysclp0

** Affects: qemu
 Importance: Undecided
 Status: New


** Tags: qemu-system-s390x

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1865160

Title:
  Unpredictable behaviour resulting in User process faults

Status in QEMU:
  New

Bug description:
  An example of the behaviour can be reproduced when using NPM, whereby
  running the command multiple times will result in a variety of error
  conditions causing the command to fail:

  Example of failure:

  Segmentation fault.] / rollbackFailedOptional: verb npm-session
  1a805a5e0ff7b8f5

  [ 3144.216869] User process fault: interruption code 0038 ilc:3 
  [ 3144.216981] Failing address: 66616c736500 TEID: 66616c7365000800
  [ 3144.217009] Fault in primary space mode while using user ASCE.
  [ 3144.217055] AS:ed28c1c7 R3:0024 

  Feb 28 14:32:08 qemus390x kernel: [ 3144.216869] User process fault: 
interruption code 0038 ilc:3 
  Feb 28 14:32:08 qemus390x kernel: [ 3144.216981] Failing address: 
66616c736500 TEID: 66616c7365000800
  Feb 28 14:32:08 qemus390x kernel: [ 3144.217009] Fault in primary space mode 
while using user ASCE.
  Feb 28 14:32:08 qemus390x kernel: [ 3

Re: [PATCH 4/5] KVM: Kick resamplefd for split kernel irqchip

[Peter Xu]

On Fri, Feb 28, 2020 at 11:34:46AM +0100, Paolo Bonzini wrote:
> On 26/02/20 23:55, Peter Xu wrote:
> > +
> > +/* Kick resamplefd if KVM is bypassed */
> > +kvm_resample_fd_notify(n);
> 
> This is only needed for level-triggered interrupts, so it can be placed
> below the test.

Yes I was thinking it the same way, however...  I noticed that VFIO
will even mask edge INTx after getting IRQ (vfio_intx_handler).  And
if look into current KVM kernel implementation, it has done the same
logic to ack edge triggerred irqs (kvm_ioapic_update_eoi_one), we
should logically ack it here too even for edge-triggered?

Thanks,

> 
> Paolo
> 
> > +if (((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1) !=
> > +IOAPIC_TRIGGER_LEVEL) {
> >  continue;
> 

-- 
Peter Xu

[PATCH] scsi/qemu-pr-helper: Fix out-of-bounds access to trnptid_list[]

[Christophe de Dinechin]

Compile error reported by gcc 10.0.1:

scsi/qemu-pr-helper.c: In function ‘multipath_pr_out’:
scsi/qemu-pr-helper.c:523:32: error: array subscript  is outside array 
bounds of ‘struct transportid *[0]’ [-Werror=array-bounds]
  523 | paramp.trnptid_list[paramp.num_transportid++] = id;
  | ~~~^~
In file included from scsi/qemu-pr-helper.c:36:
/usr/include/mpath_persist.h:168:22: note: while referencing ‘trnptid_list’
  168 |  struct transportid *trnptid_list[];
  |  ^~~~
scsi/qemu-pr-helper.c:424:35: note: defined here ‘paramp’
  424 | struct prout_param_descriptor paramp;
  |   ^~

This highlights an actual implementation issue in function multipath_pr_out.
The variable paramp is declared with type `struct prout_param_descriptor`,
which is a struct terminated by an empty array in mpath_persist.h:

struct transportid *trnptid_list[];

That empty array was filled with code that looked like that:

trnptid_list[paramp.descr.num_transportid++] = id;

This is an actual out-of-bounds access.

The fix is to replace `paramp` with an anonymous struct that adds
additional space for the data, called `trnptid_list_storage`.
That space provides MATH_MX_TIDS entries, and is not accessed directly
but through a pointer to `descr.trnptid_list`, in the unlikely case a
future compiler inserts some padding between the two structs.

Signed-off-by: Christophe de Dinechin 
---
 scsi/qemu-pr-helper.c | 18 --
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c
index 0659ceef09..01013221b3 100644
--- a/scsi/qemu-pr-helper.c
+++ b/scsi/qemu-pr-helper.c
@@ -421,7 +421,12 @@ static int multipath_pr_out(int fd, const uint8_t *cdb, 
uint8_t *sense,
 int rq_servact = cdb[1];
 int rq_scope = cdb[2] >> 4;
 int rq_type = cdb[2] & 0xf;
-struct prout_param_descriptor paramp;
+struct
+{
+struct prout_param_descriptor descr;
+struct transportid *trnptid_list_storage[MPATH_MX_TIDS];
+} paramp;
+struct transportid **trnptid_list = paramp.descr.trnptid_list;
 char transportids[PR_HELPER_DATA_SIZE];
 int r;
 
@@ -455,9 +460,9 @@ static int multipath_pr_out(int fd, const uint8_t *cdb, 
uint8_t *sense,
  * do the opposite).
  */
 memset(¶mp, 0, sizeof(paramp));
-memcpy(¶mp.key, ¶m[0], 8);
-memcpy(¶mp.sa_key, ¶m[8], 8);
-paramp.sa_flags = param[20];
+memcpy(¶mp.descr.key, ¶m[0], 8);
+memcpy(¶mp.descr.sa_key, ¶m[8], 8);
+paramp.descr.sa_flags = param[20];
 if (sz > PR_OUT_FIXED_PARAM_SIZE) {
 size_t transportid_len;
 int i, j;
@@ -520,12 +525,13 @@ static int multipath_pr_out(int fd, const uint8_t *cdb, 
uint8_t *sense,
 return CHECK_CONDITION;
 }
 
-paramp.trnptid_list[paramp.num_transportid++] = id;
+assert(paramp.descr.num_transportid < MPATH_MX_TIDS);
+trnptid_list[paramp.descr.num_transportid++] = id;
 }
 }
 
 r = mpath_persistent_reserve_out(fd, rq_servact, rq_scope, rq_type,
- ¶mp, noisy, verbose);
+ ¶mp.descr, noisy, verbose);
 return mpath_reconstruct_sense(fd, r, sense);
 }
 #endif
-- 
2.25.1

Re: [PATCH 4/5] KVM: Kick resamplefd for split kernel irqchip

[Paolo Bonzini]

On 28/02/20 15:58, Peter Xu wrote:
> Yes I was thinking it the same way, however...  I noticed that VFIO
> will even mask edge INTx after getting IRQ (vfio_intx_handler).

INTx is always level-triggered so that's okay.

> And if look into current KVM kernel implementation, it has done the
> same logic to ack edge triggerred irqs (kvm_ioapic_update_eoi_one),
> we should logically ack it here too even for edge-triggered?

Right, that's because it's using ack notifiers also for in-kernel PIT
and for RTC.  But yeah, it makes sense to have userspace IOAPIC
resamplefd match the functionality of the kernel ones.  Thanks!

Paolo

Re: [PATCH 0/5] vfio/pci: Fix up breakage against split irqchip and INTx

[Peter Xu]

On Fri, Feb 28, 2020 at 11:36:55AM +0100, Paolo Bonzini wrote:
> On 26/02/20 23:50, Peter Xu wrote:
> > VFIO INTx is not working with split irqchip.  On new kernels KVM_IRQFD
> > will directly fail with resamplefd attached so QEMU will automatically
> > fallback to the INTx slow path.  However on old kernels it's still
> > broken.
> > 
> > Only until recently I noticed that this could also break PXE boot for
> > assigned NICs [1].  My wild guess is that the PXE ROM will be mostly
> > using INTx as well, which means we can't bypass that even if we
> > enables MSI for the guest kernel.
> > 
> > This series tries to first fix this issue function-wise, then speed up
> > for the INTx again with resamplefd (mostly following the ideas
> > proposed by Paolo one year ago [2]).  My TCP_RR test shows that:
> > 
> >   - Before this series: this is broken, no number to show
> > 
> >   - After patch 1 (enable slow path): get 63% perf comparing to full
> > kernel irqchip
> 
> Oh, I thought something like patch 1 had already been applied.
> 
> One comment: because you're bypassing IOAPIC when raising the irq, the
> IOAPIC's remote_irr for example will not be set.  Most OSes probably
> don't care, but it's at least worth a comment.

Ouch I should definitely do that...  How about something like this
(in ioapic_eoi_broadcast(), I even changed kvm_resample_fd_notify to
return a boolean to show whether some GSI is kicked so for this case
we don't need to proceed on checking irr and remote irr):

/*
 * When IOAPIC is in the userspace while APIC is still in
 * the kernel (i.e., split irqchip), we have a trick to
 * kick the resamplefd logic for registered irqfds from
 * userspace to deactivate the IRQ.  When that happens, it
 * means the irq bypassed userspace IOAPIC (so the irr and
 * remote-irr of the table entry should be bypassed too
 * even if interrupt come), then we don't need to clear
 * the remote-IRR and check irr again because they'll
 * always be zeros.
 */
if (kvm_resample_fd_notify(n)) {
continue;
}

I confess this is still tricky, and actually after some careful read I
noticed you've proposed a similar kernel fix for the problem too which
I overlooked (https://patchwork.kernel.org/patch/10738541/#22609933).
My current thought is that we keep this hackery in userspace only so
we keep split+resamplefd forbidden in the kernel and be clean there.

What's your opinion?

(I should have marked this series as RFC when post)

Thanks,

-- 
Peter Xu

Re: [PATCH 0/5] vfio/pci: Fix up breakage against split irqchip and INTx

[Paolo Bonzini]

On 28/02/20 16:25, Peter Xu wrote:
> My current thought is that we keep this hackery in userspace only so
> we keep split+resamplefd forbidden in the kernel and be clean there.

It is better, yes.  The kernel solution makes some sense because split
irqchip _does_ have a concept of ioapic GSIs and routes.  But the kernel
patch would not fix the fact that remote-irr remains zero, which would
need a userspace change anyway.

Paolo

[PATCH v3 03/33] texi2pod: parse @include directives outside "@c man" blocks

[Peter Maydell]

From: Paolo Bonzini 

This enables splitting the huge qemu-doc.texi file and keeping parallel
Texinfo and rST versions of the documentation.  texi2pod is not going to
live much longer and hardly anyone cares about its upstream status,
so the temporary fork should be acceptable.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-4-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 scripts/texi2pod.pl | 36 ++--
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
index 839b7917cf7..8bfc6f6f4c4 100755
--- a/scripts/texi2pod.pl
+++ b/scripts/texi2pod.pl
@@ -143,6 +143,24 @@ while(<$inf>) {
next;
 };
 
+# Single line command handlers.
+
+/^\@include\s+(.+)$/ and do {
+   push @instack, $inf;
+   $inf = gensym();
+   $file = postprocess($1);
+
+   # Try cwd and $ibase, then explicit -I paths.
+   $done = 0;
+   foreach $path ("", $ibase, @ipath) {
+   $mypath = $file;
+   $mypath = $path . "/" . $mypath if ($path ne "");
+   open($inf, "<" . $mypath) and ($done = 1, last);
+   }
+   die "cannot find $file" if !$done;
+   next;
+};
+
 next unless $output;
 
 # Discard comments.  (Can't do it above, because then we'd never see
@@ -242,24 +260,6 @@ while(<$inf>) {
s/>/>/g;
 }
 
-# Single line command handlers.
-
-/^\@include\s+(.+)$/ and do {
-   push @instack, $inf;
-   $inf = gensym();
-   $file = postprocess($1);
-
-   # Try cwd and $ibase, then explicit -I paths.
-   $done = 0;
-   foreach $path ("", $ibase, @ipath) {
-   $mypath = $file;
-   $mypath = $path . "/" . $mypath if ($path ne "");
-   open($inf, "<" . $mypath) and ($done = 1, last);
-   }
-   die "cannot find $file" if !$done;
-   next;
-};
-
 /^\@(?:section|unnumbered|unnumberedsec|center)\s+(.+)$/
and $_ = "\n=head2 $1\n";
 /^\@subsection\s+(.+)$/
-- 
2.20.1

[PATCH v3 00/33] Convert qemu-doc to rST

[Peter Maydell]

Hi; this series does a complete conversion of qemu-doc from
Texinfo to rST, including the hxtool-generated parts and
creation of the qemu.1 manpage from rST.

It's marked v3 because it's a development of the v2 that
Paolo sent out earlier this week.

Changes from v2:
 * I made the various review-comment fixes I suggested in
   replies to Paolo's series
 * rebased on current master
 * new patches at the end of the series which do the conversion
   of the .hx file doc fragments to rST
   (I did part of this semi-by-hand and then qemu-options.hx
   entirely automatically)
 * new patches which generate the qemu.1 manpage with Sphinx
 * new patches which remove the old qemu-doc makefile runes
   and other references to it
 * new patches which delete the old texinfo sources, etc

The only thing left still using Texinfo after this is the
docs autogenerated from the QAPI doc-comments, which are
their own standalone html and manpages so not affected by this.

A couple of notes:
 * I haven't actually been in a position to test the cocoa.m
   update of the HTML filename
 * qemu.nsi (the Windows installer config file) thinks that
   qemu-doc.html is the only doc file it needs to know about,
   which is clearly wrong. However I don't have any idea about
   the Windows installer to be able to update or test it...

The conversion is a little rough around the edges in a few
place (mostly I have noted in commit messages when this is
the case) but I would like to argue for (assuming we're happy
with the series broadly) taking it into master and then refining
it in-place. Having it out-of-tree for long is an invitation
to conflicts and to accidentally losing docs updates if they
hit master as changes to the texi or hx files before this
series goes in.

You can find a prerendered set of docs at
https://people.linaro.org/~peter.maydell/qdoc-no-texi/
(the interesting part is the system emulation user's guide,
mostly), and a copy of the new manpage at
https://people.linaro.org/~peter.maydell/qemu.1
(download and examine with 'man -l path/to/qemu.1').

thanks
-- PMM

Kashyap Chamarthy (1):
  docs/system: Convert qemu-cpu-models.texi to rST

Paolo Bonzini (13):
  qemu-doc: convert user-mode emulation to a separate Sphinx manual
  qemu-doc: remove target OS documentation
  texi2pod: parse @include directives outside "@c man" blocks
  qemu-doc: split CPU models doc between MIPS and x86 parts
  qemu-doc: split qemu-doc.texi in multiple files
  qemu-doc: extract common system emulator documentation from the PC
section
  qemu-doc: move system requirements chapter inside PC section
  qemu-doc: split target sections to separate files
  qemu-doc: move qemu-tech.texi into main section
  qemu-doc: move included files to docs/system
  qemu-doc: remove indices other than findex
  docs/system: put qemu-block-drivers body in an included file
  docs/system: convert Texinfo documentation to rST

Peter Maydell (19):
  qemu-doc: Remove the "CPU emulation" part of the "Implementation
notes"
  docs: Create defs.rst.inc as a place to define substitutions
  docs/system: Convert security.texi to rST format
  docs/system: convert managed startup to rST.
  docs/system: convert the documentation of deprecated features to rST.
  hmp-commands.hx: Add rST documentation fragments
  hmp-commands-info.hx: Add rST documentation fragments
  doc/scripts/hxtool.py: Strip trailing ':' from DEFHEADING/ARCHHEADING
  docs: Roll semihosting option information into qemu-options.hx
  docs: Roll -prom-env and -g target-specific info into qemu-options.hx
  scripts/hxtool-conv: Archive script used in qemu-options.hx conversion
  qemu-options.hx: Add rST documentation fragments
  qemu-options.hx: Fix up the autogenerated rST
  docs: Split out sections for the manpage into .rst.inc files
  docs: Generate qemu.1 manpage with Sphinx
  ui/cocoa.m: Update documentation file and pathname
  docs: Stop building qemu-doc
  docs: Remove old texinfo sources
  *.hx: Remove all the STEXI/ETEXI blocks

 docs/specs/ivshmem-spec.txt   |4 +-
 Makefile  |   47 +-
 .gitignore|3 -
 MAINTAINERS   |7 +-
 docs/conf.py  |6 +
 docs/defs.rst.inc |   15 +
 docs/index.html.in|2 +-
 docs/index.rst|1 +
 docs/qemu-cpu-models.texi |  677 --
 docs/sphinx/hxtool.py |   10 +-
 docs/system/build-platforms.rst   |   79 +
 docs/system/conf.py   |8 +-
 docs/system/cpu-models-mips.rst.inc   |  105 +
 docs/system/cpu-models-x86.rst.inc|  365 +
 docs/system/deprecated.rst|  446 +
 docs/system/device-url-syntax.rst.inc |  228 +
 docs/system/gdb.rst   |   81 +
 docs/system/images.rst 

[PATCH v3 02/33] qemu-doc: remove target OS documentation

[Peter Maydell]

From: Paolo Bonzini 

This section covers OSes up to Windows 2000, and as such it is mostly
obsolete.  Zap it.

Reviewed-by: Peter Maydell 
Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-3-pbonz...@redhat.com
Signed-off-by: Peter Maydell 
---
 qemu-doc.texi | 96 ---
 1 file changed, 96 deletions(-)

diff --git a/qemu-doc.texi b/qemu-doc.texi
index 6636816b85f..e555dea4ef6 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -148,7 +148,6 @@ accelerator is required to use more than one host CPU for 
emulation.
 * vnc_security::   VNC security
 * network_tls::TLS setup for network services
 * gdb_usage::  GDB usage
-* pcsys_os_specific::  Target OS specific information
 @end menu
 
 @node pcsys_introduction
@@ -1605,101 +1604,6 @@ received: "OK"
 @end example
 @end table
 
-@node pcsys_os_specific
-@section Target OS specific information
-
-@subsection Linux
-
-To have access to SVGA graphic modes under X11, use the @code{vesa} or
-the @code{cirrus} X11 driver. For optimal performances, use 16 bit
-color depth in the guest and the host OS.
-
-When using a 2.6 guest Linux kernel, you should add the option
-@code{clock=pit} on the kernel command line because the 2.6 Linux
-kernels make very strict real time clock checks by default that QEMU
-cannot simulate exactly.
-
-When using a 2.6 guest Linux kernel, verify that the 4G/4G patch is
-not activated because QEMU is slower with this patch. The QEMU
-Accelerator Module is also much slower in this case. Earlier Fedora
-Core 3 Linux kernel (< 2.6.9-1.724_FC3) were known to incorporate this
-patch by default. Newer kernels don't have it.
-
-@subsection Windows
-
-If you have a slow host, using Windows 95 is better as it gives the
-best speed. Windows 2000 is also a good choice.
-
-@subsubsection SVGA graphic modes support
-
-QEMU emulates a Cirrus Logic GD5446 Video
-card. All Windows versions starting from Windows 95 should recognize
-and use this graphic card. For optimal performances, use 16 bit color
-depth in the guest and the host OS.
-
-If you are using Windows XP as guest OS and if you want to use high
-resolution modes which the Cirrus Logic BIOS does not support (i.e. >=
-1280x1024x16), then you should use the VESA VBE virtual graphic card
-(option @option{-std-vga}).
-
-@subsubsection CPU usage reduction
-
-Windows 9x does not correctly use the CPU HLT
-instruction. The result is that it takes host CPU cycles even when
-idle. You can install the utility from
-@url{https://web.archive.org/web/20060212132151/http://www.user.cityline.ru/~maxamn/amnhltm.zip}
-to solve this problem. Note that no such tool is needed for NT, 2000 or XP.
-
-@subsubsection Windows 2000 disk full problem
-
-Windows 2000 has a bug which gives a disk full problem during its
-installation. When installing it, use the @option{-win2k-hack} QEMU
-option to enable a specific workaround. After Windows 2000 is
-installed, you no longer need this option (this option slows down the
-IDE transfers).
-
-@subsubsection Windows 2000 shutdown
-
-Windows 2000 cannot automatically shutdown in QEMU although Windows 98
-can. It comes from the fact that Windows 2000 does not automatically
-use the APM driver provided by the BIOS.
-
-In order to correct that, do the following (thanks to Struan
-Bartlett): go to the Control Panel => Add/Remove Hardware & Next =>
-Add/Troubleshoot a device => Add a new device & Next => No, select the
-hardware from a list & Next => NT Apm/Legacy Support & Next => Next
-(again) a few times. Now the driver is installed and Windows 2000 now
-correctly instructs QEMU to shutdown at the appropriate moment.
-
-@subsubsection Share a directory between Unix and Windows
-
-See @ref{sec_invocation} about the help of the option
-@option{'-netdev user,smb=...'}.
-
-@subsubsection Windows XP security problem
-
-Some releases of Windows XP install correctly but give a security
-error when booting:
-@example
-A problem is preventing Windows from accurately checking the
-license for this computer. Error code: 0x800703e6.
-@end example
-
-The workaround is to install a service pack for XP after a boot in safe
-mode. Then reboot, and the problem should go away. Since there is no
-network while in safe mode, its recommended to download the full
-installation of SP1 or SP2 and transfer that via an ISO or using the
-vvfat block device ("-hdb fat:directory_which_holds_the_SP").
-
-@subsection MS-DOS and FreeDOS
-
-@subsubsection CPU usage reduction
-
-DOS does not correctly use the CPU HLT instruction. The result is that
-it takes host CPU cycles even when idle. You can install the utility from
-@url{https://web.archive.org/web/20051222085335/http://www.vmware.com/software/dosidle210.zip}
-to solve this problem.
-
 @node QEMU System emulator for non PC targets
 @chapter QEMU System emulator for non PC targets
 
-- 
2.20.1

[PATCH v3 01/33] qemu-doc: convert user-mode emulation to a separate Sphinx manual

[Peter Maydell]

From: Paolo Bonzini 

The final addition to the set of QEMU manuals is the user-mode emulation
manual, which right now is included in qemu-doc.texi.  Extract it and
convert it to rST, so that qemu-doc.texi covers only full system emulation.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-2-pbonz...@redhat.com
[PMM: Fix makefile conflicts; add user manual to
 index.rst and index.html.in; don't specify empty man_pages
 list; fixed a few comments to say 'user' rather than 'system']
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 Makefile|   8 +-
 docs/index.html.in  |   1 +
 docs/index.rst  |   1 +
 docs/user/conf.py   |  15 ++
 docs/user/index.rst |  16 +++
 docs/user/main.rst  | 295 ++
 qemu-doc.texi   | 337 
 7 files changed, 335 insertions(+), 338 deletions(-)
 create mode 100644 docs/user/conf.py
 create mode 100644 docs/user/index.rst
 create mode 100644 docs/user/main.rst

diff --git a/Makefile b/Makefile
index aa9cc0b5847..faa0ffa05b0 100644
--- a/Makefile
+++ b/Makefile
@@ -787,6 +787,7 @@ distclean: clean
$(call clean-manual,specs)
$(call clean-manual,system)
$(call clean-manual,tools)
+   $(call clean-manual,user)
for d in $(TARGET_DIRS); do \
rm -rf $$d || exit 1 ; \
 done
@@ -845,6 +846,7 @@ install-sphinxdocs: sphinxdocs
$(call install-manual,specs)
$(call install-manual,system)
$(call install-manual,tools)
+   $(call install-manual,user)
 
 install-doc: $(DOCS) install-sphinxdocs
$(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)"
@@ -1039,7 +1041,8 @@ sphinxdocs: $(MANUAL_BUILDDIR)/devel/index.html \
 $(MANUAL_BUILDDIR)/interop/index.html \
 $(MANUAL_BUILDDIR)/specs/index.html \
 $(MANUAL_BUILDDIR)/system/index.html \
-$(MANUAL_BUILDDIR)/tools/index.html
+$(MANUAL_BUILDDIR)/tools/index.html \
+$(MANUAL_BUILDDIR)/user/index.html
 
 # Canned command to build a single manual
 # Arguments: $1 = manual name, $2 = Sphinx builder ('html' or 'man')
@@ -1074,6 +1077,9 @@ $(MANUAL_BUILDDIR)/system/index.html: $(call 
manual-deps,system)
 $(MANUAL_BUILDDIR)/tools/index.html: $(call manual-deps,tools) 
$(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/docs/qemu-option-trace.rst.inc
$(call build-manual,tools,html)
 
+$(MANUAL_BUILDDIR)/user/index.html: $(call manual-deps,user)
+   $(call build-manual,user,html)
+
 $(call define-manpage-rule,interop,qemu-ga.8)
 
 $(call define-manpage-rule,system,qemu-block-drivers.7)
diff --git a/docs/index.html.in b/docs/index.html.in
index cf61b1cf448..a576ace8a27 100644
--- a/docs/index.html.in
+++ b/docs/index.html.in
@@ -14,6 +14,7 @@
 System Emulation Guest Hardware 
Specifications
 System Emulation User's 
Guide
 Tools Guide
+User Mode Emulation User's 
Guide
 
 
 
diff --git a/docs/index.rst b/docs/index.rst
index acd604fa8a9..376dab28850 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -15,3 +15,4 @@ Welcome to QEMU's documentation!
specs/index
system/index
tools/index
+   user/index
diff --git a/docs/user/conf.py b/docs/user/conf.py
new file mode 100644
index 000..4b09aedd454
--- /dev/null
+++ b/docs/user/conf.py
@@ -0,0 +1,15 @@
+# -*- coding: utf-8 -*-
+#
+# QEMU documentation build configuration file for the 'user' manual.
+#
+# This includes the top level conf file and then makes any necessary tweaks.
+import sys
+import os
+
+qemu_docdir = os.path.abspath("..")
+parent_config = os.path.join(qemu_docdir, "conf.py")
+exec(compile(open(parent_config, "rb").read(), parent_config, 'exec'))
+
+# This slightly misuses the 'description', but is the best way to get
+# the manual title to appear in the sidebar.
+html_theme_options['description'] = u'User Mode Emulation User''s Guide'
diff --git a/docs/user/index.rst b/docs/user/index.rst
new file mode 100644
index 000..e030dadf652
--- /dev/null
+++ b/docs/user/index.rst
@@ -0,0 +1,16 @@
+.. This is the top level page for the 'user' manual.
+
+
+QEMU User Mode Emulation User's Guide
+=
+
+This manual is the overall guide for users using QEMU
+for user-mode emulation.  In this mode, QEMU can launch
+processes compiled for one CPU on another CPU.
+
+Contents:
+
+.. toctree::
+   :maxdepth: 2
+
+   main
diff --git a/docs/user/main.rst b/docs/user/main.rst
new file mode 100644
index 000..ca69f7727d5
--- /dev/null
+++ b/docs/user/main.rst
@@ -0,0 +1,295 @@
+QEMU User space emulator
+
+
+Supported Operating Systems
+---
+
+The following OS are supported in user space emulation:
+
+-  Linux (referred as qemu-linux-user)
+
+-  BSD (referred as qemu-bsd-user)
+
+Features
+
+
+QEMU user space emulation has the following notable features:
+
+**System call translati

[PATCH v3 07/33] qemu-doc: move system requirements chapter inside PC section

[Peter Maydell]

From: Paolo Bonzini 

The system requirements documented in this chapter are limited to x86 KVM 
targets.
Clean them up and move them to the target section.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-8-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 qemu-doc.texi | 22 --
 1 file changed, 8 insertions(+), 14 deletions(-)

diff --git a/qemu-doc.texi b/qemu-doc.texi
index 88e84300e91..40fab523f35 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -38,7 +38,6 @@
 * Introduction::
 * QEMU System emulator::
 * QEMU System emulator targets::
-* System requirements::
 * Security::
 * Implementation notes::
 * Deprecated features::
@@ -190,6 +189,7 @@ various targets are mentioned in the following sections.
 @menu
 * pcsys_devices::  Peripherals
 * cpu_models_x86:: Supported CPU model configurations on x86 hosts
+* pcsys_req::  OS requirements
 @end menu
 
 @node pcsys_devices
@@ -267,6 +267,13 @@ CS4231A is the chip used in Windows Sound System and 
GUSMAX products
 @include docs/system/cpu-models-x86.texi
 @raisesections
 
+@node pcsys_req
+@subsection OS requirements
+
+On x86_64 hosts, the default set of CPU features enabled by the KVM accelerator
+require the host to be running Linux v4.5 or newer.  Red Hat Enterprise Linux
+7 is also supported, since the required functionality was backported.
+
 @node PowerPC System emulator
 @section PowerPC System emulator
 @cindex system emulation (PowerPC)
@@ -1038,19 +1045,6 @@ so should only be used with trusted guest OS.
 
 @c man end
 
-@node System requirements
-@chapter System requirements
-
-@section KVM kernel module
-
-On x86_64 hosts, the default set of CPU features enabled by the KVM accelerator
-require the host to be running Linux v4.5 or newer.
-
-The OpteronG[345] CPU models require KVM support for RDTSCP, which was
-added with Linux 4.5 which is supported by the major distros. And even
-if RHEL7 has kernel 3.10, KVM there has the required functionality there
-to make it close to a 4.5 or newer kernel.
-
 @include docs/security.texi
 
 @include qemu-tech.texi
-- 
2.20.1

[PATCH v3 04/33] qemu-doc: split CPU models doc between MIPS and x86 parts

[Peter Maydell]

From: Paolo Bonzini 

The MIPS CPU models end up in the middle of the PC documentation.  Move
them to a separate file so that they can be placed in the right section.

The man page still includes both x86 and MIPS content.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-5-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 Makefile  |  11 +-
 MAINTAINERS   |   3 +-
 docs/system/cpu-models-mips.texi  | 157 +
 .../cpu-models-x86.texi}  | 221 ++
 docs/system/qemu-cpu-models.texi  |  28 +++
 qemu-doc.texi |  12 +-
 6 files changed, 213 insertions(+), 219 deletions(-)
 create mode 100644 docs/system/cpu-models-mips.texi
 rename docs/{qemu-cpu-models.texi => system/cpu-models-x86.texi} (71%)
 create mode 100644 docs/system/qemu-cpu-models.texi

diff --git a/Makefile b/Makefile
index faa0ffa05b0..f75a7b51938 100644
--- a/Makefile
+++ b/Makefile
@@ -354,7 +354,7 @@ endif
 DOCS+=$(MANUAL_BUILDDIR)/system/qemu-block-drivers.7
 DOCS+=docs/interop/qemu-qmp-ref.html docs/interop/qemu-qmp-ref.txt 
docs/interop/qemu-qmp-ref.7
 DOCS+=docs/interop/qemu-ga-ref.html docs/interop/qemu-ga-ref.txt 
docs/interop/qemu-ga-ref.7
-DOCS+=docs/qemu-cpu-models.7
+DOCS+=docs/system/qemu-cpu-models.7
 DOCS+=$(MANUAL_BUILDDIR)/index.html
 ifdef CONFIG_VIRTFS
 DOCS+=$(MANUAL_BUILDDIR)/tools/virtfs-proxy-helper.1
@@ -780,7 +780,7 @@ distclean: clean
rm -f docs/interop/qemu-qmp-ref.txt docs/interop/qemu-ga-ref.txt
rm -f docs/interop/qemu-qmp-ref.pdf docs/interop/qemu-ga-ref.pdf
rm -f docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html
-   rm -f docs/qemu-cpu-models.7
+   rm -f docs/system/qemu-cpu-models.7
rm -rf .doctrees
$(call clean-manual,devel)
$(call clean-manual,interop)
@@ -861,7 +861,7 @@ ifdef CONFIG_POSIX
$(INSTALL_DIR) "$(DESTDIR)$(mandir)/man7"
$(INSTALL_DATA) docs/interop/qemu-qmp-ref.7 "$(DESTDIR)$(mandir)/man7"
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu-block-drivers.7 
"$(DESTDIR)$(mandir)/man7"
-   $(INSTALL_DATA) docs/qemu-cpu-models.7 "$(DESTDIR)$(mandir)/man7"
+   $(INSTALL_DATA) docs/system/qemu-cpu-models.7 "$(DESTDIR)$(mandir)/man7"
 ifeq ($(CONFIG_TOOLS),y)
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/tools/qemu-img.1 
"$(DESTDIR)$(mandir)/man1"
$(INSTALL_DIR) "$(DESTDIR)$(mandir)/man8"
@@ -,7 +,7 @@ docs/interop/qemu-ga-qapi.texi: 
qga/qapi-generated/qga-qapi-doc.texi
 
 qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi 
qemu-monitor-info.texi
 qemu.1: qemu-option-trace.texi
-docs/qemu-cpu-models.7: docs/qemu-cpu-models.texi
+docs/system/qemu-cpu-models.7: docs/system/qemu-cpu-models.texi 
docs/system/cpu-models-x86.texi docs/system/cpu-models-mips.texi
 
 html: qemu-doc.html docs/interop/qemu-qmp-ref.html 
docs/interop/qemu-ga-ref.html sphinxdocs
 info: qemu-doc.info docs/interop/qemu-qmp-ref.info 
docs/interop/qemu-ga-ref.info
@@ -1123,7 +1123,8 @@ qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
qemu-tech.texi qemu-option-trace.texi \
qemu-deprecated.texi qemu-monitor.texi \
qemu-monitor-info.texi \
-   docs/qemu-cpu-models.texi docs/security.texi
+   docs/system/cpu-models-x86.texi docs/system/cpu-models-mips.texi \
+   docs/security.texi
 
 docs/interop/qemu-ga-ref.dvi docs/interop/qemu-ga-ref.html \
 docs/interop/qemu-ga-ref.info docs/interop/qemu-ga-ref.pdf \
diff --git a/MAINTAINERS b/MAINTAINERS
index b66c46dcb9f..8c6d8f533ce 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -215,6 +215,7 @@ S: Maintained
 F: target/mips/
 F: default-configs/*mips*
 F: disas/*mips*
+F: docs/system/cpu-models-mips.texi
 F: hw/intc/mips_gic.c
 F: hw/mips/
 F: hw/misc/mips_*
@@ -319,7 +320,7 @@ F: tests/tcg/i386/
 F: tests/tcg/x86_64/
 F: hw/i386/
 F: disas/i386.c
-F: docs/qemu-cpu-models.texi
+F: docs/system/cpu-models-x86.texi
 T: git https://github.com/ehabkost/qemu.git x86-next
 
 Xtensa TCG CPUs
diff --git a/docs/system/cpu-models-mips.texi b/docs/system/cpu-models-mips.texi
new file mode 100644
index 000..6a0370cb693
--- /dev/null
+++ b/docs/system/cpu-models-mips.texi
@@ -0,0 +1,157 @@
+@node recommendations_cpu_models_MIPS
+@section Supported CPU model configurations on MIPS hosts
+
+QEMU supports variety of MIPS CPU models:
+
+@menu
+* cpu_models_MIPS32::   Supported CPU models for MIPS32 hosts
+* cpu_models_MIPS64::   Supported CPU models for MIPS64 hosts
+* cpu_models_nanoMIPS:: Supported CPU models for nanoMIPS hosts
+* preferred_cpu_models_MIPS::   Preferred CPU models for MIPS hosts
+@end menu
+
+@node cpu_models_MIPS32
+@subsection Supported CPU models for MIPS32 hosts
+
+The following CPU models are supported for use on MIPS32 hosts. Administrators 
/
+applications are recommended to us

[PATCH v3 06/33] qemu-doc: extract common system emulator documentation from the PC section

[Peter Maydell]

From: Paolo Bonzini 

Move the section on PC peripherals together with other targets.
While some x86-specific information remains in the main system
emulation chapter, it can be tackled more easily a section at a
time.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-7-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 docs/system/quickstart.texi |   2 +-
 qemu-doc.texi   | 102 
 2 files changed, 57 insertions(+), 47 deletions(-)

diff --git a/docs/system/quickstart.texi b/docs/system/quickstart.texi
index 8cd5b4bc6e5..ed7295de7a2 100644
--- a/docs/system/quickstart.texi
+++ b/docs/system/quickstart.texi
@@ -2,7 +2,7 @@
 @section Quick Start
 @cindex quick start
 
-Download and uncompress a hard disk image with Linux installed (e.g.
+Download and uncompress a PC hard disk image with Linux installed (e.g.
 @file{linux.img}) and type:
 
 @example
diff --git a/qemu-doc.texi b/qemu-doc.texi
index 33d24caf946..88e84300e91 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -36,8 +36,8 @@
 
 @menu
 * Introduction::
-* QEMU PC System emulator::
-* QEMU System emulator for non PC targets::
+* QEMU System emulator::
+* QEMU System emulator targets::
 * System requirements::
 * Security::
 * Implementation notes::
@@ -127,19 +127,16 @@ accelerator is required to use more than one host CPU for 
emulation.
 
 @end itemize
 
-
-@node QEMU PC System emulator
-@chapter QEMU PC System emulator
-@cindex system emulation (PC)
+@node QEMU System emulator
+@chapter QEMU System emulator
+@cindex system emulation
 
 @menu
-* pcsys_introduction:: Introduction
-* pcsys_quickstart::   Quick Start
+* pcsys_quickstart::   Quick start
 * sec_invocation:: Invocation
 * pcsys_keys:: Keys in the graphical frontends
 * mux_keys::   Keys in the character backend multiplexer
 * pcsys_monitor::  QEMU Monitor
-* cpu_models_x86:: Supported CPU model configurations on x86 hosts
 * disk_images::Disk Images
 * pcsys_network::  Network emulation
 * pcsys_usb::  USB emulation
@@ -150,13 +147,57 @@ accelerator is required to use more than one host CPU for 
emulation.
 * gdb_usage::  GDB usage
 @end menu
 
-@node pcsys_introduction
-@section Introduction
+@include docs/system/quickstart.texi
+@include docs/system/invocation.texi
+@include docs/system/keys.texi
+@include docs/system/mux-chardev.texi
+@include docs/system/monitor.texi
+@include docs/system/images.texi
+@include docs/system/net.texi
+@include docs/system/usb.texi
+@include docs/system/ivshmem.texi
+@include docs/system/linuxboot.texi
+@include docs/system/vnc-security.texi
+@include docs/system/tls.texi
+@include docs/system/gdb.texi
+
+@node QEMU System emulator targets
+@chapter QEMU System emulator targets
+@cindex system emulation (PC)
+
+QEMU is a generic emulator and it emulates many machines. Most of the
+options are similar for all machines. Specific information about the
+various targets are mentioned in the following sections.
+
+@menu
+* x86 (PC) System emulator::
+* PowerPC System emulator::
+* Sparc32 System emulator::
+* Sparc64 System emulator::
+* MIPS System emulator::
+* ARM System emulator::
+* ColdFire System emulator::
+* Cris System emulator::
+* Microblaze System emulator::
+* SH4 System emulator::
+* Xtensa System emulator::
+@end menu
+
+@node x86 (PC) System emulator
+@section x86 (PC) System emulator
+@cindex system emulation (PC)
+
+@menu
+* pcsys_devices::  Peripherals
+* cpu_models_x86:: Supported CPU model configurations on x86 hosts
+@end menu
+
+@node pcsys_devices
+@subsection Peripherals
 
 @c man begin DESCRIPTION
 
-The QEMU PC System emulator simulates the
-following peripherals:
+The QEMU PC System emulator simulates the following peripherals:
 
 @itemize @minus
 @item
@@ -222,40 +263,9 @@ CS4231A is the chip used in Windows Sound System and 
GUSMAX products
 
 @c man end
 
-@include docs/system/quickstart.texi
-@include docs/system/invocation.texi
-@include docs/system/keys.texi
-@include docs/system/mux-chardev.texi
-@include docs/system/monitor.texi
+@lowersections
 @include docs/system/cpu-models-x86.texi
-@include docs/system/images.texi
-@include docs/system/net.texi
-@include docs/system/usb.texi
-@include docs/system/ivshmem.texi
-@include docs/system/linuxboot.texi
-@include docs/system/vnc-security.texi
-@include docs/system/tls.texi
-@include docs/system/gdb.texi
-
-@node QEMU System emulator for non PC targets
-@chapter QEMU System emulator for non PC targets
-
-QEMU is a generic emulator and it emulates many non PC
-machines. Most of the options are similar to the PC emulator. The
-differences are mentioned in the following sections.
-
-@menu
-* PowerPC System emulator::
-* Sparc32 System emulator::
-* Sparc64 System emulator::
-* MIPS System emulator::
-* ARM System emulator::
-* ColdFire System emulator::
-* Cris System emulator::
-* Microblaze System emulator::
-* SH4 System emu

[PATCH v3 11/33] qemu-doc: move included files to docs/system

[Peter Maydell]

From: Paolo Bonzini 

Since qemu-doc.texi is mostly including files from docs/system,
move the existing include files there for consistency.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-12-pbonz...@redhat.com
[PMM: update MAINTAINERS line for qemu-option-trace.texi]
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 Makefile  | 8 
 MAINTAINERS   | 5 +++--
 qemu-deprecated.texi => docs/system/deprecated.texi   | 0
 .../system/qemu-option-trace.texi | 0
 docs/{ => system}/security.texi   | 0
 qemu-doc.texi | 4 ++--
 qemu-options.hx   | 2 +-
 7 files changed, 10 insertions(+), 9 deletions(-)
 rename qemu-deprecated.texi => docs/system/deprecated.texi (100%)
 rename qemu-option-trace.texi => docs/system/qemu-option-trace.texi (100%)
 rename docs/{ => system}/security.texi (100%)

diff --git a/Makefile b/Makefile
index 56382a45936..7e60a435426 100644
--- a/Makefile
+++ b/Makefile
@@ -1110,7 +1110,7 @@ docs/interop/qemu-ga-qapi.texi: 
qga/qapi-generated/qga-qapi-doc.texi
@cp -p $< $@
 
 qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi 
qemu-monitor-info.texi
-qemu.1: qemu-option-trace.texi
+qemu.1: docs/system/qemu-option-trace.texi
 docs/system/qemu-cpu-models.7: docs/system/qemu-cpu-models.texi 
docs/system/cpu-models-x86.texi docs/system/cpu-models-mips.texi
 
 html: qemu-doc.html docs/interop/qemu-qmp-ref.html 
docs/interop/qemu-ga-ref.html sphinxdocs
@@ -1120,8 +1120,7 @@ txt: qemu-doc.txt docs/interop/qemu-qmp-ref.txt 
docs/interop/qemu-ga-ref.txt
 
 qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
qemu-options.texi \
-   qemu-option-trace.texi \
-   qemu-deprecated.texi qemu-monitor.texi \
+   qemu-monitor.texi \
qemu-monitor-info.texi \
 docs/system/quickstart.texi \
 docs/system/invocation.texi \
@@ -1140,7 +1139,8 @@ qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
 docs/system/build-platforms.texi \
 docs/system/license.texi \
docs/system/cpu-models-x86.texi docs/system/cpu-models-mips.texi \
-   docs/security.texi
+   docs/system/deprecated.texi docs/system/qemu-option-trace.texi \
+   docs/system/security.texi
 
 docs/interop/qemu-ga-ref.dvi docs/interop/qemu-ga-ref.html \
 docs/interop/qemu-ga-ref.info docs/interop/qemu-ga-ref.pdf \
diff --git a/MAINTAINERS b/MAINTAINERS
index 8c6d8f533ce..74b025fb806 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2232,7 +2232,7 @@ M: Stefan Hajnoczi 
 S: Maintained
 F: trace/
 F: trace-events
-F: qemu-option-trace.texi
+F: docs/system/qemu-option-trace.texi
 F: scripts/tracetool.py
 F: scripts/tracetool/
 F: scripts/qemu-trace-stap*
@@ -2802,7 +2802,8 @@ F: contrib/gitdm/*
 
 Incompatible changes
 R: libvir-l...@redhat.com
-F: qemu-deprecated.texi
+F: docs/system/deprecated.texi
+F: docs/system/deprecated.rst
 
 Build System
 
diff --git a/qemu-deprecated.texi b/docs/system/deprecated.texi
similarity index 100%
rename from qemu-deprecated.texi
rename to docs/system/deprecated.texi
diff --git a/qemu-option-trace.texi b/docs/system/qemu-option-trace.texi
similarity index 100%
rename from qemu-option-trace.texi
rename to docs/system/qemu-option-trace.texi
diff --git a/docs/security.texi b/docs/system/security.texi
similarity index 100%
rename from docs/security.texi
rename to docs/system/security.texi
diff --git a/qemu-doc.texi b/qemu-doc.texi
index e4bff7edbee..d3e743719ab 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -189,9 +189,9 @@ various targets are mentioned in the following sections.
 @include docs/system/target-m68k.texi
 @include docs/system/target-xtensa.texi
 
-@include docs/security.texi
+@include docs/system/security.texi
 
-@include qemu-deprecated.texi
+@include docs/system/deprecated.texi
 
 @include docs/system/build-platforms.texi
 
diff --git a/qemu-options.hx b/qemu-options.hx
index ac315c1ac45..5fbfa2797cb 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4221,7 +4221,7 @@ HXCOMM This line is not accurate, as some sub-options are 
backend-specific but
 HXCOMM HX does not support conditional compilation of text.
 @item -trace [[enable=]@var{pattern}][,events=@var{file}][,file=@var{file}]
 @findex -trace
-@include qemu-option-trace.texi
+@include docs/system/qemu-option-trace.texi
 ETEXI
 DEF("plugin", HAS_ARG, QEMU_OPTION_plugin,
 "-plugin [file=][,arg=]\n"
-- 
2.20.1

[PATCH v3 12/33] qemu-doc: remove indices other than findex

[Peter Maydell]

From: Paolo Bonzini 

These indices are not well-maintained, and pandoc also chokes on the
directives.  Just nuke them.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-13-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 docs/system/keys.texi   | 10 
 docs/system/monitor.texi|  1 -
 docs/system/mux-chardev.texi|  7 --
 docs/system/quickstart.texi |  1 -
 docs/system/target-arm.texi |  1 -
 docs/system/target-i386.texi|  1 -
 docs/system/target-m68k.texi|  2 --
 docs/system/target-mips.texi|  2 --
 docs/system/target-ppc.texi |  1 -
 docs/system/target-sparc.texi   |  1 -
 docs/system/target-sparc64.texi |  1 -
 docs/system/target-xtensa.texi  |  1 -
 qemu-doc.texi   | 44 -
 13 files changed, 73 deletions(-)

diff --git a/docs/system/keys.texi b/docs/system/keys.texi
index 4c74b3bf4dd..c04daf54f23 100644
--- a/docs/system/keys.texi
+++ b/docs/system/keys.texi
@@ -10,23 +10,18 @@ then the modifier is Ctrl-Alt-Shift (instead of Ctrl-Alt) 
and if you use
 
 @table @key
 @item Ctrl-Alt-f
-@kindex Ctrl-Alt-f
 Toggle full screen
 
 @item Ctrl-Alt-+
-@kindex Ctrl-Alt-+
 Enlarge the screen
 
 @item Ctrl-Alt--
-@kindex Ctrl-Alt--
 Shrink the screen
 
 @item Ctrl-Alt-u
-@kindex Ctrl-Alt-u
 Restore the screen's un-scaled dimensions
 
 @item Ctrl-Alt-n
-@kindex Ctrl-Alt-n
 Switch to virtual console 'n'. Standard console mappings are:
 @table @emph
 @item 1
@@ -38,14 +33,9 @@ Serial port
 @end table
 
 @item Ctrl-Alt
-@kindex Ctrl-Alt
 Toggle mouse and keyboard grab.
 @end table
 
-@kindex Ctrl-Up
-@kindex Ctrl-Down
-@kindex Ctrl-PageUp
-@kindex Ctrl-PageDown
 In the virtual consoles, you can use @key{Ctrl-Up}, @key{Ctrl-Down},
 @key{Ctrl-PageUp} and @key{Ctrl-PageDown} to move in the back log.
 
diff --git a/docs/system/monitor.texi b/docs/system/monitor.texi
index c5b6a9b38e4..b41b144885d 100644
--- a/docs/system/monitor.texi
+++ b/docs/system/monitor.texi
@@ -1,6 +1,5 @@
 @node pcsys_monitor
 @section QEMU Monitor
-@cindex QEMU monitor
 
 The QEMU monitor is used to give complex commands to the QEMU
 emulator. You can use it to:
diff --git a/docs/system/mux-chardev.texi b/docs/system/mux-chardev.texi
index c9a2d14cb88..b21c2c56540 100644
--- a/docs/system/mux-chardev.texi
+++ b/docs/system/mux-chardev.texi
@@ -12,26 +12,19 @@ you're using the default.
 
 @table @key
 @item Ctrl-a h
-@kindex Ctrl-a h
 Print this help
 @item Ctrl-a x
-@kindex Ctrl-a x
 Exit emulator
 @item Ctrl-a s
-@kindex Ctrl-a s
 Save disk data back to file (if -snapshot)
 @item Ctrl-a t
-@kindex Ctrl-a t
 Toggle console timestamps
 @item Ctrl-a b
-@kindex Ctrl-a b
 Send break (magic sysrq in Linux)
 @item Ctrl-a c
-@kindex Ctrl-a c
 Rotate between the frontends connected to the multiplexer (usually
 this switches between the monitor and the console)
 @item Ctrl-a Ctrl-a
-@kindex Ctrl-a Ctrl-a
 Send the escape character to the frontend
 @end table
 @c man end
diff --git a/docs/system/quickstart.texi b/docs/system/quickstart.texi
index ed7295de7a2..baceaa96eb2 100644
--- a/docs/system/quickstart.texi
+++ b/docs/system/quickstart.texi
@@ -1,6 +1,5 @@
 @node pcsys_quickstart
 @section Quick Start
-@cindex quick start
 
 Download and uncompress a PC hard disk image with Linux installed (e.g.
 @file{linux.img}) and type:
diff --git a/docs/system/target-arm.texi b/docs/system/target-arm.texi
index 040d77b5e05..c56b5f6ebfe 100644
--- a/docs/system/target-arm.texi
+++ b/docs/system/target-arm.texi
@@ -1,6 +1,5 @@
 @node ARM System emulator
 @section ARM System emulator
-@cindex system emulation (ARM)
 
 Use the executable @file{qemu-system-arm} to simulate a ARM
 machine. The ARM Integrator/CP board is emulated with the following
diff --git a/docs/system/target-i386.texi b/docs/system/target-i386.texi
index edd23fa8df5..cc352b89a84 100644
--- a/docs/system/target-i386.texi
+++ b/docs/system/target-i386.texi
@@ -1,6 +1,5 @@
 @node x86 (PC) System emulator
 @section x86 (PC) System emulator
-@cindex system emulation (PC)
 
 @menu
 * pcsys_devices::  Peripherals
diff --git a/docs/system/target-m68k.texi b/docs/system/target-m68k.texi
index b5bc9df40ae..a77b19ea0f1 100644
--- a/docs/system/target-m68k.texi
+++ b/docs/system/target-m68k.texi
@@ -1,7 +1,5 @@
 @node ColdFire System emulator
 @section ColdFire System emulator
-@cindex system emulation (ColdFire)
-@cindex system emulation (M68K)
 
 Use the executable @file{qemu-system-m68k} to simulate a ColdFire machine.
 The emulator is able to boot a uClinux kernel.
diff --git a/docs/system/target-mips.texi b/docs/system/target-mips.texi
index f722c00912a..fe12ee94c73 100644
--- a/docs/system/target-mips.texi
+++ b/docs/system/target-mips.texi
@@ -1,6 +1,5 @@
 @node MIPS System emulator
 @section MIPS System emulator
-@cindex system emulation (MIPS)
 
 @menu
 * recommendations_cpu_models_MIPS:: Supported CPU model configurations on MIPS 
hosts
@@ -126,7 +125,

[PATCH v3 17/33] docs/system: convert managed startup to rST.

[Peter Maydell]

Fix one typo in the process and format more option and
command names as literal text, but make no significant
changes to the content.

Signed-off-by: Peter Maydell 
Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-17-pbonz...@redhat.com
---
 docs/system/index.rst   |  2 +-
 docs/system/managed-startup.rst | 35 +
 2 files changed, 36 insertions(+), 1 deletion(-)
 create mode 100644 docs/system/managed-startup.rst

diff --git a/docs/system/index.rst b/docs/system/index.rst
index 5034f903407..d655344fffd 100644
--- a/docs/system/index.rst
+++ b/docs/system/index.rst
@@ -13,6 +13,6 @@ Contents:
 
 .. toctree::
:maxdepth: 2
-
+   managed-startup
security
vfio-ap
diff --git a/docs/system/managed-startup.rst b/docs/system/managed-startup.rst
new file mode 100644
index 000..9bcf98ea790
--- /dev/null
+++ b/docs/system/managed-startup.rst
@@ -0,0 +1,35 @@
+Managed start up options
+
+
+In system mode emulation, it's possible to create a VM in a paused
+state using the ``-S`` command line option. In this state the machine
+is completely initialized according to command line options and ready
+to execute VM code but VCPU threads are not executing any code. The VM
+state in this paused state depends on the way QEMU was started. It
+could be in:
+
+- initial state (after reset/power on state)
+- with direct kernel loading, the initial state could be amended to execute
+  code loaded by QEMU in the VM's RAM and with incoming migration
+- with incoming migration, initial state will be amended with the migrated
+  machine state after migration completes
+
+This paused state is typically used by users to query machine state and/or
+additionally configure the machine (by hotplugging devices) in runtime before
+allowing VM code to run.
+
+However, at the ``-S`` pause point, it's impossible to configure options
+that affect initial VM creation (like: ``-smp``/``-m``/``-numa`` ...) or
+cold plug devices. The experimental ``--preconfig`` command line option
+allows pausing QEMU before the initial VM creation, in a "preconfig" state,
+where additional queries and configuration can be performed via QMP
+before moving on to the resulting configuration startup. In the
+preconfig state, QEMU only allows a limited set of commands over the
+QMP monitor, where the commands do not depend on an initialized
+machine, including but not limited to:
+
+- ``qmp_capabilities``
+- ``query-qmp-schema``
+- ``query-commands``
+- ``query-status``
+- ``x-exit-preconfig``
-- 
2.20.1

[PATCH v3 10/33] qemu-doc: move qemu-tech.texi into main section

[Peter Maydell]

From: Paolo Bonzini 

The only remaining content in qemu-tech.texi is a few paragraphs
about managed start up options.  Move them in the main section
about full system emulation.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-11-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 Makefile   | 2 +-
 qemu-tech.texi => docs/system/managed-startup.texi | 9 +
 qemu-doc.texi  | 5 ++---
 3 files changed, 4 insertions(+), 12 deletions(-)
 rename qemu-tech.texi => docs/system/managed-startup.texi (92%)

diff --git a/Makefile b/Makefile
index 9790a0fd15b..56382a45936 100644
--- a/Makefile
+++ b/Makefile
@@ -1120,7 +1120,7 @@ txt: qemu-doc.txt docs/interop/qemu-qmp-ref.txt 
docs/interop/qemu-ga-ref.txt
 
 qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
qemu-options.texi \
-   qemu-tech.texi qemu-option-trace.texi \
+   qemu-option-trace.texi \
qemu-deprecated.texi qemu-monitor.texi \
qemu-monitor-info.texi \
 docs/system/quickstart.texi \
diff --git a/qemu-tech.texi b/docs/system/managed-startup.texi
similarity index 92%
rename from qemu-tech.texi
rename to docs/system/managed-startup.texi
index 35da6a40af1..ec168095cc4 100644
--- a/qemu-tech.texi
+++ b/docs/system/managed-startup.texi
@@ -1,11 +1,4 @@
-@node Implementation notes
-@appendix Implementation notes
-
-@menu
-* Managed start up options::
-@end menu
-
-@node Managed start up options
+@node managed_startup
 @section Managed start up options
 
 In system mode emulation, it's possible to create a VM in a paused state using
diff --git a/qemu-doc.texi b/qemu-doc.texi
index f702dce4557..e4bff7edbee 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -39,7 +39,6 @@
 * QEMU System emulator::
 * QEMU System emulator targets::
 * Security::
-* Implementation notes::
 * Deprecated features::
 * Recently removed features::
 * Supported build platforms::
@@ -144,6 +143,7 @@ accelerator is required to use more than one host CPU for 
emulation.
 * vnc_security::   VNC security
 * network_tls::TLS setup for network services
 * gdb_usage::  GDB usage
+* managed_startup::Managed startup options
 @end menu
 
 @include docs/system/quickstart.texi
@@ -159,6 +159,7 @@ accelerator is required to use more than one host CPU for 
emulation.
 @include docs/system/vnc-security.texi
 @include docs/system/tls.texi
 @include docs/system/gdb.texi
+@include docs/system/managed-startup.texi
 
 @node QEMU System emulator targets
 @chapter QEMU System emulator targets
@@ -190,8 +191,6 @@ various targets are mentioned in the following sections.
 
 @include docs/security.texi
 
-@include qemu-tech.texi
-
 @include qemu-deprecated.texi
 
 @include docs/system/build-platforms.texi
-- 
2.20.1

[PATCH v3 09/33] qemu-doc: Remove the "CPU emulation" part of the "Implementation notes"

[Peter Maydell]

The "CPU emulation" part of the "Implementation notes" in
qemu-tech.texi looks like it is documenting what features of various
CPUs we do or don't emulate.  However:
 * it covers only six of our 21 guest architectures
 * the last time anybody updated it for actual content was in
   2011/2012 for Xtensa; the content for the other five
   architectures is even older, being from 2008 or before!

What we have is out of date, misleading and incomplete.
Just delete this part of the document rather than trying to
convert it to rST.

(It would be nice eventually to have documentation of the
scope and limitations of our emulation; but we will want to
separate out the generic "system emulation" information from
the parts that are specific to linux-user anyway, as they will
be in different manuals.)

Signed-off-by: Peter Maydell 
Message-id: 20200226113034.6741-10-pbonz...@redhat.com
Message-Id: <20200225154121.21116-3-peter.mayd...@linaro.org>
Signed-off-by: Paolo Bonzini 
---
 qemu-tech.texi | 153 -
 1 file changed, 153 deletions(-)

diff --git a/qemu-tech.texi b/qemu-tech.texi
index 0380de77b62..35da6a40af1 100644
--- a/qemu-tech.texi
+++ b/qemu-tech.texi
@@ -2,162 +2,9 @@
 @appendix Implementation notes
 
 @menu
-* CPU emulation::
 * Managed start up options::
 @end menu
 
-@node CPU emulation
-@section CPU emulation
-
-@menu
-* x86:: x86 and x86-64 emulation
-* ARM:: ARM emulation
-* MIPS::MIPS emulation
-* PPC:: PowerPC emulation
-* SPARC::   Sparc32 and Sparc64 emulation
-* Xtensa::  Xtensa emulation
-@end menu
-
-@node x86
-@subsection x86 and x86-64 emulation
-
-QEMU x86 target features:
-
-@itemize
-
-@item The virtual x86 CPU supports 16 bit and 32 bit addressing with 
segmentation.
-LDT/GDT and IDT are emulated. VM86 mode is also supported to run
-DOSEMU. There is some support for MMX/3DNow!, SSE, SSE2, SSE3, SSSE3,
-and SSE4 as well as x86-64 SVM.
-
-@item Support of host page sizes bigger than 4KB in user mode emulation.
-
-@item QEMU can emulate itself on x86.
-
-@item An extensive Linux x86 CPU test program is included 
@file{tests/test-i386}.
-It can be used to test other x86 virtual CPUs.
-
-@end itemize
-
-Current QEMU limitations:
-
-@itemize
-
-@item Limited x86-64 support.
-
-@item IPC syscalls are missing.
-
-@item The x86 segment limits and access rights are not tested at every
-memory access (yet). Hopefully, very few OSes seem to rely on that for
-normal use.
-
-@end itemize
-
-@node ARM
-@subsection ARM emulation
-
-@itemize
-
-@item Full ARM 7 user emulation.
-
-@item NWFPE FPU support included in user Linux emulation.
-
-@item Can run most ARM Linux binaries.
-
-@end itemize
-
-@node MIPS
-@subsection MIPS emulation
-
-@itemize
-
-@item The system emulation allows full MIPS32/MIPS64 Release 2 emulation,
-including privileged instructions, FPU and MMU, in both little and big
-endian modes.
-
-@item The Linux userland emulation can run many 32 bit MIPS Linux binaries.
-
-@end itemize
-
-Current QEMU limitations:
-
-@itemize
-
-@item Self-modifying code is not always handled correctly.
-
-@item 64 bit userland emulation is not implemented.
-
-@item The system emulation is not complete enough to run real firmware.
-
-@item The watchpoint debug facility is not implemented.
-
-@end itemize
-
-@node PPC
-@subsection PowerPC emulation
-
-@itemize
-
-@item Full PowerPC 32 bit emulation, including privileged instructions,
-FPU and MMU.
-
-@item Can run most PowerPC Linux binaries.
-
-@end itemize
-
-@node SPARC
-@subsection Sparc32 and Sparc64 emulation
-
-@itemize
-
-@item Full SPARC V8 emulation, including privileged
-instructions, FPU and MMU. SPARC V9 emulation includes most privileged
-and VIS instructions, FPU and I/D MMU. Alignment is fully enforced.
-
-@item Can run most 32-bit SPARC Linux binaries, SPARC32PLUS Linux binaries and
-some 64-bit SPARC Linux binaries.
-
-@end itemize
-
-Current QEMU limitations:
-
-@itemize
-
-@item IPC syscalls are missing.
-
-@item Floating point exception support is buggy.
-
-@item Atomic instructions are not correctly implemented.
-
-@item There are still some problems with Sparc64 emulators.
-
-@end itemize
-
-@node Xtensa
-@subsection Xtensa emulation
-
-@itemize
-
-@item Core Xtensa ISA emulation, including most options: code density,
-loop, extended L32R, 16- and 32-bit multiplication, 32-bit division,
-MAC16, miscellaneous operations, boolean, FP coprocessor, coprocessor
-context, debug, multiprocessor synchronization,
-conditional store, exceptions, relocatable vectors, unaligned exception,
-interrupts (including high priority and timer), hardware alignment,
-region protection, region translation, MMU, windowed registers, thread
-pointer, processor ID.
-
-@item Not implemented options: data/instruction cache (including cache
-prefetch and locking), XLMI, processor interface. Also options not
-covered by the core ISA (e.g. FLIX, wide branches) are not implemented.
-
-@item Can run most Xte

[PATCH v3 14/33] docs: Create defs.rst.inc as a place to define substitutions

[Peter Maydell]

Rather than accumulating generally useful rST substitution
definitions in individual rST files, create a defs.rst.inc where we
can define these.  To start with it has the |qemu_system| definition
from qemu-block-drivers.rst.

Add a comment noting a pitfall where putting literal markup in the
definition of |qemu_system| makes it misrender manpage output; this
means the point-of-use must handle the literal markup (which is
almost always done by having it inside a parsed-literal block).

Signed-off-by: Peter Maydell 
---
 Makefile   |  1 +
 docs/conf.py   |  6 ++
 docs/defs.rst.inc  | 11 +++
 docs/system/qemu-block-drivers.rst |  2 --
 4 files changed, 18 insertions(+), 2 deletions(-)
 create mode 100644 docs/defs.rst.inc

diff --git a/Makefile b/Makefile
index 7e60a435426..26bbd334438 100644
--- a/Makefile
+++ b/Makefile
@@ -1052,6 +1052,7 @@ sphinxdocs: $(MANUAL_BUILDDIR)/devel/index.html \
 build-manual = $(call quiet-command,CONFDIR="$(qemu_confdir)" $(SPHINX_BUILD) 
$(if $(V),,-q) -W -b $2 -D version=$(VERSION) -D release="$(FULL_VERSION)" -d 
.doctrees/$1-$2 $(SRC_PATH)/docs/$1 $(MANUAL_BUILDDIR)/$1 
,"SPHINX","$(MANUAL_BUILDDIR)/$1")
 # We assume all RST files in the manual's directory are used in it
 manual-deps = $(wildcard $(SRC_PATH)/docs/$1/*.rst) \
+  $(SRC_PATH)/docs/defs.rst.inc \
   $(SRC_PATH)/docs/$1/conf.py $(SRC_PATH)/docs/conf.py
 # Macro to write out the rule and dependencies for building manpages
 # Usage: $(call define-manpage-rule,manualname,manpage1 
manpage2...[,extradeps])
diff --git a/docs/conf.py b/docs/conf.py
index 7588bf192ee..960043cb860 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -132,6 +132,12 @@ suppress_warnings = ["ref.option"]
 # style document building; our Makefile always sets the variable.
 confdir = os.getenv('CONFDIR', "/etc/qemu")
 rst_epilog = ".. |CONFDIR| replace:: ``" + confdir + "``\n"
+# We slurp in the defs.rst.inc and literally include it into rst_epilog,
+# because Sphinx's include:: directive doesn't work with absolute paths
+# and there isn't any one single relative path that will work for all
+# documents and for both via-make and direct sphinx-build invocation.
+with open(os.path.join(qemu_docdir, 'defs.rst.inc')) as f:
+rst_epilog += f.read()
 
 # -- Options for HTML output --
 
diff --git a/docs/defs.rst.inc b/docs/defs.rst.inc
new file mode 100644
index 000..ae80d2f35d8
--- /dev/null
+++ b/docs/defs.rst.inc
@@ -0,0 +1,11 @@
+..
+   Generally useful rST substitution definitions. This is included for
+   all rST files as part of the epilogue by docs/conf.py.  conf.py
+   also defines some dynamically generated substitutions like CONFDIR.
+
+   Note that |qemu_system| is intended to be used inside a parsed-literal
+   block: the definition must not include extra literal formatting with
+   ``..``: this works in the HTML output but the manpages will end up
+   misrendered with following normal text incorrectly in boldface.
+
+.. |qemu_system| replace:: qemu-system-x86_64
diff --git a/docs/system/qemu-block-drivers.rst 
b/docs/system/qemu-block-drivers.rst
index 7ca890ea23a..bd99d4fa8eb 100644
--- a/docs/system/qemu-block-drivers.rst
+++ b/docs/system/qemu-block-drivers.rst
@@ -3,8 +3,6 @@
 QEMU block drivers reference
 
 
-.. |qemu_system| replace:: qemu-system-x86_64
-
 Synopsis
 
 
-- 
2.20.1

[PATCH v3 16/33] docs/system: Convert security.texi to rST format

[Peter Maydell]

security.texi is included from qemu-doc.texi but is not used
in the qemu.1 manpage. So we can do a straightforward conversion
of the contents, which go into the system manual.

Signed-off-by: Peter Maydell 
Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-16-pbonz...@redhat.com
---
 docs/system/index.rst|   1 +
 docs/system/security.rst | 173 +++
 2 files changed, 174 insertions(+)
 create mode 100644 docs/system/security.rst

diff --git a/docs/system/index.rst b/docs/system/index.rst
index fc774a18b54..5034f903407 100644
--- a/docs/system/index.rst
+++ b/docs/system/index.rst
@@ -14,4 +14,5 @@ Contents:
 .. toctree::
:maxdepth: 2
 
+   security
vfio-ap
diff --git a/docs/system/security.rst b/docs/system/security.rst
new file mode 100644
index 000..f2092c8768b
--- /dev/null
+++ b/docs/system/security.rst
@@ -0,0 +1,173 @@
+Security
+
+
+Overview
+
+
+This chapter explains the security requirements that QEMU is designed to meet
+and principles for securely deploying QEMU.
+
+Security Requirements
+-
+
+QEMU supports many different use cases, some of which have stricter security
+requirements than others.  The community has agreed on the overall security
+requirements that users may depend on.  These requirements define what is
+considered supported from a security perspective.
+
+Virtualization Use Case
+'''
+
+The virtualization use case covers cloud and virtual private server (VPS)
+hosting, as well as traditional data center and desktop virtualization.  These
+use cases rely on hardware virtualization extensions to execute guest code
+safely on the physical CPU at close-to-native speed.
+
+The following entities are untrusted, meaning that they may be buggy or
+malicious:
+
+- Guest
+- User-facing interfaces (e.g. VNC, SPICE, WebSocket)
+- Network protocols (e.g. NBD, live migration)
+- User-supplied files (e.g. disk images, kernels, device trees)
+- Passthrough devices (e.g. PCI, USB)
+
+Bugs affecting these entities are evaluated on whether they can cause damage in
+real-world use cases and treated as security bugs if this is the case.
+
+Non-virtualization Use Case
+'''
+
+The non-virtualization use case covers emulation using the Tiny Code Generator
+(TCG).  In principle the TCG and device emulation code used in conjunction with
+the non-virtualization use case should meet the same security requirements as
+the virtualization use case.  However, for historical reasons much of the
+non-virtualization use case code was not written with these security
+requirements in mind.
+
+Bugs affecting the non-virtualization use case are not considered security
+bugs at this time.  Users with non-virtualization use cases must not rely on
+QEMU to provide guest isolation or any security guarantees.
+
+Architecture
+
+
+This section describes the design principles that ensure the security
+requirements are met.
+
+Guest Isolation
+'''
+
+Guest isolation is the confinement of guest code to the virtual machine.  When
+guest code gains control of execution on the host this is called escaping the
+virtual machine.  Isolation also includes resource limits such as throttling of
+CPU, memory, disk, or network.  Guests must be unable to exceed their resource
+limits.
+
+QEMU presents an attack surface to the guest in the form of emulated devices.
+The guest must not be able to gain control of QEMU.  Bugs in emulated devices
+could allow malicious guests to gain code execution in QEMU.  At this point the
+guest has escaped the virtual machine and is able to act in the context of the
+QEMU process on the host.
+
+Guests often interact with other guests and share resources with them.  A
+malicious guest must not gain control of other guests or access their data.
+Disk image files and network traffic must be protected from other guests unless
+explicitly shared between them by the user.
+
+Principle of Least Privilege
+
+
+The principle of least privilege states that each component only has access to
+the privileges necessary for its function.  In the case of QEMU this means that
+each process only has access to resources belonging to the guest.
+
+The QEMU process should not have access to any resources that are inaccessible
+to the guest.  This way the guest does not gain anything by escaping into the
+QEMU process since it already has access to those same resources from within
+the guest.
+
+Following the principle of least privilege immediately fulfills guest isolation
+requirements.  For example, guest A only has access to its own disk image file
+``a.img`` and not guest B's disk image file ``b.img``.
+
+In reality certain resources are inaccessible to the guest but must be
+available to QEMU to perform its function.  For example, host system calls are
+necessary for QEMU but are not exposed to guests.  A guest that escapes i

[PATCH v3 15/33] docs/system: Convert qemu-cpu-models.texi to rST

[Peter Maydell]

From: Kashyap Chamarthy 

This doc was originally written by Daniel P. Berrangé
, introduced via commit[1]: 2544e9e4aa (docs: add
guidance on configuring CPU models for x86, 2018-06-27).

In this patch:

  - 1-1 conversion of Texinfo to rST, besides a couple of minor
tweaks that are too trivial to mention.   (Thanks to Stephen
Finucane on IRC for the suggestion to use rST "definition lists"
instead of bullets in some places.)

Further modifications will be done via a separate patch.

  - rST and related infra changes: manual page generation, Makefile
fixes, clean up references to qemu-cpu-models.texi, update year in
the copyright notice, etc.

[1] https://git.qemu.org/?p=qemu.git;a=commit;h=2544e9e4aa

As part of the conversion, we use a more generic 'author' attribution
for the manpage than we previously had, as agreed with the original
author Dan Berrange.

Signed-off-by: Kashyap Chamarthy 
Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-15-pbonz...@redhat.com
[Move macros to defs.rst.inc, split in x86 and MIPS parts,
 make qemu-cpu-models.rst a standalone document. - Paolo]
Signed-off-by: Paolo Bonzini 
[PMM: Move defs.rst.inc setup to its own commit;
 fix minor issues with MAINTAINERS file updates;
 drop copyright date change; keep capitalization of
 "QEMU Project developers" consistent with other uses;
 minor Makefile fixups]
Signed-off-by: Peter Maydell 
---
 Makefile|   8 +-
 MAINTAINERS |   2 +
 docs/system/conf.py |   6 +-
 docs/system/cpu-models-mips.rst.inc | 105 
 docs/system/cpu-models-x86.rst.inc  | 365 
 docs/system/qemu-cpu-models.rst |  20 ++
 docs/system/qemu-cpu-models.texi|  28 ---
 7 files changed, 500 insertions(+), 34 deletions(-)
 create mode 100644 docs/system/cpu-models-mips.rst.inc
 create mode 100644 docs/system/cpu-models-x86.rst.inc
 create mode 100644 docs/system/qemu-cpu-models.rst
 delete mode 100644 docs/system/qemu-cpu-models.texi

diff --git a/Makefile b/Makefile
index 26bbd334438..f8642cd28a1 100644
--- a/Makefile
+++ b/Makefile
@@ -354,7 +354,7 @@ endif
 DOCS+=$(MANUAL_BUILDDIR)/system/qemu-block-drivers.7
 DOCS+=docs/interop/qemu-qmp-ref.html docs/interop/qemu-qmp-ref.txt 
docs/interop/qemu-qmp-ref.7
 DOCS+=docs/interop/qemu-ga-ref.html docs/interop/qemu-ga-ref.txt 
docs/interop/qemu-ga-ref.7
-DOCS+=docs/system/qemu-cpu-models.7
+DOCS+=$(MANUAL_BUILDDIR)/system/qemu-cpu-models.7
 DOCS+=$(MANUAL_BUILDDIR)/index.html
 ifdef CONFIG_VIRTFS
 DOCS+=$(MANUAL_BUILDDIR)/tools/virtfs-proxy-helper.1
@@ -780,7 +780,6 @@ distclean: clean
rm -f docs/interop/qemu-qmp-ref.txt docs/interop/qemu-ga-ref.txt
rm -f docs/interop/qemu-qmp-ref.pdf docs/interop/qemu-ga-ref.pdf
rm -f docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html
-   rm -f docs/system/qemu-cpu-models.7
rm -rf .doctrees
$(call clean-manual,devel)
$(call clean-manual,interop)
@@ -861,7 +860,7 @@ ifdef CONFIG_POSIX
$(INSTALL_DIR) "$(DESTDIR)$(mandir)/man7"
$(INSTALL_DATA) docs/interop/qemu-qmp-ref.7 "$(DESTDIR)$(mandir)/man7"
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu-block-drivers.7 
"$(DESTDIR)$(mandir)/man7"
-   $(INSTALL_DATA) docs/system/qemu-cpu-models.7 "$(DESTDIR)$(mandir)/man7"
+   $(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu-cpu-models.7 
"$(DESTDIR)$(mandir)/man7"
 ifeq ($(CONFIG_TOOLS),y)
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/tools/qemu-img.1 
"$(DESTDIR)$(mandir)/man1"
$(INSTALL_DIR) "$(DESTDIR)$(mandir)/man8"
@@ -1083,7 +1082,7 @@ $(MANUAL_BUILDDIR)/user/index.html: $(call 
manual-deps,user)
 
 $(call define-manpage-rule,interop,qemu-ga.8)
 
-$(call define-manpage-rule,system,qemu-block-drivers.7)
+$(call define-manpage-rule,system,qemu-block-drivers.7 qemu-cpu-models.7)
 
 $(call define-manpage-rule,tools,\
qemu-img.1 qemu-nbd.8 qemu-trace-stap.1\
@@ -1112,7 +,6 @@ docs/interop/qemu-ga-qapi.texi: 
qga/qapi-generated/qga-qapi-doc.texi
 
 qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi 
qemu-monitor-info.texi
 qemu.1: docs/system/qemu-option-trace.texi
-docs/system/qemu-cpu-models.7: docs/system/qemu-cpu-models.texi 
docs/system/cpu-models-x86.texi docs/system/cpu-models-mips.texi
 
 html: qemu-doc.html docs/interop/qemu-qmp-ref.html 
docs/interop/qemu-ga-ref.html sphinxdocs
 info: qemu-doc.info docs/interop/qemu-qmp-ref.info 
docs/interop/qemu-ga-ref.info
diff --git a/MAINTAINERS b/MAINTAINERS
index 74b025fb806..1712eb90851 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -216,6 +216,7 @@ F: target/mips/
 F: default-configs/*mips*
 F: disas/*mips*
 F: docs/system/cpu-models-mips.texi
+F: docs/system/cpu-models-mips.rst.inc
 F: hw/intc/mips_gic.c
 F: hw/mips/
 F: hw/misc/mips_*
@@ -321,6 +322,7 @@ F: tests/tcg/x86_64/
 F: hw/i386/
 F: disas/i386.c
 F: docs/system/cpu-models-x86.texi
+F: docs/system/cpu-models-x86.rst.inc
 T: git https:/

[PATCH v3 13/33] docs/system: put qemu-block-drivers body in an included file

[Peter Maydell]

From: Paolo Bonzini 

This removes the "only" directives, and lets us use the conventional
"DESCRIPTION" section in the manpage.

This temporarily drops the qemu-block-drivers documentation
from the system manual, but it will be put back (in the
right place in the toctree) in a later commit.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-14-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
[PMM: Added commit message note about temporarily losing
qemu-block-drivers from the system manual]
Signed-off-by: Peter Maydell 
---
 docs/system/index.rst |   1 -
 docs/system/qemu-block-drivers.rst| 987 +-
 ...drivers.rst => qemu-block-drivers.rst.inc} |  59 +-
 3 files changed, 26 insertions(+), 1021 deletions(-)
 copy docs/system/{qemu-block-drivers.rst => qemu-block-drivers.rst.inc} (96%)

diff --git a/docs/system/index.rst b/docs/system/index.rst
index 1a4b2c82ace..fc774a18b54 100644
--- a/docs/system/index.rst
+++ b/docs/system/index.rst
@@ -14,5 +14,4 @@ Contents:
 .. toctree::
:maxdepth: 2
 
-   qemu-block-drivers
vfio-ap
diff --git a/docs/system/qemu-block-drivers.rst 
b/docs/system/qemu-block-drivers.rst
index 388adbefbf4..7ca890ea23a 100644
--- a/docs/system/qemu-block-drivers.rst
+++ b/docs/system/qemu-block-drivers.rst
@@ -1,985 +1,22 @@
+:orphan:
+
 QEMU block drivers reference
 
 
 .. |qemu_system| replace:: qemu-system-x86_64
 
-..
-   We put the 'Synopsis' and 'See also' sections into the manpage, but not
-   the HTML. This makes the HTML docs read better and means the ToC in
-   the index has a more useful set of entries. Ideally, the section
-   headings 'Disk image file formats' would be top-level headings for
-   the HTML, but sub-headings of the conventional manpage 'Description'
-   header for the manpage. Unfortunately, due to deficiencies in
-   the Sphinx 'only' directive, this isn't possible: they must be headers
-   at the same level as 'Synopsis' and 'See also', otherwise Sphinx's
-   identification of which header underline style is which gets confused.
+Synopsis
+
 
-.. only:: man
+QEMU block driver reference manual
 
-  Synopsis
-  
+Description
+---
 
-  QEMU block driver reference manual
+.. include:: qemu-block-drivers.rst.inc
 
-Disk image file formats

+See also
+
 
-QEMU supports many image file formats that can be used with VMs as well as with
-any of the tools (like ``qemu-img``). This includes the preferred formats
-raw and qcow2 as well as formats that are supported for compatibility with
-older QEMU versions or other hypervisors.
-
-Depending on the image format, different options can be passed to
-``qemu-img create`` and ``qemu-img convert`` using the ``-o`` option.
-This section describes each format and the options that are supported for it.
-
-.. program:: image-formats
-.. option:: raw
-
-  Raw disk image format. This format has the advantage of
-  being simple and easily exportable to all other emulators. If your
-  file system supports *holes* (for example in ext2 or ext3 on
-  Linux or NTFS on Windows), then only the written sectors will reserve
-  space. Use ``qemu-img info`` to know the real size used by the
-  image or ``ls -ls`` on Unix/Linux.
-
-  Supported options:
-
-  .. program:: raw
-  .. option:: preallocation
-
-Preallocation mode (allowed values: ``off``, ``falloc``,
-``full``). ``falloc`` mode preallocates space for image by
-calling ``posix_fallocate()``. ``full`` mode preallocates space
-for image by writing data to underlying storage. This data may or
-may not be zero, depending on the storage location.
-
-.. program:: image-formats
-.. option:: qcow2
-
-  QEMU image format, the most versatile format. Use it to have smaller
-  images (useful if your filesystem does not supports holes, for example
-  on Windows), zlib based compression and support of multiple VM
-  snapshots.
-
-  Supported options:
-
-  .. program:: qcow2
-  .. option:: compat
-
-Determines the qcow2 version to use. ``compat=0.10`` uses the
-traditional image format that can be read by any QEMU since 0.10.
-``compat=1.1`` enables image format extensions that only QEMU 1.1 and
-newer understand (this is the default). Amongst others, this includes
-zero clusters, which allow efficient copy-on-read for sparse images.
-
-  .. option:: backing_file
-
-File name of a base image (see ``create`` subcommand)
-
-  .. option:: backing_fmt
-
-Image format of the base image
-
-  .. option:: encryption
-
-This option is deprecated and equivalent to ``encrypt.format=aes``
-
-  .. option:: encrypt.format
-
-If this is set to ``luks``, it requests that the qcow2 payload (not
-qcow2 header) be encrypted using the LUKS format. The passphrase to
-use to unlock the LUKS key slot is given by the ``encrypt.key-secret``
-parameter. LUKS encryption parameters can be tuned with the other
-

[PATCH v3 18/33] docs/system: convert the documentation of deprecated features to rST.

[Peter Maydell]

We put the whole of this document into the system manual, though
technically a few parts of it apply to qemu-img or qemu-nbd which are
otherwise documented in tools/.

We only make formatting fixes, except for one use of 'appendix' which
we change to 'section' because this isn't an appendix in the Sphinx
manual.

Signed-off-by: Peter Maydell 
Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-18-pbonz...@redhat.com
---
 docs/system/deprecated.rst | 446 +
 docs/system/index.rst  |   1 +
 2 files changed, 447 insertions(+)
 create mode 100644 docs/system/deprecated.rst

diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
new file mode 100644
index 000..1eaa559079b
--- /dev/null
+++ b/docs/system/deprecated.rst
@@ -0,0 +1,446 @@
+Deprecated features
+===
+
+In general features are intended to be supported indefinitely once
+introduced into QEMU. In the event that a feature needs to be removed,
+it will be listed in this section. The feature will remain functional
+for 2 releases prior to actual removal. Deprecated features may also
+generate warnings on the console when QEMU starts up, or if activated
+via a monitor command, however, this is not a mandatory requirement.
+
+Prior to the 2.10.0 release there was no official policy on how
+long features would be deprecated prior to their removal, nor
+any documented list of which features were deprecated. Thus
+any features deprecated prior to 2.10.0 will be treated as if
+they were first deprecated in the 2.10.0 release.
+
+What follows is a list of all features currently marked as
+deprecated.
+
+System emulator command line arguments
+--
+
+``-machine enforce-config-section=on|off`` (since 3.1)
+''
+
+The ``enforce-config-section`` parameter is replaced by the
+``-global migration.send-configuration={on|off}`` option.
+
+``-no-kvm`` (since 1.3.0)
+'
+
+The ``-no-kvm`` argument is now a synonym for setting ``-accel tcg``.
+
+``-usbdevice`` (since 2.10.0)
+'
+
+The ``-usbdevice DEV`` argument is now a synonym for setting
+the ``-device usb-DEV`` argument instead. The deprecated syntax
+would automatically enable USB support on the machine type.
+If using the new syntax, USB support must be explicitly
+enabled via the ``-machine usb=on`` argument.
+
+``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
+'''
+
+The 'file' driver for drives is no longer appropriate for character or host
+devices and will only accept regular files (S_IFREG). The correct driver
+for these file types is 'host_cdrom' or 'host_device' as appropriate.
+
+``-net ...,name=``\ *name* (since 3.1)
+''
+
+The ``name`` parameter of the ``-net`` option is a synonym
+for the ``id`` parameter, which should now be used instead.
+
+``-smp`` (invalid topologies) (since 3.1)
+'
+
+CPU topology properties should describe whole machine topology including
+possible CPUs.
+
+However, historically it was possible to start QEMU with an incorrect topology
+where *n* <= *sockets* * *cores* * *threads* < *maxcpus*,
+which could lead to an incorrect topology enumeration by the guest.
+Support for invalid topologies will be removed, the user must ensure
+topologies described with -smp include all possible cpus, i.e.
+*sockets* * *cores* * *threads* = *maxcpus*.
+
+``-vnc acl`` (since 4.0.0)
+''
+
+The ``acl`` option to the ``-vnc`` argument has been replaced
+by the ``tls-authz`` and ``sasl-authz`` options.
+
+``QEMU_AUDIO_`` environment variables and ``-audio-help`` (since 4.0)
+'
+
+The ``-audiodev`` argument is now the preferred way to specify audio
+backend settings instead of environment variables.  To ease migration to
+the new format, the ``-audiodev-help`` option can be used to convert
+the current values of the environment variables to ``-audiodev`` options.
+
+Creating sound card devices and vnc without ``audiodev=`` property (since 4.2)
+''
+
+When not using the deprecated legacy audio config, each sound card
+should specify an ``audiodev=`` property.  Additionally, when using
+vnc, you should specify an ``audiodev=`` propery if you plan to
+transmit audio through the VNC protocol.
+
+``-mon ...,control=readline,pretty=on|off`` (since 4.1)
+'''
+
+The ``pretty=on|off`` switch has no effect for HMP monitors, but is
+silently ignored. Using the switch with HMP monitors will become an
+error in the future.
+
+``-realtime`` (since 4.1)
+'
+
+The ``-realtime mlock=on|off`` argument has been replaced by the
+``

[PATCH v3 08/33] qemu-doc: split target sections to separate files

[Peter Maydell]

From: Paolo Bonzini 

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-9-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 docs/system/target-arm.texi | 264 ++
 docs/system/target-i386.texi|  92 
 docs/system/target-m68k.texi|  46 ++
 docs/system/target-mips.texi| 152 ++
 docs/system/target-ppc.texi |  78 +++
 docs/system/target-sparc.texi   |  96 
 docs/system/target-sparc64.texi |  61 +++
 docs/system/target-xtensa.texi  |  56 ++
 qemu-doc.texi   | 873 +---
 9 files changed, 853 insertions(+), 865 deletions(-)
 create mode 100644 docs/system/target-arm.texi
 create mode 100644 docs/system/target-i386.texi
 create mode 100644 docs/system/target-m68k.texi
 create mode 100644 docs/system/target-mips.texi
 create mode 100644 docs/system/target-ppc.texi
 create mode 100644 docs/system/target-sparc.texi
 create mode 100644 docs/system/target-sparc64.texi
 create mode 100644 docs/system/target-xtensa.texi

diff --git a/docs/system/target-arm.texi b/docs/system/target-arm.texi
new file mode 100644
index 000..040d77b5e05
--- /dev/null
+++ b/docs/system/target-arm.texi
@@ -0,0 +1,264 @@
+@node ARM System emulator
+@section ARM System emulator
+@cindex system emulation (ARM)
+
+Use the executable @file{qemu-system-arm} to simulate a ARM
+machine. The ARM Integrator/CP board is emulated with the following
+devices:
+
+@itemize @minus
+@item
+ARM926E, ARM1026E, ARM946E, ARM1136 or Cortex-A8 CPU
+@item
+Two PL011 UARTs
+@item
+SMC 91c111 Ethernet adapter
+@item
+PL110 LCD controller
+@item
+PL050 KMI with PS/2 keyboard and mouse.
+@item
+PL181 MultiMedia Card Interface with SD card.
+@end itemize
+
+The ARM Versatile baseboard is emulated with the following devices:
+
+@itemize @minus
+@item
+ARM926E, ARM1136 or Cortex-A8 CPU
+@item
+PL190 Vectored Interrupt Controller
+@item
+Four PL011 UARTs
+@item
+SMC 91c111 Ethernet adapter
+@item
+PL110 LCD controller
+@item
+PL050 KMI with PS/2 keyboard and mouse.
+@item
+PCI host bridge.  Note the emulated PCI bridge only provides access to
+PCI memory space.  It does not provide access to PCI IO space.
+This means some devices (eg. ne2k_pci NIC) are not usable, and others
+(eg. rtl8139 NIC) are only usable when the guest drivers use the memory
+mapped control registers.
+@item
+PCI OHCI USB controller.
+@item
+LSI53C895A PCI SCSI Host Bus Adapter with hard disk and CD-ROM devices.
+@item
+PL181 MultiMedia Card Interface with SD card.
+@end itemize
+
+Several variants of the ARM RealView baseboard are emulated,
+including the EB, PB-A8 and PBX-A9.  Due to interactions with the
+bootloader, only certain Linux kernel configurations work out
+of the box on these boards.
+
+Kernels for the PB-A8 board should have CONFIG_REALVIEW_HIGH_PHYS_OFFSET
+enabled in the kernel, and expect 512M RAM.  Kernels for The PBX-A9 board
+should have CONFIG_SPARSEMEM enabled, CONFIG_REALVIEW_HIGH_PHYS_OFFSET
+disabled and expect 1024M RAM.
+
+The following devices are emulated:
+
+@itemize @minus
+@item
+ARM926E, ARM1136, ARM11MPCore, Cortex-A8 or Cortex-A9 MPCore CPU
+@item
+ARM AMBA Generic/Distributed Interrupt Controller
+@item
+Four PL011 UARTs
+@item
+SMC 91c111 or SMSC LAN9118 Ethernet adapter
+@item
+PL110 LCD controller
+@item
+PL050 KMI with PS/2 keyboard and mouse
+@item
+PCI host bridge
+@item
+PCI OHCI USB controller
+@item
+LSI53C895A PCI SCSI Host Bus Adapter with hard disk and CD-ROM devices
+@item
+PL181 MultiMedia Card Interface with SD card.
+@end itemize
+
+The XScale-based clamshell PDA models ("Spitz", "Akita", "Borzoi"
+and "Terrier") emulation includes the following peripherals:
+
+@itemize @minus
+@item
+Intel PXA270 System-on-chip (ARM V5TE core)
+@item
+NAND Flash memory
+@item
+IBM/Hitachi DSCM microdrive in a PXA PCMCIA slot - not in "Akita"
+@item
+On-chip OHCI USB controller
+@item
+On-chip LCD controller
+@item
+On-chip Real Time Clock
+@item
+TI ADS7846 touchscreen controller on SSP bus
+@item
+Maxim MAX analog-digital converter on I@math{^2}C bus
+@item
+GPIO-connected keyboard controller and LEDs
+@item
+Secure Digital card connected to PXA MMC/SD host
+@item
+Three on-chip UARTs
+@item
+WM8750 audio CODEC on I@math{^2}C and I@math{^2}S busses
+@end itemize
+
+The Palm Tungsten|E PDA (codename "Cheetah") emulation includes the
+following elements:
+
+@itemize @minus
+@item
+Texas Instruments OMAP310 System-on-chip (ARM 925T core)
+@item
+ROM and RAM memories (ROM firmware image can be loaded with -option-rom)
+@item
+On-chip LCD controller
+@item
+On-chip Real Time Clock
+@item
+TI TSC2102i touchscreen controller / analog-digital converter / Audio
+CODEC, connected through MicroWire and I@math{^2}S busses
+@item
+GPIO-connected matrix keypad
+@item
+Secure Digital card connected to OMAP MMC/SD host
+@item
+Three on-chip UARTs
+@end itemize
+
+Nokia N800 and N810 internet tablets (known also as RX-34 and RX-44 / 48)
+emulation

[PATCH v3 23/33] docs: Roll semihosting option information into qemu-options.hx

[Peter Maydell]

Currently the per-target documentation for those targets that
implement semihosting includes a bit of text that goes into both the
manual and the manpage about options specific to the target.  This
text is redundant with the earlier generic option description of the
semihosting option produced from qemu-options.hx. To avoid having
to create a lot of stub include files to include into the rST
generated qemu.1 manpage, roll target-specific bits of information
into the qemu-options.hx text, so the user doesn't have to look
in two places for this information.

Signed-off-by: Peter Maydell 
---
Yes, it would in theory make sense to do this before generating
the rst from the texi docs, but I didn't realize this was a
good idea until well after Paolo had done that part.
---
 docs/system/target-arm.rst | 10 --
 docs/system/target-arm.texi| 18 --
 docs/system/target-m68k.rst| 11 ---
 docs/system/target-m68k.texi   | 19 ---
 docs/system/target-xtensa.rst  | 12 
 docs/system/target-xtensa.texi | 20 
 qemu-options.hx| 18 ++
 7 files changed, 18 insertions(+), 90 deletions(-)

diff --git a/docs/system/target-arm.rst b/docs/system/target-arm.rst
index 0490be55871..d2a3b44ce88 100644
--- a/docs/system/target-arm.rst
+++ b/docs/system/target-arm.rst
@@ -215,13 +215,3 @@ emulation includes the following elements:
 
 A Linux 2.6 test image is available on the QEMU web site. More
 information is available in the QEMU mailing-list archive.
-
-The following options are specific to the ARM emulation:
-
-``-semihosting``
-   Enable semihosting syscall emulation.
-
-   On ARM this implements the \"Angel\" interface.
-
-   Note that this allows guest direct access to the host filesystem, so
-   should only be used with trusted guest OS.
diff --git a/docs/system/target-arm.texi b/docs/system/target-arm.texi
index c56b5f6ebfe..eb80dd35f0b 100644
--- a/docs/system/target-arm.texi
+++ b/docs/system/target-arm.texi
@@ -243,21 +243,3 @@ Three on-chip UARTs
 A Linux 2.6 test image is available on the QEMU web site. More
 information is available in the QEMU mailing-list archive.
 
-@c man begin OPTIONS
-
-The following options are specific to the ARM emulation:
-
-@table @option
-
-@item -semihosting
-Enable semihosting syscall emulation.
-
-On ARM this implements the "Angel" interface.
-
-Note that this allows guest direct access to the host filesystem,
-so should only be used with trusted guest OS.
-
-@end table
-
-@c man end
-
diff --git a/docs/system/target-m68k.rst b/docs/system/target-m68k.rst
index 50b7dd9d639..d28d3b92e5e 100644
--- a/docs/system/target-m68k.rst
+++ b/docs/system/target-m68k.rst
@@ -19,14 +19,3 @@ The AN5206 emulation includes the following devices:
 -  MCF5206 ColdFire V2 Microprocessor.
 
 -  Two on-chip UARTs.
-
-The following options are specific to the ColdFire emulation:
-
-``-semihosting``
-   Enable semihosting syscall emulation.
-
-   On M68K this implements the \"ColdFire GDB\" interface used by
-   libgloss.
-
-   Note that this allows guest direct access to the host filesystem, so
-   should only be used with trusted guest OS.
diff --git a/docs/system/target-m68k.texi b/docs/system/target-m68k.texi
index a77b19ea0f1..dcce7bc8c56 100644
--- a/docs/system/target-m68k.texi
+++ b/docs/system/target-m68k.texi
@@ -23,22 +23,3 @@ MCF5206 ColdFire V2 Microprocessor.
 @item
 Two on-chip UARTs.
 @end itemize
-
-@c man begin OPTIONS
-
-The following options are specific to the ColdFire emulation:
-
-@table @option
-
-@item -semihosting
-Enable semihosting syscall emulation.
-
-On M68K this implements the "ColdFire GDB" interface used by libgloss.
-
-Note that this allows guest direct access to the host filesystem,
-so should only be used with trusted guest OS.
-
-@end table
-
-@c man end
-
diff --git a/docs/system/target-xtensa.rst b/docs/system/target-xtensa.rst
index 43cab8dc4da..8d703ad769e 100644
--- a/docs/system/target-xtensa.rst
+++ b/docs/system/target-xtensa.rst
@@ -25,15 +25,3 @@ The Avnet LX60/LX110/LX200 emulation supports:
 -  16550 UART
 
 -  OpenCores 10/100 Mbps Ethernet MAC
-
-The following options are specific to the Xtensa emulation:
-
-``-semihosting``
-   Enable semihosting syscall emulation.
-
-   Xtensa semihosting provides basic file IO calls, such as
-   open/read/write/seek/select. Tensilica baremetal libc for ISS and
-   linux platform \"sim\" use this interface.
-
-   Note that this allows guest direct access to the host filesystem, so
-   should only be used with trusted guest OS.
diff --git a/docs/system/target-xtensa.texi b/docs/system/target-xtensa.texi
index 40327de6fa7..1e6c04dccd6 100644
--- a/docs/system/target-xtensa.texi
+++ b/docs/system/target-xtensa.texi
@@ -33,23 +33,3 @@ A range of Xtensa CPUs, default is the DC232B
 @item
 OpenCores 10/100 Mbps Ethernet MAC
 @end itemize
-
-@c man begin OPTIONS
-
-The following options are specific to the Xtensa e

[PATCH v3 24/33] docs: Roll -prom-env and -g target-specific info into qemu-options.hx

[Peter Maydell]

The SPARC and PPC targets currently have a fragment of target-specific
information about the -g and -prom options which would be better placed
as part of the general documentation of those options in qemu-options.hx.
Move the relevant information to those locations.

SPARC also has a bit of text about the -M option which is out of
date and provides no useful information over the generic documentation
of that option, so just delete it.

The motivation here is again to avoid having to awkwardly include
this text into the rST version of the qemu.1 manpage.

Signed-off-by: Peter Maydell 
---
 docs/system/target-ppc.rst  | 14 --
 docs/system/target-ppc.texi | 25 -
 docs/system/target-sparc.rst| 19 ---
 docs/system/target-sparc.texi   | 27 ---
 docs/system/target-sparc64.rst  | 12 
 docs/system/target-sparc64.texi | 22 --
 qemu-options.hx | 19 +++
 7 files changed, 19 insertions(+), 119 deletions(-)

diff --git a/docs/system/target-ppc.rst b/docs/system/target-ppc.rst
index 43fadf3c00b..a2f04c533c2 100644
--- a/docs/system/target-ppc.rst
+++ b/docs/system/target-ppc.rst
@@ -43,19 +43,5 @@ the g3beige and mac99 PowerMac and the 40p machines. 
OpenBIOS is a free
 (GPL v2) portable firmware implementation. The goal is to implement a
 100% IEEE 1275-1994 (referred to as Open Firmware) compliant firmware.
 
-The following options are specific to the PowerPC emulation:
-
-``-g WxH[xDEPTH]``
-   Set the initial VGA graphic mode. The default is 800x600x32.
-
-``-prom-env string``
-   Set OpenBIOS variables in NVRAM, for example:
-
-   ::
-
-  qemu-system-ppc -prom-env 'auto-boot?=false' \
-   -prom-env 'boot-device=hd:2,\yaboot' \
-   -prom-env 'boot-args=conf=hd:2,\yaboot.conf'
-
 More information is available at
 http://perso.magic.fr/l_indien/qemu-ppc/.
diff --git a/docs/system/target-ppc.texi b/docs/system/target-ppc.texi
index 55f98f65b12..5c83d4f68e7 100644
--- a/docs/system/target-ppc.texi
+++ b/docs/system/target-ppc.texi
@@ -47,31 +47,6 @@ for the g3beige and mac99 PowerMac and the 40p machines. 
OpenBIOS is a free
 (GPL v2) portable firmware implementation. The goal is to implement a 100%
 IEEE 1275-1994 (referred to as Open Firmware) compliant firmware.
 
-@c man begin OPTIONS
-
-The following options are specific to the PowerPC emulation:
-
-@table @option
-
-@item -g @var{W}x@var{H}[x@var{DEPTH}]
-
-Set the initial VGA graphic mode. The default is 800x600x32.
-
-@item -prom-env @var{string}
-
-Set OpenBIOS variables in NVRAM, for example:
-
-@example
-qemu-system-ppc -prom-env 'auto-boot?=false' \
- -prom-env 'boot-device=hd:2,\yaboot' \
- -prom-env 'boot-args=conf=hd:2,\yaboot.conf'
-@end example
-
-@end table
-
-@c man end
-
-
 More information is available at
 @url{http://perso.magic.fr/l_indien/qemu-ppc/}.
 
diff --git a/docs/system/target-sparc.rst b/docs/system/target-sparc.rst
index 589c88d1756..b55f8d09e9c 100644
--- a/docs/system/target-sparc.rst
+++ b/docs/system/target-sparc.rst
@@ -60,22 +60,3 @@ QEMU web site. There are still issues with NetBSD and 
OpenBSD, but most
 kernel versions work. Please note that currently older Solaris kernels
 don't work probably due to interface issues between OpenBIOS and
 Solaris.
-
-The following options are specific to the Sparc32 emulation:
-
-``-g WxHx[xDEPTH]``
-   Set the initial graphics mode. For TCX, the default is 1024x768x8
-   with the option of 1024x768x24. For cgthree, the default is
-   1024x768x8 with the option of 1152x900x8 for people who wish to use
-   OBP.
-
-``-prom-env string``
-   Set OpenBIOS variables in NVRAM, for example:
-
-   ::
-
-  qemu-system-sparc -prom-env 'auto-boot?=false' \
-   -prom-env 'boot-device=sd(0,2,0):d' -prom-env 'boot-args=linux single'
-
-``-M [SS-4|SS-5|SS-10|SS-20|SS-600MP|LX|Voyager|SPARCClassic] [|SPARCbook]``
-   Set the emulated machine type. Default is SS-5.
diff --git a/docs/system/target-sparc.texi b/docs/system/target-sparc.texi
index 7748001f734..99fbf820b42 100644
--- a/docs/system/target-sparc.texi
+++ b/docs/system/target-sparc.texi
@@ -64,32 +64,5 @@ most kernel versions work. Please note that currently older 
Solaris kernels
 don't work probably due to interface issues between OpenBIOS and
 Solaris.
 
-@c man begin OPTIONS
-
-The following options are specific to the Sparc32 emulation:
-
-@table @option
-
-@item -g @var{W}x@var{H}x[x@var{DEPTH}]
-
-Set the initial graphics mode. For TCX, the default is 1024x768x8 with the
-option of 1024x768x24. For cgthree, the default is 1024x768x8 with the option
-of 1152x900x8 for people who wish to use OBP.
-
-@item -prom-env @var{string}
-
-Set OpenBIOS variables in NVRAM, for example:
-
-@example
-qemu-system-sparc -prom-env 'auto-boot?=false' \
- -prom-env 'boot-device=sd(0,2,0):d' -prom-env 'boot-args=linux single'
-@end example
-
-@item -M [SS-4|SS-5|SS-10|SS-20|SS-600MP|LX|Voyager|SPAR

[PATCH v3 22/33] doc/scripts/hxtool.py: Strip trailing ':' from DEFHEADING/ARCHHEADING

[Peter Maydell]

In hxtool files, section headings defined with the DEFHEADING
and ARCHHEADING macros have a trailing ':'
  DEFHEADING(Standard options:)

This is for the benefit of the --help output. For consistency
with the rest of the rST documentation, strip any trailing ':'
when we construct headings with the Sphinx hxtool extension.
This makes the table of contents look neater.

This only affects generation of documentation from qemu-options.hx,
which we will start doing in a later commit.

Signed-off-by: Peter Maydell 
---
 docs/sphinx/hxtool.py | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/docs/sphinx/hxtool.py b/docs/sphinx/hxtool.py
index 5d6736f3002..7dd223fe362 100644
--- a/docs/sphinx/hxtool.py
+++ b/docs/sphinx/hxtool.py
@@ -60,8 +60,9 @@ def parse_defheading(file, lnum, line):
 # empty we ignore the directive -- these are used only to add
 # blank lines in the plain-text content of the --help output.
 #
-# Return the heading text
-match = re.match(r'DEFHEADING\((.*)\)', line)
+# Return the heading text. We strip out any trailing ':' for
+# consistency with other headings in the rST documentation.
+match = re.match(r'DEFHEADING\((.*?):?\)', line)
 if match is None:
 serror(file, lnum, "Invalid DEFHEADING line")
 return match.group(1)
@@ -72,8 +73,9 @@ def parse_archheading(file, lnum, line):
 # though note that the 'some string' could be the empty string.
 # As with DEFHEADING, empty string ARCHHEADINGs will be ignored.
 #
-# Return the heading text
-match = re.match(r'ARCHHEADING\((.*),.*\)', line)
+# Return the heading text. We strip out any trailing ':' for
+# consistency with other headings in the rST documentation.
+match = re.match(r'ARCHHEADING\((.*?):?,.*\)', line)
 if match is None:
 serror(file, lnum, "Invalid ARCHHEADING line")
 return match.group(1)
-- 
2.20.1

[PATCH v3 28/33] docs: Split out sections for the manpage into .rst.inc files

[Peter Maydell]

Sphinx doesn't have very good facilities for marking chunks
of documentation as "put this in the manpage only". So instead
we move the parts we want to put into both the HTML manuals
and the manpage into their own .rst.inc files, which we can
include from both the main manual rst files and a new toplevel
rst file that will be the skeleton of the qemu.1 manpage.

In this commit, just split out the parts of the documentation
that go in the manpage.

Signed-off-by: Peter Maydell 
---
 ...vocation.rst => device-url-syntax.rst.inc} |  16 --
 docs/system/invocation.rst| 228 +-
 docs/system/keys.rst  |  36 +--
 docs/system/{keys.rst => keys.rst.inc}|   5 -
 docs/system/mux-chardev.rst   |  28 +--
 .../{mux-chardev.rst => mux-chardev.rst.inc}  |   5 -
 ...rget-i386.rst => target-i386-desc.rst.inc} |  22 --
 docs/system/target-i386.rst   |  63 +
 8 files changed, 4 insertions(+), 399 deletions(-)
 copy docs/system/{invocation.rst => device-url-syntax.rst.inc} (96%)
 copy docs/system/{keys.rst => keys.rst.inc} (90%)
 copy docs/system/{mux-chardev.rst => mux-chardev.rst.inc} (87%)
 copy docs/system/{target-i386.rst => target-i386-desc.rst.inc} (76%)

diff --git a/docs/system/invocation.rst b/docs/system/device-url-syntax.rst.inc
similarity index 96%
copy from docs/system/invocation.rst
copy to docs/system/device-url-syntax.rst.inc
index d77dd13d4f7..88d7a372a74 100644
--- a/docs/system/invocation.rst
+++ b/docs/system/device-url-syntax.rst.inc
@@ -1,19 +1,3 @@
-.. _sec_005finvocation:
-
-Invocation
---
-
-.. parsed-literal::
-
-   |qemu_system| [options] [disk_image]
-
-disk_image is a raw hard disk image for IDE hard disk 0. Some targets do
-not need a disk image.
-
-.. hxtool-doc:: qemu-options.hx
-
-Device URL Syntax
-~
 
 In addition to using normal file images for the emulated storage
 devices, QEMU can also use networked resources such as iSCSI devices.
diff --git a/docs/system/invocation.rst b/docs/system/invocation.rst
index d77dd13d4f7..4ba38fc23d2 100644
--- a/docs/system/invocation.rst
+++ b/docs/system/invocation.rst
@@ -15,230 +15,4 @@ not need a disk image.
 Device URL Syntax
 ~
 
-In addition to using normal file images for the emulated storage
-devices, QEMU can also use networked resources such as iSCSI devices.
-These are specified using a special URL syntax.
-
-``iSCSI``
-   iSCSI support allows QEMU to access iSCSI resources directly and use
-   as images for the guest storage. Both disk and cdrom images are
-   supported.
-
-   Syntax for specifying iSCSI LUNs is
-   "iscsi://[:]//"
-
-   By default qemu will use the iSCSI initiator-name
-   'iqn.2008-11.org.linux-kvm[:]' but this can also be set from
-   the command line or a configuration file.
-
-   Since version Qemu 2.4 it is possible to specify a iSCSI request
-   timeout to detect stalled requests and force a reestablishment of the
-   session. The timeout is specified in seconds. The default is 0 which
-   means no timeout. Libiscsi 1.15.0 or greater is required for this
-   feature.
-
-   Example (without authentication):
-
-   .. parsed-literal::
-
-  |qemu_system| -iscsi initiator-name=iqn.2001-04.com.example:my-initiator 
\
-   -cdrom iscsi://192.0.2.1/iqn.2001-04.com.example/2 \
-   -drive file=iscsi://192.0.2.1/iqn.2001-04.com.example/1
-
-   Example (CHAP username/password via URL):
-
-   .. parsed-literal::
-
-  |qemu_system| -drive 
file=iscsi://user%password@192.0.2.1/iqn.2001-04.com.example/1
-
-   Example (CHAP username/password via environment variables):
-
-   .. parsed-literal::
-
-  LIBISCSI_CHAP_USERNAME="user" \
-  LIBISCSI_CHAP_PASSWORD="password" \
-  |qemu_system| -drive file=iscsi://192.0.2.1/iqn.2001-04.com.example/1
-
-``NBD``
-   QEMU supports NBD (Network Block Devices) both using TCP protocol as
-   well as Unix Domain Sockets. With TCP, the default port is 10809.
-
-   Syntax for specifying a NBD device using TCP, in preferred URI form:
-   "nbd://[:]/[]"
-
-   Syntax for specifying a NBD device using Unix Domain Sockets;
-   remember that '?' is a shell glob character and may need quoting:
-   "nbd+unix:///[]?socket="
-
-   Older syntax that is also recognized:
-   "nbd::[:exportname=]"
-
-   Syntax for specifying a NBD device using Unix Domain Sockets
-   "nbd:unix:[:exportname=]"
-
-   Example for TCP
-
-   .. parsed-literal::
-
-  |qemu_system| --drive file=nbd:192.0.2.1:3
-
-   Example for Unix Domain Sockets
-
-   .. parsed-literal::
-
-  |qemu_system| --drive file=nbd:unix:/tmp/nbd-socket
-
-``SSH``
-   QEMU supports SSH (Secure Shell) access to remote disks.
-
-   Examples:
-
-   .. parsed-literal::
-
-  |qemu_system| -drive file=ssh://user@host/path/to/disk.img
-  |qemu_system| -drive 
file.driver=ssh,file.user=user,file.host=host,file.port=22,file.path=/path/to/disk.img
-
-   Curre

[PATCH v3 05/33] qemu-doc: split qemu-doc.texi in multiple files

[Peter Maydell]

From: Paolo Bonzini 

In order to facilitate the reorganization of qemu-doc.texi content,
as well as the conversion to rST/Sphinx, split it in multiple .texi
files that are included from docs/system.

The "other devices" section is renamed to ivshmem and placed last.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-6-pbonz...@redhat.com
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 Makefile |   16 +
 docs/system/build-platforms.texi |   67 ++
 docs/system/gdb.texi |   71 ++
 docs/system/images.texi  |   88 ++
 docs/system/invocation.texi  |  240 +
 docs/system/ivshmem.texi |   60 ++
 docs/system/keys.texi|   53 ++
 docs/system/license.texi |9 +
 docs/system/linuxboot.texi   |   27 +
 docs/system/monitor.texi |   35 +
 docs/system/mux-chardev.texi |   51 ++
 docs/system/net.texi |   96 ++
 docs/system/quickstart.texi  |   13 +
 docs/system/tls.texi |  329 +++
 docs/system/usb.texi |  115 +++
 docs/system/vnc-security.texi|  196 
 qemu-doc.texi| 1471 +-
 17 files changed, 1483 insertions(+), 1454 deletions(-)
 create mode 100644 docs/system/build-platforms.texi
 create mode 100644 docs/system/gdb.texi
 create mode 100644 docs/system/images.texi
 create mode 100644 docs/system/invocation.texi
 create mode 100644 docs/system/ivshmem.texi
 create mode 100644 docs/system/keys.texi
 create mode 100644 docs/system/license.texi
 create mode 100644 docs/system/linuxboot.texi
 create mode 100644 docs/system/monitor.texi
 create mode 100644 docs/system/mux-chardev.texi
 create mode 100644 docs/system/net.texi
 create mode 100644 docs/system/quickstart.texi
 create mode 100644 docs/system/tls.texi
 create mode 100644 docs/system/usb.texi
 create mode 100644 docs/system/vnc-security.texi

diff --git a/Makefile b/Makefile
index f75a7b51938..9790a0fd15b 100644
--- a/Makefile
+++ b/Makefile
@@ -1123,6 +1123,22 @@ qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
qemu-tech.texi qemu-option-trace.texi \
qemu-deprecated.texi qemu-monitor.texi \
qemu-monitor-info.texi \
+docs/system/quickstart.texi \
+docs/system/invocation.texi \
+docs/system/keys.texi \
+docs/system/mux-chardev.texi \
+docs/system/monitor.texi \
+docs/system/cpu-models-x86.texi \
+docs/system/images.texi \
+docs/system/net.texi \
+docs/system/usb.texi \
+docs/system/ivshmem.texi \
+docs/system/linuxboot.texi \
+docs/system/vnc-security.texi \
+docs/system/tls.texi \
+docs/system/gdb.texi \
+docs/system/build-platforms.texi \
+docs/system/license.texi \
docs/system/cpu-models-x86.texi docs/system/cpu-models-mips.texi \
docs/security.texi
 
diff --git a/docs/system/build-platforms.texi b/docs/system/build-platforms.texi
new file mode 100644
index 000..531ef5bed44
--- /dev/null
+++ b/docs/system/build-platforms.texi
@@ -0,0 +1,67 @@
+@node Supported build platforms
+@appendix Supported build platforms
+
+QEMU aims to support building and executing on multiple host OS platforms.
+This appendix outlines which platforms are the major build targets. These
+platforms are used as the basis for deciding upon the minimum required
+versions of 3rd party software QEMU depends on. The supported platforms
+are the targets for automated testing performed by the project when patches
+are submitted for review, and tested before and after merge.
+
+If a platform is not listed here, it does not imply that QEMU won't work.
+If an unlisted platform has comparable software versions to a listed platform,
+there is every expectation that it will work. Bug reports are welcome for
+problems encountered on unlisted platforms unless they are clearly older
+vintage than what is described here.
+
+Note that when considering software versions shipped in distros as support
+targets, QEMU considers only the version number, and assumes the features in
+that distro match the upstream release with the same version. In other words,
+if a distro backports extra features to the software in their distro, QEMU
+upstream code will not add explicit support for those backports, unless the
+feature is auto-detectable in a manner that works for the upstream releases
+too.
+
+The Repology site @url{https://repology.org} is a useful resource to identify
+currently shipped versions of software in various operating systems, though
+it does not cover all distros listed below.
+
+@section Linux OS
+
+For distributions with frequent, short-lifetime releases, the project will
+aim to support all versions that are not end of life by their respective
+vendors. For the purposes of identifying supported software versions, the
+project will look at Fedora, Ubuntu, and openSUSE distros. Other short-
+lifetime di

[PATCH v3 21/33] hmp-commands-info.hx: Add rST documentation fragments

[Peter Maydell]

Add the rST versions of the documentation fragments.  Once we've
converted fully from Texinfo to rST we can remove the ETEXI
fragments; for the moment we need both.

Note that most of the SRST fragments are 2-space indented so that the
'info foo' documentation entries appear as a sublist under the 'info'
entry in the top level list.

Again, all we need to do to put the documentation in the Sphinx manual
is a one-line hxtool-doc invocation.

Signed-off-by: Peter Maydell 
---
 Makefile|   2 +-
 docs/system/monitor.rst |   4 +-
 hmp-commands-info.hx| 271 
 3 files changed, 273 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 22427376b48..98ef06ab8e8 100644
--- a/Makefile
+++ b/Makefile
@@ -1071,7 +1071,7 @@ $(MANUAL_BUILDDIR)/interop/index.html: $(call 
manual-deps,interop)
 $(MANUAL_BUILDDIR)/specs/index.html: $(call manual-deps,specs)
$(call build-manual,specs,html)
 
-$(MANUAL_BUILDDIR)/system/index.html: $(call manual-deps,system) 
$(SRC_PATH)/hmp-commands.hx
+$(MANUAL_BUILDDIR)/system/index.html: $(call manual-deps,system) 
$(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/hmp-commands-info.hx
$(call build-manual,system,html)
 
 $(MANUAL_BUILDDIR)/tools/index.html: $(call manual-deps,tools) 
$(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/docs/qemu-option-trace.rst.inc
diff --git a/docs/system/monitor.rst b/docs/system/monitor.rst
index 52b0f18cfec..0bcd5da2164 100644
--- a/docs/system/monitor.rst
+++ b/docs/system/monitor.rst
@@ -21,9 +21,7 @@ The following commands are available:
 
 .. hxtool-doc:: hmp-commands.hx
 
-..
-  TODO: convert and reference hmp-commands-info.hx
-
+.. hxtool-doc:: hmp-commands-info.hx
 
 Integer expressions
 ~~~
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 257ee7d7a3f..1730f866cde 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -4,6 +4,12 @@ HXCOMM discarded from C version
 HXCOMM DEF(command, args, callback, arg_string, help) is used to construct
 HXCOMM monitor info commands
 HXCOMM HXCOMM can be used for comments, discarded from both texi and C
+HXCOMM
+HXCOMM In this file, generally SRST fragments should have two extra
+HXCOMM spaces of indent, so that the documentation list item for "info foo"
+HXCOMM appears inside the documentation list item for the top level
+HXCOMM "info" documentation entry. The exception is the first SRST
+HXCOMM fragment that defines that top level entry.
 
 STEXI
 @table @option
@@ -12,6 +18,11 @@ STEXI
 Show various information about the system state.
 @table @option
 ETEXI
+SRST
+``info`` *subcommand*
+  Show various information about the system state.
+
+ERST
 
 {
 .name   = "version",
@@ -27,6 +38,10 @@ STEXI
 @findex info version
 Show the version of QEMU.
 ETEXI
+SRST
+  ``info version``
+Show the version of QEMU.
+ERST
 
 {
 .name   = "network",
@@ -41,6 +56,10 @@ STEXI
 @findex info network
 Show the network state.
 ETEXI
+SRST
+  ``info network``
+Show the network state.
+ERST
 
 {
 .name   = "chardev",
@@ -56,6 +75,10 @@ STEXI
 @findex info chardev
 Show the character devices.
 ETEXI
+SRST
+  ``info chardev``
+Show the character devices.
+ERST
 
 {
 .name   = "block",
@@ -71,6 +94,10 @@ STEXI
 @findex info block
 Show info of one block device or all block devices.
 ETEXI
+SRST
+  ``info block``
+Show info of one block device or all block devices.
+ERST
 
 {
 .name   = "blockstats",
@@ -85,6 +112,10 @@ STEXI
 @findex info blockstats
 Show block device statistics.
 ETEXI
+SRST
+  ``info blockstats``
+Show block device statistics.
+ERST
 
 {
 .name   = "block-jobs",
@@ -99,6 +130,10 @@ STEXI
 @findex info block-jobs
 Show progress of ongoing block device operations.
 ETEXI
+SRST
+  ``info block-jobs``
+Show progress of ongoing block device operations.
+ERST
 
 {
 .name   = "registers",
@@ -113,6 +148,10 @@ STEXI
 @findex info registers
 Show the cpu registers.
 ETEXI
+SRST
+  ``info registers``
+Show the cpu registers.
+ERST
 
 #if defined(TARGET_I386)
 {
@@ -130,6 +169,10 @@ STEXI
 @findex info lapic
 Show local APIC state
 ETEXI
+SRST
+  ``info lapic``
+Show local APIC state
+ERST
 
 #if defined(TARGET_I386)
 {
@@ -146,6 +189,10 @@ STEXI
 @findex info ioapic
 Show io APIC state
 ETEXI
+SRST
+  ``info ioapic``
+Show io APIC state
+ERST
 
 {
 .name   = "cpus",
@@ -160,6 +207,10 @@ STEXI
 @findex info cpus
 Show infos for each CPU.
 ETEXI
+SRST
+  ``info cpus``
+Show infos for each CPU.
+ERST
 
 {
 .name   = "history",
@@ -175,6 +226,10 @@ STEXI
 @findex info history
 Show the command line history.
 ETEXI
+SRST
+  ``info history``
+Show the command line history.
+ERST
 
 {
 .name   = "irq",
@@ -189,6 +244,10 @@ STEXI
 @findex info irq
 Show the interrupts statistics (if available).
 ETEXI
+SRST
+  ``info i

[PATCH v3 25/33] scripts/hxtool-conv: Archive script used in qemu-options.hx conversion

[Peter Maydell]

This commit archives the perl script used to do conversion of the
STEXI/ETEXI blocks in qemu-options.hx. (The other .hx files were
manually converted, but qemu-options.hx is complicated enough that
I felt I needed some scripting.)

Signed-off-by: Peter Maydell 
---
Please don't critique the script, it is purely for a one-off
conversion job, and I then did manual fixups on the output
to get the changes in the following patch. I merely felt it
was potentially useful to archive a copy of the mechanism used.
Or we could drop this patch if that's not needed.
---
 scripts/hxtool-conv.pl | 137 +
 1 file changed, 137 insertions(+)
 create mode 100755 scripts/hxtool-conv.pl

diff --git a/scripts/hxtool-conv.pl b/scripts/hxtool-conv.pl
new file mode 100755
index 000..eede40b3462
--- /dev/null
+++ b/scripts/hxtool-conv.pl
@@ -0,0 +1,137 @@
+#!/usr/bin/perl -w
+#
+# Script to convert .hx file STEXI/ETEXI blocks to SRST/ERST
+#
+# Copyright (C) 2020 Linaro
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# (at your option) any later version. See the COPYING file in the
+# top-level directory.
+
+# This script was only ever intended as a one-off conversion operation.
+# Please excuse the places where it is a bit hacky.
+# Some manual intervention after the conversion is expected, as are
+# some warnings from makeinfo.
+# Warning: this script is not idempotent: don't try to run it on
+# a .hx file that already has SRST/ERST sections.
+
+# Expected usage:
+# scripts/hxtool-conv.pl file.hx > file.hx.new
+
+use utf8;
+
+my $reading_texi = 0;
+my $texiblock = '';
+my @tables = ();
+
+sub update_tables($) {
+my ($texi) = @_;
+# Update our list of open table directives: every @table
+# line in the texi fragment is added to the list, and every
+# @end table line means we remove an entry from the list.
+# If this fragment had a completely self contained table with
+# both the @table and @end table lines, this will be a no-op.
+foreach (split(/\n/, $texi)) {
+push @tables, $_ if /^\@table/;
+pop @tables if /^\@end table/;
+}
+}
+
+sub only_table_directives($) {
+# Return true if every line in the fragment is a start or end table 
directive
+my ($texi) = @_;
+foreach (split(/\n/, $texi)) {
+return 0 unless /^\@table/ or /^\@end table/;
+}
+return 1;
+}
+
+sub output_rstblock($) {
+# Write the output to /tmp/frag.texi, wrapped in whatever current @table
+# lines we need.
+my ($texi) = @_;
+
+# As a special case, if this fragment is only table directives and
+# nothing else, update our set of open table directives but otherwise
+# ignore it. This avoids emitting an empty SRST/ERST block.
+if (only_table_directives($texi)) {
+update_tables($texi);
+return;
+}
+
+open(my $fragfh, '>', '/tmp/frag.texi');
+# First output the currently active set of open table directives
+print $fragfh join("\n", @tables);
+# Next, update our list of open table directives.
+# We need to do this before we emit the closing table directives
+# so that we emit the right number if this fragment had an
+# unbalanced set of directives.
+update_tables($texi);
+# Then emit the texi fragment itself.
+print $fragfh "\n$texi\n";
+# Finally, add the necessary closing table directives.
+print $fragfh "\@end table\n" x scalar @tables;
+close $fragfh;
+
+# Now invoke makeinfo/pandoc on it and slurp the results into a string
+open(my $fh, '-|', "makeinfo --force -o - --docbook "
+ . "-D 'qemu_system_x86 QEMU_SYSTEM_X86_MACRO' "
+ . "-D 'qemu_system QEMU_SYSTEM_MACRO'  /tmp/frag.texi "
+ . " | pandoc  -f docbook -t rst")
+or die "can't start makeinfo/pandoc: $!";
+
+binmode $fh, ':encoding(utf8)';
+
+print "SRST\n";
+
+# Slurp the whole thing into a string so we can do multiline
+# string matches on it.
+my $rst = do {
+local $/ = undef;
+<$fh>;
+};
+$rst =~ s/^-  − /-  /gm;
+$rst =~ s/“/"/gm;
+$rst =~ s/”/"/gm;
+$rst =~ s/‘/'/gm;
+$rst =~ s/’/'/gm;
+$rst =~ s/QEMU_SYSTEM_MACRO/|qemu_system|/g;
+$rst =~ s/QEMU_SYSTEM_X86_MACRO/|qemu_system_x86|/g;
+$rst =~ s/(?=::\n\n +\|qemu)/.. parsed-literal/g;
+$rst =~ s/:\n\n::$/::/gm;
+
+# Fix up the invalid reference format makeinfo/pandoc emit:
+# `Some string here <#anchorname>`__
+# should be:
+# :ref:`anchorname`
+$rst =~ s/\`[^<`]+\<\#([^>]+)\>\`__/:ref:`$1`/gm;
+print $rst;
+
+close $fh or die "error on close: $!";
+print "ERST\n";
+}
+
+# Read the whole .hx input file.
+while (<>) {
+# Always print the current line
+print;
+if (/STEXI/) {
+$reading_texi = 1;
+$texiblock = '';
+next;
+}
+if (/ETEXI/) {
+$reading_texi = 0;
+# dump RST version of block
+output_rstbloc

[PATCH v3 19/33] docs/system: convert Texinfo documentation to rST

[Peter Maydell]

From: Paolo Bonzini 

Apart from targets.rst, which was written by hand, this is an automated
conversion obtained with the following command:

  makeinfo --force -o - --docbook \
-D 'qemu_system_x86 QEMU_SYSTEM_X86_MACRO' \
-D 'qemu_system QEMU_SYSTEM_MACRO' \
$texi | pandoc -f docbook -t rst+smart | perl -e '
  $/=undef;
  $_ = <>;
  s/^-  − /-  /gm;
  s/QEMU_SYSTEM_MACRO/|qemu_system|/g;
  s/QEMU_SYSTEM_X86_MACRO/|qemu_system_x86|/g;
  s/(?=::\n\n +\|qemu)/.. parsed-literal/g;
  s/:\n\n::$/::/gm;
  print' > $rst

In addition, the following changes were made manually:

- target-i386.rst and target-mips.rst: replace CPU model documentation with
  an include directive

- monitor.rst: replace the command section with a comment

- images.rst: add toctree

- target-arm.rst: Replace use of :math: (which Sphinx complains
  about) with :sup:, and hide it behind |I2C| and |I2C| substitutions.

Content that is not @included remains exclusive to qemu-doc.texi.

Signed-off-by: Paolo Bonzini 
Message-id: 20200226113034.6741-19-pbonz...@redhat.com
[PMM: Fixed target-arm.rst use of :math:; remove out of date
 note about images.rst from commit message; fixed expansion
 of |qemu_system_x86|; use parsed-literal in invocation.rst
 when we want to use |qemu_system_x86|; fix incorrect subsection
 level for "OS requirements" in target-i386.rst; fix incorrect
 syntax for making links to other sections of the manual]
Reviewed-by: Peter Maydell 
Signed-off-by: Peter Maydell 
---
 docs/defs.rst.inc   |  12 +-
 docs/system/build-platforms.rst |  79 
 docs/system/gdb.rst |  81 
 docs/system/images.rst  |  85 +
 docs/system/index.rst   |  19 +-
 docs/system/invocation.rst  | 242 +++
 docs/system/ivshmem.rst |  64 +++
 docs/system/keys.rst|  40 
 docs/system/license.rst |  11 ++
 docs/system/linuxboot.rst   |  30 +++
 docs/system/monitor.rst |  25 +++
 docs/system/mux-chardev.rst |  32 
 docs/system/net.rst | 100 ++
 docs/system/quickstart.rst  |  13 ++
 docs/system/target-arm.rst  | 227 ++
 docs/system/target-i386.rst |  84 
 docs/system/target-m68k.rst |  32 
 docs/system/target-mips.rst | 120 
 docs/system/target-ppc.rst  |  61 ++
 docs/system/target-sparc.rst|  81 
 docs/system/target-sparc64.rst  |  49 +
 docs/system/target-xtensa.rst   |  39 
 docs/system/targets.rst |  19 ++
 docs/system/tls.rst | 328 
 docs/system/usb.rst | 137 +
 docs/system/vnc-security.rst| 202 
 26 files changed, 2207 insertions(+), 5 deletions(-)
 create mode 100644 docs/system/build-platforms.rst
 create mode 100644 docs/system/gdb.rst
 create mode 100644 docs/system/images.rst
 create mode 100644 docs/system/invocation.rst
 create mode 100644 docs/system/ivshmem.rst
 create mode 100644 docs/system/keys.rst
 create mode 100644 docs/system/license.rst
 create mode 100644 docs/system/linuxboot.rst
 create mode 100644 docs/system/monitor.rst
 create mode 100644 docs/system/mux-chardev.rst
 create mode 100644 docs/system/net.rst
 create mode 100644 docs/system/quickstart.rst
 create mode 100644 docs/system/target-arm.rst
 create mode 100644 docs/system/target-i386.rst
 create mode 100644 docs/system/target-m68k.rst
 create mode 100644 docs/system/target-mips.rst
 create mode 100644 docs/system/target-ppc.rst
 create mode 100644 docs/system/target-sparc.rst
 create mode 100644 docs/system/target-sparc64.rst
 create mode 100644 docs/system/target-xtensa.rst
 create mode 100644 docs/system/targets.rst
 create mode 100644 docs/system/tls.rst
 create mode 100644 docs/system/usb.rst
 create mode 100644 docs/system/vnc-security.rst

diff --git a/docs/defs.rst.inc b/docs/defs.rst.inc
index ae80d2f35d8..48d05aaf33e 100644
--- a/docs/defs.rst.inc
+++ b/docs/defs.rst.inc
@@ -3,9 +3,13 @@
all rST files as part of the epilogue by docs/conf.py.  conf.py
also defines some dynamically generated substitutions like CONFDIR.
 
-   Note that |qemu_system| is intended to be used inside a parsed-literal
-   block: the definition must not include extra literal formatting with
-   ``..``: this works in the HTML output but the manpages will end up
-   misrendered with following normal text incorrectly in boldface.
+   Note that |qemu_system| and |qemu_system_x86| are intended to be
+   used inside a parsed-literal block: the definition must not include
+   extra literal formatting with ``..``: this works in the HTML output
+   but the manpages will end up misrendered with following normal text
+   incorrectly in boldface.
 
 .. |qemu_system| replace:: qemu-system-x86_64
+.. |qemu_system_x86| replace:: qemu_system-x86_64
+.. |I2C| replace:: I\ :sup:`2`\ C
+.. |I2S| replace:: I\ :

[PATCH v3 32/33] docs: Remove old texinfo sources

[Peter Maydell]

We can now delete the old .texi files, which we have been keeping in
the tree as a parallel set of documentation to the new rST sources.
The only remaining use of Texinfo is the autogenerated manuals
and HTML documents created from the QAPI JSON doc comments.

Signed-off-by: Peter Maydell 
---
 MAINTAINERS|   5 +-
 docs/system/build-platforms.texi   |  67 
 docs/system/cpu-models-mips.texi   | 157 --
 docs/system/cpu-models-x86.texi| 482 -
 docs/system/deprecated.texi| 377 --
 docs/system/gdb.texi   |  71 -
 docs/system/images.texi|  88 --
 docs/system/invocation.texi| 240 --
 docs/system/ivshmem.texi   |  60 
 docs/system/keys.texi  |  43 ---
 docs/system/license.texi   |   9 -
 docs/system/linuxboot.texi |  27 --
 docs/system/managed-startup.texi   |  35 ---
 docs/system/monitor.texi   |  34 --
 docs/system/mux-chardev.texi   |  44 ---
 docs/system/net.texi   |  96 --
 docs/system/qemu-option-trace.texi |  28 --
 docs/system/quickstart.texi|  12 -
 docs/system/security.texi  | 167 --
 docs/system/target-arm.texi| 245 ---
 docs/system/target-i386.texi   |  91 --
 docs/system/target-m68k.texi   |  25 --
 docs/system/target-mips.texi   | 150 -
 docs/system/target-ppc.texi|  52 
 docs/system/target-sparc.texi  |  68 
 docs/system/target-sparc64.texi|  38 ---
 docs/system/target-xtensa.texi |  35 ---
 docs/system/tls.texi   | 329 
 docs/system/usb.texi   | 115 ---
 docs/system/vnc-security.texi  | 196 
 qemu-doc.texi  | 201 
 31 files changed, 1 insertion(+), 3586 deletions(-)
 delete mode 100644 docs/system/build-platforms.texi
 delete mode 100644 docs/system/cpu-models-mips.texi
 delete mode 100644 docs/system/cpu-models-x86.texi
 delete mode 100644 docs/system/deprecated.texi
 delete mode 100644 docs/system/gdb.texi
 delete mode 100644 docs/system/images.texi
 delete mode 100644 docs/system/invocation.texi
 delete mode 100644 docs/system/ivshmem.texi
 delete mode 100644 docs/system/keys.texi
 delete mode 100644 docs/system/license.texi
 delete mode 100644 docs/system/linuxboot.texi
 delete mode 100644 docs/system/managed-startup.texi
 delete mode 100644 docs/system/monitor.texi
 delete mode 100644 docs/system/mux-chardev.texi
 delete mode 100644 docs/system/net.texi
 delete mode 100644 docs/system/qemu-option-trace.texi
 delete mode 100644 docs/system/quickstart.texi
 delete mode 100644 docs/system/security.texi
 delete mode 100644 docs/system/target-arm.texi
 delete mode 100644 docs/system/target-i386.texi
 delete mode 100644 docs/system/target-m68k.texi
 delete mode 100644 docs/system/target-mips.texi
 delete mode 100644 docs/system/target-ppc.texi
 delete mode 100644 docs/system/target-sparc.texi
 delete mode 100644 docs/system/target-sparc64.texi
 delete mode 100644 docs/system/target-xtensa.texi
 delete mode 100644 docs/system/tls.texi
 delete mode 100644 docs/system/usb.texi
 delete mode 100644 docs/system/vnc-security.texi
 delete mode 100644 qemu-doc.texi

diff --git a/MAINTAINERS b/MAINTAINERS
index 1712eb90851..45dd83b8b56 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -215,7 +215,6 @@ S: Maintained
 F: target/mips/
 F: default-configs/*mips*
 F: disas/*mips*
-F: docs/system/cpu-models-mips.texi
 F: docs/system/cpu-models-mips.rst.inc
 F: hw/intc/mips_gic.c
 F: hw/mips/
@@ -321,7 +320,6 @@ F: tests/tcg/i386/
 F: tests/tcg/x86_64/
 F: hw/i386/
 F: disas/i386.c
-F: docs/system/cpu-models-x86.texi
 F: docs/system/cpu-models-x86.rst.inc
 T: git https://github.com/ehabkost/qemu.git x86-next
 
@@ -2234,7 +2232,7 @@ M: Stefan Hajnoczi 
 S: Maintained
 F: trace/
 F: trace-events
-F: docs/system/qemu-option-trace.texi
+F: docs/qemu-option-trace.rst.inc
 F: scripts/tracetool.py
 F: scripts/tracetool/
 F: scripts/qemu-trace-stap*
@@ -2804,7 +2802,6 @@ F: contrib/gitdm/*
 
 Incompatible changes
 R: libvir-l...@redhat.com
-F: docs/system/deprecated.texi
 F: docs/system/deprecated.rst
 
 Build System
diff --git a/docs/system/build-platforms.texi b/docs/system/build-platforms.texi
deleted file mode 100644
index 531ef5bed44..000
--- a/docs/system/build-platforms.texi
+++ /dev/null
@@ -1,67 +0,0 @@
-@node Supported build platforms
-@appendix Supported build platforms
-
-QEMU aims to support building and executing on multiple host OS platforms.
-This appendix outlines which platforms are the major build targets. These
-platforms are used as the basis for deciding upon the minimum required
-versions of 3rd party software QEMU depends on. The supported platforms
-are the targets for automated testing performed by the project when patches
-are submitted for review, and tested before and after merge.
-
-If a pl

[PATCH v3 30/33] ui/cocoa.m: Update documentation file and pathname

[Peter Maydell]

We want to stop generating the old qemu-doc.html; first we
must update places that refer to it so they instead go to
our top level index.html documentation landing page.
The Cocoa UI has a menu option to bring up the documentation;
make it point to the new top level index.html instead.

Signed-off-by: Peter Maydell 
---
 ui/cocoa.m | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ui/cocoa.m b/ui/cocoa.m
index 747a70839af..cb556e4e668 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1174,7 +1174,7 @@ QemuCocoaView *cocoaView;
 - (void) openDocumentation: (NSString *) filename
 {
 /* Where to look for local files */
-NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", @"../"};
+NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
@"../docs/"};
 NSString *full_file_path;
 
 /* iterate thru the possible paths until the file is found */
@@ -1198,7 +1198,7 @@ QemuCocoaView *cocoaView;
 {
 COCOA_DEBUG("QemuCocoaAppController: showQEMUDoc\n");
 
-[self openDocumentation: @"qemu-doc.html"];
+[self openDocumentation: @"index.html"];
 }
 
 /* Stretches video to fit host monitor size */
-- 
2.20.1

[PATCH v3 20/33] hmp-commands.hx: Add rST documentation fragments

[Peter Maydell]

Add the rST versions of the documentation fragments.  Once we've
converted fully from Texinfo to rST we can remove the ETEXI
fragments; for the moment we need both.

Since the only consumer of the hmp-commands hxtool documentation
is the HTML manual, all we need to do for the monitor command
documentation to appear in the Sphinx system manual is add the
one line that invokes the hxtool extension on the .hx file.

Signed-off-by: Peter Maydell 
---
 Makefile|   2 +-
 docs/system/monitor.rst |  10 +-
 hmp-commands.hx | 688 
 3 files changed, 698 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index f8642cd28a1..22427376b48 100644
--- a/Makefile
+++ b/Makefile
@@ -1071,7 +1071,7 @@ $(MANUAL_BUILDDIR)/interop/index.html: $(call 
manual-deps,interop)
 $(MANUAL_BUILDDIR)/specs/index.html: $(call manual-deps,specs)
$(call build-manual,specs,html)
 
-$(MANUAL_BUILDDIR)/system/index.html: $(call manual-deps,system)
+$(MANUAL_BUILDDIR)/system/index.html: $(call manual-deps,system) 
$(SRC_PATH)/hmp-commands.hx
$(call build-manual,system,html)
 
 $(MANUAL_BUILDDIR)/tools/index.html: $(call manual-deps,tools) 
$(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/docs/qemu-option-trace.rst.inc
diff --git a/docs/system/monitor.rst b/docs/system/monitor.rst
index 482f391f327..52b0f18cfec 100644
--- a/docs/system/monitor.rst
+++ b/docs/system/monitor.rst
@@ -14,8 +14,16 @@ You can use it to:
 
 -  Inspect the VM state without an external debugger.
 
+Commands
+
+
+The following commands are available:
+
+.. hxtool-doc:: hmp-commands.hx
+
 ..
-  The commands section goes here once it's converted from Texinfo to RST.
+  TODO: convert and reference hmp-commands-info.hx
+
 
 Integer expressions
 ~~~
diff --git a/hmp-commands.hx b/hmp-commands.hx
index dc23185de43..9aa59f52312 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -23,6 +23,10 @@ STEXI
 @findex help
 Show the help for all commands or just for command @var{cmd}.
 ETEXI
+SRST
+``help`` or ``?`` [*cmd*]
+  Show the help for all commands or just for command *cmd*.
+ERST
 
 {
 .name   = "commit",
@@ -42,6 +46,16 @@ the backing file, the backing file will not be truncated.  
If you want the
 backing file to match the size of the smaller snapshot, you can safely truncate
 it yourself once the commit operation successfully completes.
 ETEXI
+SRST
+``commit``
+  Commit changes to the disk images (if -snapshot is used) or backing files.
+  If the backing file is smaller than the snapshot, then the backing file
+  will be resized to be the same size as the snapshot.  If the snapshot is
+  smaller than the backing file, the backing file will not be truncated.
+  If you want the backing file to match the size of the smaller snapshot,
+  you can safely truncate it yourself once the commit operation successfully
+  completes.
+ERST
 
 {
 .name   = "q|quit",
@@ -56,6 +70,10 @@ STEXI
 @findex quit
 Quit the emulator.
 ETEXI
+SRST
+``q`` or ``quit``
+  Quit the emulator.
+ERST
 
 {
 .name   = "exit_preconfig",
@@ -75,6 +93,14 @@ and via the QMP monitor during the preconfig state. The 
command is only
 available during the preconfig state (i.e. when the --preconfig command
 line option was in use).
 ETEXI
+SRST
+``exit_preconfig``
+  This command makes QEMU exit the preconfig state and proceed with
+  VM initialization using configuration data provided on the command line
+  and via the QMP monitor during the preconfig state. The command is only
+  available during the preconfig state (i.e. when the --preconfig command
+  line option was in use).
+ERST
 
 {
 .name   = "block_resize",
@@ -92,6 +118,13 @@ action to see the updated size.  Resize to a lower size is 
supported,
 but should be used with extreme caution.  Note that this command only
 resizes image files, it can not resize block devices like LVM volumes.
 ETEXI
+SRST
+``block_resize``
+  Resize a block image while a guest is running.  Usually requires guest
+  action to see the updated size.  Resize to a lower size is supported,
+  but should be used with extreme caution.  Note that this command only
+  resizes image files, it can not resize block devices like LVM volumes.
+ERST
 
 {
 .name   = "block_stream",
@@ -106,6 +139,10 @@ STEXI
 @findex block_stream
 Copy data from a backing file into a block device.
 ETEXI
+SRST
+``block_stream``
+  Copy data from a backing file into a block device.
+ERST
 
 {
 .name   = "block_job_set_speed",
@@ -120,6 +157,10 @@ STEXI
 @findex block_job_set_speed
 Set maximum speed for a background block operation.
 ETEXI
+SRST
+``block_job_set_speed``
+  Set maximum speed for a background block operation.
+ERST
 
 {
 .name   = "block_job_cancel",
@@ -136,6 +177,10 @@ STEXI
 @findex block_job_cancel
 Stop an active background block operation (streaming, mirroring).
 ETEXI
+SRST

[PATCH v3 27/33] qemu-options.hx: Fix up the autogenerated rST

[Peter Maydell]

This commit contains hand-written fixes for some issues with the
autogenerated rST fragments in qemu-options.hx:

 * Sphinx complains about the UTF-8 art table in the documentation of
   the -drive option.  Replace it with a proper rST format table.

 * rST does not like definition list entries with no actual
   definition, but it is possible to work around this by putting a
   single escaped literal space as the definition line.

 * The "-g widthxheight" option documentation suffers particularly
   badly from losing the distinction between italics and fixed-width
   as a result of the auto conversion, so put it back in again.

 * The script missed some places that use the |qemu_system| etc
   macros and need to be marked up as parsed-literal blocks.

 * The script autogenerated an expanded out version of the
   contents of qemu-option-trace.texi; replace it with an
   qemu-option-trace.rst.inc include.

This is sufficient that we can enable inclusion of the
option documentation from invocation.rst.

Signed-off-by: Peter Maydell 
---
I opted not to try to reinstate all the italics-vs-fixed
distinctions in the option headings, in the interests
of expediency. I'm happy to go back and add this in later
once the whole conversion has landed but I think we are
better off doing a mostly-good-enough conversion quickly
and then refining it in-tree rather than having it out
of tree for a long time and getting hit by problems as
documentation is updated by other patches...
---
 Makefile   |   2 +-
 docs/system/invocation.rst |   2 +
 qemu-options.hx| 143 +
 3 files changed, 84 insertions(+), 63 deletions(-)

diff --git a/Makefile b/Makefile
index 98ef06ab8e8..f7d08997dbf 100644
--- a/Makefile
+++ b/Makefile
@@ -1071,7 +1071,7 @@ $(MANUAL_BUILDDIR)/interop/index.html: $(call 
manual-deps,interop)
 $(MANUAL_BUILDDIR)/specs/index.html: $(call manual-deps,specs)
$(call build-manual,specs,html)
 
-$(MANUAL_BUILDDIR)/system/index.html: $(call manual-deps,system) 
$(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/hmp-commands-info.hx
+$(MANUAL_BUILDDIR)/system/index.html: $(call manual-deps,system) 
$(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/hmp-commands-info.hx 
$(SRC_PATH)/qemu-options.hx
$(call build-manual,system,html)
 
 $(MANUAL_BUILDDIR)/tools/index.html: $(call manual-deps,tools) 
$(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/docs/qemu-option-trace.rst.inc
diff --git a/docs/system/invocation.rst b/docs/system/invocation.rst
index c112bcb45a4..d77dd13d4f7 100644
--- a/docs/system/invocation.rst
+++ b/docs/system/invocation.rst
@@ -10,6 +10,8 @@ Invocation
 disk_image is a raw hard disk image for IDE hard disk 0. Some targets do
 not need a disk image.
 
+.. hxtool-doc:: qemu-options.hx
+
 Device URL Syntax
 ~
 
diff --git a/qemu-options.hx b/qemu-options.hx
index 043e425b61e..f7061212ceb 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -448,7 +448,17 @@ cache, size is 10KB, policy is write-back, the cache Line 
size is 8 bytes:
 
 ETEXI
 SRST
-``-numa 
node[,mem=size][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]``;
 \ ``-numa 
node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]``;
 \ ``-numa dist,src=source,dst=destination,val=distance``; \ ``-numa 
cpu,node-id=node[,socket-id=x][,core-id=y][,thread-id=z]``; \ ``-numa 
hmat-lb,initiator=node,target=node,hierarchy=hierarchy,data-type=tpye[,latency=lat][,bandwidth=bw]``;
 \ ``-numa 
hmat-cache,node-id=node,size=size,level=level[,associativity=str][,policy=str][,line=size]``
+``-numa 
node[,mem=size][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]``
+  \ 
+``-numa 
node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=initiator]``
+  \
+``-numa dist,src=source,dst=destination,val=distance``
+  \ 
+``-numa cpu,node-id=node[,socket-id=x][,core-id=y][,thread-id=z]``
+  \ 
+``-numa 
hmat-lb,initiator=node,target=node,hierarchy=hierarchy,data-type=tpye[,latency=lat][,bandwidth=bw]``
+  \ 
+``-numa 
hmat-cache,node-id=node,size=size,level=level[,associativity=str][,policy=str][,line=size]``
 Define a NUMA node and assign RAM and VCPUs to it. Set the NUMA
 distance from a source node to a destination node. Set the ACPI
 Heterogeneous Memory Attributes for the given nodes.
@@ -687,7 +697,9 @@ driver=@var{driver},property=@var{prop},value=@var{value}.  
The
 longhand syntax works even when @var{driver} contains a dot.
 ETEXI
 SRST
-``-global driver.prop=value``; \ ``-global 
driver=driver,property=property,value=value``
+``-global driver.prop=value``
+  \ 
+``-global driver=driver,property=property,value=value``
 Set default value of driver's property prop to value, e.g.:
 
 .. parsed-literal::
@@ -788,7 +800,7 @@ SRST
 it. This only effects when boot priority is changed by bootindex
 options. The default is non-strict boot.
 
-::
+.. parsed-literal::
 
 # try to boot from network fi

[PATCH v3 29/33] docs: Generate qemu.1 manpage with Sphinx

[Peter Maydell]

Generate the qemu.1 manpage using Sphinx; we do this with a new
top-level rst source file which is just the skeleton of the manpage
and which includes .rst.inc fragments where it needs to incorporate
sections from the larger HTML manuals.

Signed-off-by: Peter Maydell 
---
 Makefile | 10 
 docs/system/conf.py  |  2 ++
 docs/system/qemu-manpage.rst | 45 
 3 files changed, 51 insertions(+), 6 deletions(-)
 create mode 100644 docs/system/qemu-manpage.rst

diff --git a/Makefile b/Makefile
index f7d08997dbf..ef10b9a031d 100644
--- a/Makefile
+++ b/Makefile
@@ -344,7 +344,8 @@ MANUAL_BUILDDIR := docs
 endif
 
 ifdef BUILD_DOCS
-DOCS=qemu-doc.html qemu-doc.txt qemu.1
+DOCS=qemu-doc.html qemu-doc.txt
+DOCS+=$(MANUAL_BUILDDIR)/system/qemu.1
 DOCS+=$(MANUAL_BUILDDIR)/tools/qemu-img.1
 DOCS+=$(MANUAL_BUILDDIR)/tools/qemu-nbd.8
 DOCS+=$(MANUAL_BUILDDIR)/interop/qemu-ga.8
@@ -856,7 +857,7 @@ install-doc: $(DOCS) install-sphinxdocs
$(INSTALL_DATA) docs/interop/qemu-qmp-ref.txt "$(DESTDIR)$(qemu_docdir)"
 ifdef CONFIG_POSIX
$(INSTALL_DIR) "$(DESTDIR)$(mandir)/man1"
-   $(INSTALL_DATA) qemu.1 "$(DESTDIR)$(mandir)/man1"
+   $(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu.1 
"$(DESTDIR)$(mandir)/man1"
$(INSTALL_DIR) "$(DESTDIR)$(mandir)/man7"
$(INSTALL_DATA) docs/interop/qemu-qmp-ref.7 "$(DESTDIR)$(mandir)/man7"
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu-block-drivers.7 
"$(DESTDIR)$(mandir)/man7"
@@ -1082,7 +1083,7 @@ $(MANUAL_BUILDDIR)/user/index.html: $(call 
manual-deps,user)
 
 $(call define-manpage-rule,interop,qemu-ga.8)
 
-$(call define-manpage-rule,system,qemu-block-drivers.7 qemu-cpu-models.7)
+$(call define-manpage-rule,system,qemu.1 qemu-block-drivers.7 
qemu-cpu-models.7)
 
 $(call define-manpage-rule,tools,\
qemu-img.1 qemu-nbd.8 qemu-trace-stap.1\
@@ -1109,9 +1110,6 @@ docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
 docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi-doc.texi
@cp -p $< $@
 
-qemu.1: qemu-doc.texi qemu-options.texi qemu-monitor.texi 
qemu-monitor-info.texi
-qemu.1: docs/system/qemu-option-trace.texi
-
 html: qemu-doc.html docs/interop/qemu-qmp-ref.html 
docs/interop/qemu-ga-ref.html sphinxdocs
 info: qemu-doc.info docs/interop/qemu-qmp-ref.info 
docs/interop/qemu-ga-ref.info
 pdf: qemu-doc.pdf docs/interop/qemu-qmp-ref.pdf docs/interop/qemu-ga-ref.pdf
diff --git a/docs/system/conf.py b/docs/system/conf.py
index 23cab3fb364..6251849fefc 100644
--- a/docs/system/conf.py
+++ b/docs/system/conf.py
@@ -17,6 +17,8 @@ html_theme_options['description'] = u'System Emulation 
User''s Guide'
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
 man_pages = [
+('qemu-manpage', 'qemu', u'QEMU User Documentation',
+ ['Fabrice Bellard'], 1),
 ('qemu-block-drivers', 'qemu-block-drivers',
  u'QEMU block drivers reference',
  ['Fabrice Bellard and the QEMU Project developers'], 7),
diff --git a/docs/system/qemu-manpage.rst b/docs/system/qemu-manpage.rst
new file mode 100644
index 000..e9a25d0680f
--- /dev/null
+++ b/docs/system/qemu-manpage.rst
@@ -0,0 +1,45 @@
+:orphan:
+
+..
+   This file is the skeleton for the qemu.1 manpage. It mostly
+   should simply include the .rst.inc files corresponding to the
+   parts of the documentation that go in the manpage as well as the
+   HTML manual.
+
+Title
+=
+
+Synopsis
+
+
+.. parsed-literal::
+
+   |qemu_system| [options] [disk_image]
+
+Description
+---
+
+.. include:: target-i386-desc.rst.inc
+
+Options
+---
+
+disk_image is a raw hard disk image for IDE hard disk 0. Some targets do
+not need a disk image.
+
+.. hxtool-doc:: qemu-options.hx
+
+.. include:: keys.rst.inc
+
+.. include:: mux-chardev.rst.inc
+
+Notes
+-
+
+.. include:: device-url-syntax.rst.inc
+
+See also
+
+
+The HTML documentation of QEMU for more precise information and Linux
+user mode emulator invocation.
-- 
2.20.1

[PATCH v3 31/33] docs: Stop building qemu-doc

[Peter Maydell]

Stop building the old texinfo qemu-doc; all its contents are
now available in the Sphinx-generated manuals and manpages.

Signed-off-by: Peter Maydell 
---
 docs/specs/ivshmem-spec.txt |  4 ++--
 Makefile| 39 -
 .gitignore  |  3 ---
 docs/index.html.in  |  1 -
 4 files changed, 6 insertions(+), 41 deletions(-)

diff --git a/docs/specs/ivshmem-spec.txt b/docs/specs/ivshmem-spec.txt
index 042f7eae225..1beb3a01ec3 100644
--- a/docs/specs/ivshmem-spec.txt
+++ b/docs/specs/ivshmem-spec.txt
@@ -38,8 +38,8 @@ There are two basic configurations:
   Interrupts are message-signaled (MSI-X).  vectors=N configures the
   number of vectors to use.
 
-For more details on ivshmem device properties, see The QEMU Emulator
-User Documentation (qemu-doc.*).
+For more details on ivshmem device properties, see the QEMU Emulator
+user documentation.
 
 
 == The ivshmem PCI device's guest interface ==
diff --git a/Makefile b/Makefile
index ef10b9a031d..9d4b2241265 100644
--- a/Makefile
+++ b/Makefile
@@ -344,7 +344,6 @@ MANUAL_BUILDDIR := docs
 endif
 
 ifdef BUILD_DOCS
-DOCS=qemu-doc.html qemu-doc.txt
 DOCS+=$(MANUAL_BUILDDIR)/system/qemu.1
 DOCS+=$(MANUAL_BUILDDIR)/tools/qemu-img.1
 DOCS+=$(MANUAL_BUILDDIR)/tools/qemu-nbd.8
@@ -768,10 +767,6 @@ distclean: clean
rm -f $(SUBDIR_DEVICES_MAK)
rm -f po/*.mo tests/qemu-iotests/common.env
rm -f roms/seabios/config.mak roms/vgabios/config.mak
-   rm -f qemu-doc.info qemu-doc.aux qemu-doc.cp qemu-doc.cps
-   rm -f qemu-doc.fn qemu-doc.fns qemu-doc.info qemu-doc.ky qemu-doc.kys
-   rm -f qemu-doc.log qemu-doc.pdf qemu-doc.pg qemu-doc.toc qemu-doc.tp
-   rm -f qemu-doc.vr qemu-doc.txt
rm -f qemu-plugins-ld.symbols qemu-plugins-ld64.symbols
rm -f config.log
rm -f linux-headers/asm
@@ -851,8 +846,6 @@ install-sphinxdocs: sphinxdocs
 install-doc: $(DOCS) install-sphinxdocs
$(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)"
$(INSTALL_DATA) $(MANUAL_BUILDDIR)/index.html "$(DESTDIR)$(qemu_docdir)"
-   $(INSTALL_DATA) qemu-doc.html "$(DESTDIR)$(qemu_docdir)"
-   $(INSTALL_DATA) qemu-doc.txt "$(DESTDIR)$(qemu_docdir)"
$(INSTALL_DATA) docs/interop/qemu-qmp-ref.html 
"$(DESTDIR)$(qemu_docdir)"
$(INSTALL_DATA) docs/interop/qemu-qmp-ref.txt "$(DESTDIR)$(qemu_docdir)"
 ifdef CONFIG_POSIX
@@ -1110,34 +1103,10 @@ docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
 docs/interop/qemu-ga-qapi.texi: qga/qapi-generated/qga-qapi-doc.texi
@cp -p $< $@
 
-html: qemu-doc.html docs/interop/qemu-qmp-ref.html 
docs/interop/qemu-ga-ref.html sphinxdocs
-info: qemu-doc.info docs/interop/qemu-qmp-ref.info 
docs/interop/qemu-ga-ref.info
-pdf: qemu-doc.pdf docs/interop/qemu-qmp-ref.pdf docs/interop/qemu-ga-ref.pdf
-txt: qemu-doc.txt docs/interop/qemu-qmp-ref.txt docs/interop/qemu-ga-ref.txt
-
-qemu-doc.html qemu-doc.info qemu-doc.pdf qemu-doc.txt: \
-   qemu-options.texi \
-   qemu-monitor.texi \
-   qemu-monitor-info.texi \
-docs/system/quickstart.texi \
-docs/system/invocation.texi \
-docs/system/keys.texi \
-docs/system/mux-chardev.texi \
-docs/system/monitor.texi \
-docs/system/cpu-models-x86.texi \
-docs/system/images.texi \
-docs/system/net.texi \
-docs/system/usb.texi \
-docs/system/ivshmem.texi \
-docs/system/linuxboot.texi \
-docs/system/vnc-security.texi \
-docs/system/tls.texi \
-docs/system/gdb.texi \
-docs/system/build-platforms.texi \
-docs/system/license.texi \
-   docs/system/cpu-models-x86.texi docs/system/cpu-models-mips.texi \
-   docs/system/deprecated.texi docs/system/qemu-option-trace.texi \
-   docs/system/security.texi
+html: docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html sphinxdocs
+info: docs/interop/qemu-qmp-ref.info docs/interop/qemu-ga-ref.info
+pdf: docs/interop/qemu-qmp-ref.pdf docs/interop/qemu-ga-ref.pdf
+txt: docs/interop/qemu-qmp-ref.txt docs/interop/qemu-ga-ref.txt
 
 docs/interop/qemu-ga-ref.dvi docs/interop/qemu-ga-ref.html \
 docs/interop/qemu-ga-ref.info docs/interop/qemu-ga-ref.pdf \
diff --git a/.gitignore b/.gitignore
index bc0a035f9cc..0c5af83aa74 100644
--- a/.gitignore
+++ b/.gitignore
@@ -46,9 +46,6 @@
 !/qapi/qapi-visit-core.c
 /qapi/qapi-visit.[ch]
 /qapi/qapi-doc.texi
-/qemu-doc.html
-/qemu-doc.info
-/qemu-doc.txt
 /qemu-edid
 /qemu-img
 /qemu-nbd
diff --git a/docs/index.html.in b/docs/index.html.in
index a576ace8a27..cc19aad2ec5 100644
--- a/docs/index.html.in
+++ b/docs/index.html.in
@@ -7,7 +7,6 @@
 
 QEMU @@VERSION@@ Documentation
 
-User Documentation
 QMP Reference Manual
 Guest Agent Protocol 
Reference
 System Emulation Management and 
Interoperability Guide
-- 
2.20.1

Re: [PATCH v2 0/2] hw/arm/xilinx_zynq: Fix USB port instantiation

[Peter Maydell]

On Thu, 20 Feb 2020 at 15:05, Peter Maydell  wrote:
>
> On Sat, 15 Feb 2020 at 12:23, Guenter Roeck  wrote:
> >
> > USB ports on Xilinx Zync must be instantiated as TYPE_CHIPIDEA to work.
> > Linux expects and checks various chipidea registers, which do not exist
> > with the basic ehci emulation. This patch series fixes the problem.
> >
> > The first patch in the series fixes the actual problem.
> >
> > The second patch removes the now obsolete explicit Xilinx
> > support from the EHCI code.
> >
> > v2: Introduced summary
> >
> > 
> > Guenter Roeck (2):
> >   hw/arm/xilinx_zynq: Fix USB port instantiation
> >   hw/usb/hcd-ehci-sysbus: Remove obsolete xlnx,ps7-usb class
>
> Xilinx folks -- could you provide a reviewed-by or acked-by
> for this series, please?

No? Oh, well, applied to target-arm.next anyway.

thanks
-- PMM

Re: [PULL 00/15] Pull migration patches

[Peter Maydell]

On Fri, 28 Feb 2020 at 09:25, Juan Quintela  wrote:
>
> The following changes since commit 8b6b68e05b43f976714ca1d2afe01a64e1d82cba:
>
>   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging 
> (2020-02-27 19:15:15 +)
>
> are available in the Git repository at:
>
>   https://github.com/juanquintela/qemu.git tags/pull-migration-pull-request
>
> for you to fetch changes up to f51d0b4178738bba87d796eba7444f6cdb3aa0fd:
>
>   savevm: Don't call colo_init_ram_cache twice (2020-02-28 10:13:54 +0100)
>
> 
> Migration pull request
>
> - cleanup redundant initilazation (chen)
> - fix memleak in test_load_qlist (chen)
> - several colo fixes (hailiang)
> - multifd compression (juan)
> - release list after use (pan)
> - rename BLOCK_SIZE (stefan)
>
> Please apply.
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
for any user-visible changes.

-- PMM

Re: [PATCH 0/4] tests/acceptance: nSeries and integrator/CP tests

[Peter Maydell]

On Tue, 25 Feb 2020 at 17:25, Philippe Mathieu-Daudé  wrote:
>
> This series collects patches adding tests for arm machines.
> All patches are reviewed.
>
> Philippe Mathieu-Daudé (2):
>   tests/acceptance: Extract boot_integratorcp() from test_integratorcp()
>   tests/acceptance/integratorcp: Verify Tux is displayed on framebuffer
>
> Thomas Huth (2):
>   tests/acceptance: Add a test for the N800 and N810 arm machines
>   tests/acceptance: Add a test for the integratorcp arm machine
>
>  MAINTAINERS  |  2 +
>  tests/acceptance/machine_arm_integratorcp.py | 99 
>  tests/acceptance/machine_arm_n8x0.py | 49 ++
>  3 files changed, 150 insertions(+)
>  create mode 100644 tests/acceptance/machine_arm_integratorcp.py
>  create mode 100644 tests/acceptance/machine_arm_n8x0.py



Applied to target-arm.next, thanks.

-- PMM

Re: [PATCH] tests/acceptance/ppc_prep_40p: Use cdn.netbsd.org hostname

[Alex Bennée]

Philippe Mathieu-Daudé  writes:

> Use NetBSD content delivery network to get faster downloads.

Even with this patch I get failures on my big dev box:

 (48/67) 
tests/acceptance/ppc_prep_40p.py:IbmPrep40pMachine.test_openbios_and_netbsd: 
INTERRUPTED: Failed to fetch NetBSD-7.1.2-prep.iso.\nRunner error occurred: 
Timeout reached\nOriginal status: ERROR\n{'name': 
'48-tests/acceptance/ppc_prep_40p.py:IbmPrep40pMachine.test_openbios_and_netbsd',
 'logdir': '/home/alex/lsrc/qemu.git/builds/all/tests/results/jo... (60.31 s)

I think ultimately a whole ISO download is just too much for an
acceptance test.

>
> Suggested-by: Kamil Rytarowski 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  tests/acceptance/ppc_prep_40p.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tests/acceptance/ppc_prep_40p.py 
> b/tests/acceptance/ppc_prep_40p.py
> index efe06037ba..6729d96f5e 100644
> --- a/tests/acceptance/ppc_prep_40p.py
> +++ b/tests/acceptance/ppc_prep_40p.py
> @@ -34,7 +34,7 @@ def test_factory_firmware_and_netbsd(self):
>  '7020-40p/P12H0456.IMG')
>  bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03'
>  bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash)
> -drive_url = ('https://ftp.netbsd.org/pub/NetBSD/NetBSD-archive/'
> +drive_url = ('https://cdn.netbsd.org/pub/NetBSD/NetBSD-archive/'
>   'NetBSD-4.0/prep/installation/floppy/generic_com0.fs')
>  drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb'
>  drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash)
> @@ -67,7 +67,7 @@ def test_openbios_and_netbsd(self):
>  :avocado: tags=arch:ppc
>  :avocado: tags=machine:40p
>  """
> -drive_url = ('https://ftp.netbsd.org/pub/NetBSD/iso/7.1.2/'
> +drive_url = ('https://cdn.netbsd.org/pub/NetBSD/iso/7.1.2/'
>   'NetBSD-7.1.2-prep.iso')
>  drive_hash = 'ac6fa2707d888b36d6fa64de6e7fe48e'
>  drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash,


-- 
Alex Bennée

[PATCH v2 2/5] vfio/pci: Use kvm_irqchip_add_irqfd_notifier_gsi() for irqfds

[Peter Xu]

VFIO is currently the only one left that is not using the generic
function (kvm_irqchip_add_irqfd_notifier_gsi()) to register irqfds.
Let VFIO use the common framework too.

Follow up patches will introduce extra features for kvm irqfd, so that
VFIO can easily leverage that after the switch.

Reviewed-by: Eric Auger 
Reviewed-by: Cornelia Huck 
Signed-off-by: Peter Xu 
---
 hw/vfio/pci.c | 37 +++--
 1 file changed, 15 insertions(+), 22 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 98e0e0c994..09703362df 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -115,11 +115,7 @@ static void vfio_intx_eoi(VFIODevice *vbasedev)
 static void vfio_intx_enable_kvm(VFIOPCIDevice *vdev, Error **errp)
 {
 #ifdef CONFIG_KVM
-struct kvm_irqfd irqfd = {
-.fd = event_notifier_get_fd(&vdev->intx.interrupt),
-.gsi = vdev->intx.route.irq,
-.flags = KVM_IRQFD_FLAG_RESAMPLE,
-};
+int irq_fd = event_notifier_get_fd(&vdev->intx.interrupt);
 Error *err = NULL;
 
 if (vdev->no_kvm_intx || !kvm_irqfds_enabled() ||
@@ -141,7 +137,7 @@ static void vfio_intx_enable_kvm(VFIOPCIDevice *vdev, Error 
**errp)
 }
 
 /* Get to a known interrupt state */
-qemu_set_fd_handler(irqfd.fd, NULL, NULL, vdev);
+qemu_set_fd_handler(irq_fd, NULL, NULL, vdev);
 vfio_mask_single_irqindex(&vdev->vbasedev, VFIO_PCI_INTX_IRQ_INDEX);
 vdev->intx.pending = false;
 pci_irq_deassert(&vdev->pdev);
@@ -152,17 +148,18 @@ static void vfio_intx_enable_kvm(VFIOPCIDevice *vdev, 
Error **errp)
 goto fail;
 }
 
-/* KVM triggers it, VFIO listens for it */
-irqfd.resamplefd = event_notifier_get_fd(&vdev->intx.unmask);
-
-if (kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd)) {
+if (kvm_irqchip_add_irqfd_notifier_gsi(kvm_state,
+   &vdev->intx.interrupt,
+   &vdev->intx.unmask,
+   vdev->intx.route.irq)) {
 error_setg_errno(errp, errno, "failed to setup resample irqfd");
 goto fail_irqfd;
 }
 
 if (vfio_set_irq_signaling(&vdev->vbasedev, VFIO_PCI_INTX_IRQ_INDEX, 0,
VFIO_IRQ_SET_ACTION_UNMASK,
-   irqfd.resamplefd, &err)) {
+   event_notifier_get_fd(&vdev->intx.unmask),
+   &err)) {
 error_propagate(errp, err);
 goto fail_vfio;
 }
@@ -177,12 +174,12 @@ static void vfio_intx_enable_kvm(VFIOPCIDevice *vdev, 
Error **errp)
 return;
 
 fail_vfio:
-irqfd.flags = KVM_IRQFD_FLAG_DEASSIGN;
-kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd);
+kvm_irqchip_remove_irqfd_notifier_gsi(kvm_state, &vdev->intx.interrupt,
+  vdev->intx.route.irq);
 fail_irqfd:
 event_notifier_cleanup(&vdev->intx.unmask);
 fail:
-qemu_set_fd_handler(irqfd.fd, vfio_intx_interrupt, NULL, vdev);
+qemu_set_fd_handler(irq_fd, vfio_intx_interrupt, NULL, vdev);
 vfio_unmask_single_irqindex(&vdev->vbasedev, VFIO_PCI_INTX_IRQ_INDEX);
 #endif
 }
@@ -190,12 +187,6 @@ fail:
 static void vfio_intx_disable_kvm(VFIOPCIDevice *vdev)
 {
 #ifdef CONFIG_KVM
-struct kvm_irqfd irqfd = {
-.fd = event_notifier_get_fd(&vdev->intx.interrupt),
-.gsi = vdev->intx.route.irq,
-.flags = KVM_IRQFD_FLAG_DEASSIGN,
-};
-
 if (!vdev->intx.kvm_accel) {
 return;
 }
@@ -209,7 +200,8 @@ static void vfio_intx_disable_kvm(VFIOPCIDevice *vdev)
 pci_irq_deassert(&vdev->pdev);
 
 /* Tell KVM to stop listening for an INTx irqfd */
-if (kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd)) {
+if (kvm_irqchip_remove_irqfd_notifier_gsi(kvm_state, &vdev->intx.interrupt,
+  vdev->intx.route.irq)) {
 error_report("vfio: Error: Failed to disable INTx irqfd: %m");
 }
 
@@ -217,7 +209,8 @@ static void vfio_intx_disable_kvm(VFIOPCIDevice *vdev)
 event_notifier_cleanup(&vdev->intx.unmask);
 
 /* QEMU starts listening for interrupt events. */
-qemu_set_fd_handler(irqfd.fd, vfio_intx_interrupt, NULL, vdev);
+qemu_set_fd_handler(event_notifier_get_fd(&vdev->intx.interrupt),
+vfio_intx_interrupt, NULL, vdev);
 
 vdev->intx.kvm_accel = false;
 
-- 
2.24.1

[PATCH v2 0/5] vfio/pci: Fix up breakage against split irqchip and INTx

[Peter Xu]

v2:
- pick tags
- don't register resamplefd with KVM kernel when the userspace
  resamplefd path is enabled (should enable fast path on new kernels)
- fix resamplefd mem leak
- fix commit message of patch 4 [Eric]
- let kvm_resample_fd_notify() return a boolean, skip ioapic check if
  returned true
- more comments here and there in the code to state the fact that
  userspace ioapic irr & remote-irr are bypassed [Paolo]

VFIO INTx is not working with split irqchip.  On new kernels KVM_IRQFD
will directly fail with resamplefd attached so QEMU will automatically
fallback to the INTx slow path.  However on old kernels it's still
broken.

Only until recently I noticed that this could also break PXE boot for
assigned NICs [1].  My wild guess is that the PXE ROM will be mostly
using INTx as well, which means we can't bypass that even if we
enables MSI for the guest kernel.

This series tries to first fix this issue function-wise, then speed up
for the INTx again with resamplefd (mostly following the ideas
proposed by Paolo one year ago [2]).  My TCP_RR test shows that:

  - Before this series: this is broken, no number to show

  - After patch 1 (enable slow path): get 63% perf comparing to full
kernel irqchip

  - After whole series (enable fast path partly, irq injection will be
the same as fast path, however userspace needs to intercept for
EOI broadcast to resamplefd, though should still be faster than
the MMIO trick for intx eoi): get 93% perf comparing to full
kernel irqchip, which is a 46% performance boost

I think we can consider to apply patch 1 even sooner than the rest of
the series to unbreak intx+split first.

The whole test matrix for reference:

  |--+-+---+--+|
  | IRQ type | irqchip | TCP_STREAM (Gbps) | TCP_RR (pps) | note   |
  |--+-+---+--+|
  | msi  | on  |  9.39 |17567 ||
  | nomsi| on  |  9.29 |14056 ||
  | msi  | split   |  9.36 |17330 ||
  | nomsi| split   | / |/ | currently broken   |
  | nomsi| split   |  8.98 | 8977 | after patch 1  |
  | nomsi| split   |  9.21 |13142 | after whole series |
  |--+-+---+--+|

Any review comment is welcomed.  Thanks,

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1786404
[2] https://patchwork.kernel.org/patch/10738541/#22609933

Peter Xu (5):
  vfio/pci: Disable INTx fast path if using split irqchip
  vfio/pci: Use kvm_irqchip_add_irqfd_notifier_gsi() for irqfds
  KVM: Pass EventNotifier into kvm_irqchip_assign_irqfd
  KVM: Kick resamplefd for split kernel irqchip
  Revert "vfio/pci: Disable INTx fast path if using split irqchip"

 accel/kvm/kvm-all.c| 101 +
 accel/kvm/trace-events |   1 +
 hw/intc/ioapic.c   |  23 +-
 hw/vfio/pci.c  |  37 ++-
 include/sysemu/kvm.h   |   7 +++
 5 files changed, 137 insertions(+), 32 deletions(-)

-- 
2.24.1

[PATCH v2 1/5] vfio/pci: Disable INTx fast path if using split irqchip

[Peter Xu]

It's currently broken.  Let's use the slow path to at least make it
functional.

Tested-by: Eric Auger 
Reviewed-by: Eric Auger 
Signed-off-by: Peter Xu 
---
 hw/vfio/pci.c | 12 
 1 file changed, 12 insertions(+)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 5e75a95129..98e0e0c994 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -128,6 +128,18 @@ static void vfio_intx_enable_kvm(VFIOPCIDevice *vdev, 
Error **errp)
 return;
 }
 
+if (kvm_irqchip_is_split()) {
+/*
+ * VFIO INTx is currently not working with split kernel
+ * irqchip for level triggered interrupts.  Go the slow path
+ * as long as split is enabled so we can be at least
+ * functional (even with poor performance).
+ *
+ * TODO: Remove this after all things fixed up.
+ */
+return;
+}
+
 /* Get to a known interrupt state */
 qemu_set_fd_handler(irqfd.fd, NULL, NULL, vdev);
 vfio_mask_single_irqindex(&vdev->vbasedev, VFIO_PCI_INTX_IRQ_INDEX);
-- 
2.24.1

[PATCH v2 4/5] KVM: Kick resamplefd for split kernel irqchip

[Peter Xu]

This is majorly only for X86 because that's the only one that supports
split irqchip for now.

When the irqchip is split, we face a dilemma that KVM irqfd will be
enabled, however the slow irqchip is still running in the userspace.
It means that the resamplefd in the kernel irqfds won't take any
effect and it will miss to ack INTx interrupts on EOIs.

One example is split irqchip with VFIO INTx, which will break if we
use the VFIO INTx fast path.

This patch can potentially supports the VFIO fast path again for INTx,
that the IRQ delivery will still use the fast path, while we don't
need to trap MMIOs in QEMU for the device to emulate the EIOs (see the
callers of vfio_eoi() hook).  However the EOI of the INTx will still
need to be done from the userspace by caching all the resamplefds in
QEMU and kick properly for IOAPIC EOI broadcast.

This is tricky because in this case the userspace ioapic irr &
remote-irr will be bypassed.  However such a change will greatly boost
performance for assigned devices using INTx irqs (TCP_RR boosts 46%
after this patch applied).

When the userspace is responsible for the resamplefd kickup, don't
register it on the kvm_irqfd anymore, because on newer kernels (after
commit 654f1f13ea56, 5.2+) the KVM_IRQFD will fail if with both split
irqchip and resamplefd.  This will make sure that the fast path will
work for all supported kernels.

https://patchwork.kernel.org/patch/10738541/#22609933

Suggested-by: Paolo Bonzini 
Signed-off-by: Peter Xu 
---
 accel/kvm/kvm-all.c| 85 +-
 accel/kvm/trace-events |  1 +
 hw/intc/ioapic.c   | 23 +++-
 include/sysemu/kvm.h   |  7 
 4 files changed, 112 insertions(+), 4 deletions(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index d49b74512a..89771ea114 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -159,9 +159,65 @@ static const KVMCapabilityInfo kvm_required_capabilites[] 
= {
 static NotifierList kvm_irqchip_change_notifiers =
 NOTIFIER_LIST_INITIALIZER(kvm_irqchip_change_notifiers);
 
+struct KVMResampleFd {
+int gsi;
+EventNotifier *resample_event;
+QLIST_ENTRY(KVMResampleFd) node;
+};
+typedef struct KVMResampleFd KVMResampleFd;
+
+/*
+ * Only used with split irqchip where we need to do the resample fd
+ * kick for the kernel from userspace.
+ */
+static QLIST_HEAD(, KVMResampleFd) kvm_resample_fd_list =
+QLIST_HEAD_INITIALIZER(kvm_resample_fd_list);
+
 #define kvm_slots_lock(kml)  qemu_mutex_lock(&(kml)->slots_lock)
 #define kvm_slots_unlock(kml)qemu_mutex_unlock(&(kml)->slots_lock)
 
+static inline void kvm_resample_fd_remove(int gsi)
+{
+KVMResampleFd *rfd;
+
+QLIST_FOREACH(rfd, &kvm_resample_fd_list, node) {
+if (rfd->gsi == gsi) {
+QLIST_REMOVE(rfd, node);
+g_free(rfd);
+break;
+}
+}
+}
+
+static inline void kvm_resample_fd_insert(int gsi, EventNotifier *event)
+{
+KVMResampleFd *rfd = g_new0(KVMResampleFd, 1);
+
+rfd->gsi = gsi;
+rfd->resample_event = event;
+
+QLIST_INSERT_HEAD(&kvm_resample_fd_list, rfd, node);
+}
+
+bool kvm_resample_fd_notify(int gsi)
+{
+KVMResampleFd *rfd;
+
+if (!kvm_irqchip_is_split()) {
+return false;
+}
+
+QLIST_FOREACH(rfd, &kvm_resample_fd_list, node) {
+if (rfd->gsi == gsi) {
+event_notifier_set(rfd->resample_event);
+trace_kvm_resample_fd_notify(gsi);
+return true;
+}
+}
+
+return false;
+}
+
 int kvm_get_max_memslots(void)
 {
 KVMState *s = KVM_STATE(current_accel());
@@ -1642,8 +1698,33 @@ static int kvm_irqchip_assign_irqfd(KVMState *s, 
EventNotifier *event,
 };
 
 if (rfd != -1) {
-irqfd.flags |= KVM_IRQFD_FLAG_RESAMPLE;
-irqfd.resamplefd = rfd;
+assert(assign);
+if (kvm_irqchip_is_split()) {
+/*
+ * When the slow irqchip (e.g. IOAPIC) is in the
+ * userspace, KVM kernel resamplefd will not work because
+ * the EOI of the interrupt will be delivered to userspace
+ * instead, so the KVM kernel resamplefd kick will be
+ * skipped.  The userspace here mimics what the kernel
+ * provides with resamplefd, remember the resamplefd and
+ * kick it when we receive EOI of this IRQ.
+ *
+ * This is hackery because IOAPIC is mostly bypassed
+ * (except EOI broadcasts) when irqfd is used.  However
+ * this can bring much performance back for split irqchip
+ * with INTx IRQs (for VFIO, this gives 93% perf of the
+ * full fast path, which is 46% perf boost comparing to
+ * the INTx slow path).
+ */
+kvm_resample_fd_insert(virq, resample);
+} else {
+irqfd.flags |= KVM_IRQFD_FLAG_RESAMPLE;
+irqfd.resamplefd = rfd;
+}
+} else if (!assign) {
+if (

[PATCH v2 3/5] KVM: Pass EventNotifier into kvm_irqchip_assign_irqfd

[Peter Xu]

So that kvm_irqchip_assign_irqfd() can have access to the
EventNotifiers, especially the resample event.  It is needed in follow
up patch to cache and kick resamplefds from QEMU.

Reviewed-by: Eric Auger 
Signed-off-by: Peter Xu 
---
 accel/kvm/kvm-all.c | 16 ++--
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 439a4efe52..d49b74512a 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1628,9 +1628,13 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, 
MSIMessage msg,
 return kvm_update_routing_entry(s, &kroute);
 }
 
-static int kvm_irqchip_assign_irqfd(KVMState *s, int fd, int rfd, int virq,
+static int kvm_irqchip_assign_irqfd(KVMState *s, EventNotifier *event,
+EventNotifier *resample, int virq,
 bool assign)
 {
+int fd = event_notifier_get_fd(event);
+int rfd = resample ? event_notifier_get_fd(resample) : -1;
+
 struct kvm_irqfd irqfd = {
 .fd = fd,
 .gsi = virq,
@@ -1735,7 +1739,9 @@ int kvm_irqchip_add_hv_sint_route(KVMState *s, uint32_t 
vcpu, uint32_t sint)
 return -ENOSYS;
 }
 
-static int kvm_irqchip_assign_irqfd(KVMState *s, int fd, int virq, bool assign)
+static int kvm_irqchip_assign_irqfd(KVMState *s, EventNotifier *event,
+EventNotifier *resample, int virq,
+bool assign)
 {
 abort();
 }
@@ -1749,15 +1755,13 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, 
MSIMessage msg)
 int kvm_irqchip_add_irqfd_notifier_gsi(KVMState *s, EventNotifier *n,
EventNotifier *rn, int virq)
 {
-return kvm_irqchip_assign_irqfd(s, event_notifier_get_fd(n),
-   rn ? event_notifier_get_fd(rn) : -1, virq, true);
+return kvm_irqchip_assign_irqfd(s, n, rn, virq, true);
 }
 
 int kvm_irqchip_remove_irqfd_notifier_gsi(KVMState *s, EventNotifier *n,
   int virq)
 {
-return kvm_irqchip_assign_irqfd(s, event_notifier_get_fd(n), -1, virq,
-   false);
+return kvm_irqchip_assign_irqfd(s, n, NULL, virq, false);
 }
 
 int kvm_irqchip_add_irqfd_notifier(KVMState *s, EventNotifier *n,
-- 
2.24.1

[PATCH v2 5/5] Revert "vfio/pci: Disable INTx fast path if using split irqchip"

[Peter Xu]

With the resamplefd list introduced, we can savely enable VFIO INTx
fast path again with split irqchip so it can still be faster than the
complete slow path.

Signed-off-by: Peter Xu 
---
 hw/vfio/pci.c | 12 
 1 file changed, 12 deletions(-)

diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 09703362df..1c0aa27386 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -124,18 +124,6 @@ static void vfio_intx_enable_kvm(VFIOPCIDevice *vdev, 
Error **errp)
 return;
 }
 
-if (kvm_irqchip_is_split()) {
-/*
- * VFIO INTx is currently not working with split kernel
- * irqchip for level triggered interrupts.  Go the slow path
- * as long as split is enabled so we can be at least
- * functional (even with poor performance).
- *
- * TODO: Remove this after all things fixed up.
- */
-return;
-}
-
 /* Get to a known interrupt state */
 qemu_set_fd_handler(irq_fd, NULL, NULL, vdev);
 vfio_mask_single_irqindex(&vdev->vbasedev, VFIO_PCI_INTX_IRQ_INDEX);
-- 
2.24.1

Re: [PATCH v4 1/7] target/arm: Improve masking of HCR RES0 bits

[Peter Maydell]

On Tue, 25 Feb 2020 at 18:08, Richard Henderson
 wrote:
>
> Don't merely start with v8.0, handle v7VE as well.
> Notice writes from aarch32 mode, and the bits that
> ought not be settable from there.
>
> Suggested-by: Peter Maydell 
> Signed-off-by: Richard Henderson 
> ---
>  target/arm/helper.c | 17 +++--
>  1 file changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/target/arm/helper.c b/target/arm/helper.c
> index 79db169e04..d65160fdb3 100644
> --- a/target/arm/helper.c
> +++ b/target/arm/helper.c
> @@ -5089,8 +5089,13 @@ static const ARMCPRegInfo el3_no_el2_v8_cp_reginfo[] = 
> {
>  static void hcr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t 
> value)
>  {
>  ARMCPU *cpu = env_archcpu(env);
> -/* Begin with bits defined in base ARMv8.0.  */
> -uint64_t valid_mask = MAKE_64BIT_MASK(0, 34);
> +uint64_t valid_mask;
> +
> +if (arm_feature(env, ARM_FEATURE_V8)) {
> +valid_mask = MAKE_64BIT_MASK(0, 34);  /* ARMv8.0 */
> +} else {
> +valid_mask = MAKE_64BIT_MASK(0, 28);  /* ARMv7VE */
> +}
>
>  if (arm_feature(env, ARM_FEATURE_EL3)) {
>  valid_mask &= ~HCR_HCD;
> @@ -5114,6 +5119,14 @@ static void hcr_write(CPUARMState *env, const 
> ARMCPRegInfo *ri, uint64_t value)
>  valid_mask |= HCR_API | HCR_APK;
>  }
>
> +if (ri->state == ARM_CP_STATE_AA32) {
> +/*
> + * Writes from aarch32 mode have more RES0 bits.
> + * This includes TDZ, RW, E2H, and more.
> + */
> +valid_mask &= ~0xff80ff8c9000ull;
> +}

Isn't bit HCR2 bit 16 (aka bit 32+16==48 here) also RES0 from AArch32 ?
I suppose it's RES0 from AArch64 too, but as far as what we've
implemented goes so are a bunch of other bits.

I'm not really a fan of the hex-number here either, given we
have HCR_* constants.

thanks
-- PMM

Re: [PATCH v4 2/7] target/arm: Honor the HCR_EL2.{TVM,TRVM} bits

[Peter Maydell]

On Tue, 25 Feb 2020 at 18:08, Richard Henderson
 wrote:
>
> These bits trap EL1 access to various virtual memory controls.
>
> Buglink: https://bugs.launchpad.net/bugs/1855072
> Signed-off-by: Richard Henderson 
> ---
> v2: Include TTBCR.
> v4: Include not_v8_cp_reginfo, lpae_cp_reginfo, CONTEXTIDR_S;
> exclude not_v7_cp_reginfo (pmm).

Reviewed-by: Peter Maydell 

thanks
-- PMM