Re: [PATCH] Resolves: https://gitlab.com/qemu-project/qemu/-/issues/542

[Thomas Huth]

On 12/03/2022 09.26, zzl wrote:

---
  hw/audio/intel-hda.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/audio/intel-hda.c b/hw/audio/intel-hda.c
index 5f8a878f20..ff32936cfd 100644
--- a/hw/audio/intel-hda.c
+++ b/hw/audio/intel-hda.c
@@ -328,7 +328,7 @@ static void intel_hda_corb_run(IntelHDAState *d)
  dprint(d, 2, "%s: corb ring empty\n", __func__);
  return;
  }
-if (d->rirb_count == d->rirb_cnt) {
+if (d->rirb_count >= d->rirb_cnt) {
  dprint(d, 2, "%s: rirb count reached\n", __func__);
  return;
  }


 Hi!

Please see https://www.qemu.org/docs/master/devel/submitting-a-patch.html 
for information how to correclty submit a patch. Especially you need to 
supply a "Signed-off-by" line, too, and please use a proper subject for the 
mail and place the "Resolves:" line into the patch description instead.


Apart from that, the bug ticket already links a patch series with a proposed 
fix ... could you elaborate why this needs a different approach?


 Thomas

Re: [PATCH 3/3] tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

[Christian Borntraeger]

Am 11.03.22 um 21:32 schrieb Richard Henderson:

On 3/11/22 10:49, Ilya Leoshkevich wrote:

+    size_t length = 0x10006;
+    unsigned char *buf;
+    int i;
+
+    buf = mmap(NULL, length, PROT_READ | PROT_WRITE | PROT_EXEC,
+   MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+    assert(buf != MAP_FAILED);


I'm thinking exit success here, as such a large allocation may well fail 
depending on the host.


What about using MAP_NORESERVE ?

[PATCH] i386/cpu: Remove the deprecated cpu model 'Icelake-Client'

[Robert Hoo]

Icelake, is the codename for Intel 3rd generation Xeon Scalable server
processors. There isn't ever client variants. This "Icelake-Client" CPU
model was added wrongly and imaginarily.

It has been deprecated since v5.2, now it's time to remove it completely
from code.

Signed-off-by: Robert Hoo 
---
Note: This patch is a reword, rebase and resend of
https://lore.kernel.org/qemu-devel/1619660147-136679-1-git-send-email-robert...@linux.intel.com/
---
 docs/about/deprecated.rst   |   6 --
 docs/about/removed-features.rst |   6 ++
 target/i386/cpu.c   | 122 
 3 files changed, 6 insertions(+), 128 deletions(-)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index cf02ef6821..1adc762958 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -289,12 +289,6 @@ from Linux upstream kernel, declare it deprecated.
 System emulator CPUS
 
 
-``Icelake-Client`` CPU Model (since 5.2)
-
-
-``Icelake-Client`` CPU Models are deprecated. Use ``Icelake-Server`` CPU
-Models instead.
-
 MIPS ``I7200`` CPU Model (since 5.2)
 
 
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 4b831ea291..3fb5b28518 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -538,6 +538,12 @@ Support for this CPU was removed from the upstream Linux 
kernel, and
 there is no available upstream toolchain to build binaries for it.
 Removed without replacement.
 
+x86 ``Icelake-Client`` CPU (removed in 7.1)
+'''
+
+There isn't ever Icelake Client CPU, it is some wrong and imaginary one.
+Use ``Icelake-Server`` instead.
+
 System emulator machines
 
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 6c7ef1099b..82f1cecf76 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -3238,128 +3238,6 @@ static const X86CPUDefinition builtin_x86_defs[] = {
 { /* end of list */ }
 }
 },
-{
-.name = "Icelake-Client",
-.level = 0xd,
-.vendor = CPUID_VENDOR_INTEL,
-.family = 6,
-.model = 126,
-.stepping = 0,
-.features[FEAT_1_EDX] =
-CPUID_VME | CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | CPUID_MMX |
-CPUID_CLFLUSH | CPUID_PSE36 | CPUID_PAT | CPUID_CMOV | CPUID_MCA |
-CPUID_PGE | CPUID_MTRR | CPUID_SEP | CPUID_APIC | CPUID_CX8 |
-CPUID_MCE | CPUID_PAE | CPUID_MSR | CPUID_TSC | CPUID_PSE |
-CPUID_DE | CPUID_FP87,
-.features[FEAT_1_ECX] =
-CPUID_EXT_AVX | CPUID_EXT_XSAVE | CPUID_EXT_AES |
-CPUID_EXT_POPCNT | CPUID_EXT_X2APIC | CPUID_EXT_SSE42 |
-CPUID_EXT_SSE41 | CPUID_EXT_CX16 | CPUID_EXT_SSSE3 |
-CPUID_EXT_PCLMULQDQ | CPUID_EXT_SSE3 |
-CPUID_EXT_TSC_DEADLINE_TIMER | CPUID_EXT_FMA | CPUID_EXT_MOVBE |
-CPUID_EXT_PCID | CPUID_EXT_F16C | CPUID_EXT_RDRAND,
-.features[FEAT_8000_0001_EDX] =
-CPUID_EXT2_LM | CPUID_EXT2_RDTSCP | CPUID_EXT2_NX |
-CPUID_EXT2_SYSCALL,
-.features[FEAT_8000_0001_ECX] =
-CPUID_EXT3_ABM | CPUID_EXT3_LAHF_LM | CPUID_EXT3_3DNOWPREFETCH,
-.features[FEAT_8000_0008_EBX] =
-CPUID_8000_0008_EBX_WBNOINVD,
-.features[FEAT_7_0_EBX] =
-CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_BMI1 |
-CPUID_7_0_EBX_HLE | CPUID_7_0_EBX_AVX2 | CPUID_7_0_EBX_SMEP |
-CPUID_7_0_EBX_BMI2 | CPUID_7_0_EBX_ERMS | CPUID_7_0_EBX_INVPCID |
-CPUID_7_0_EBX_RTM | CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_ADX |
-CPUID_7_0_EBX_SMAP,
-.features[FEAT_7_0_ECX] =
-CPUID_7_0_ECX_AVX512_VBMI | CPUID_7_0_ECX_UMIP | CPUID_7_0_ECX_PKU 
|
-CPUID_7_0_ECX_AVX512_VBMI2 | CPUID_7_0_ECX_GFNI |
-CPUID_7_0_ECX_VAES | CPUID_7_0_ECX_VPCLMULQDQ |
-CPUID_7_0_ECX_AVX512VNNI | CPUID_7_0_ECX_AVX512BITALG |
-CPUID_7_0_ECX_AVX512_VPOPCNTDQ,
-.features[FEAT_7_0_EDX] =
-CPUID_7_0_EDX_SPEC_CTRL | CPUID_7_0_EDX_SPEC_CTRL_SSBD,
-/* XSAVES is added in version 3 */
-.features[FEAT_XSAVE] =
-CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
-CPUID_XSAVE_XGETBV1,
-.features[FEAT_6_EAX] =
-CPUID_6_EAX_ARAT,
-/* Missing: Mode-based execute control (XS/XU), processor tracing, TSC 
scaling */
-.features[FEAT_VMX_BASIC] = MSR_VMX_BASIC_INS_OUTS |
- MSR_VMX_BASIC_TRUE_CTLS,
-.features[FEAT_VMX_ENTRY_CTLS] = VMX_VM_ENTRY_IA32E_MODE |
- VMX_VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL | 
VMX_VM_ENTRY_LOAD_IA32_PAT |
- VMX_VM_ENTRY_LOAD_DEBUG_CONTROLS | VMX_VM_ENTRY_LOAD_IA32_EFER,
-.features[FEAT_VMX_EPT_VPID_CAPS] = MSR_VMX_EPT_EXECONLY |
- MSR_VMX_EPT_PAGE_WALK_LE

Re: [RESEND PATCH v3 1/7] target/riscv: Add initial support for native debug

[Bin Meng]

On Wed, Jan 19, 2022 at 11:16 AM Alistair Francis  wrote:
>
> On Wed, Jan 5, 2022 at 1:09 PM Bin Meng  wrote:
> >
> > From: Bin Meng 
> >
> > This adds initial support for the native debug via the Trigger Module,
> > as defined in the RISC-V Debug Specification [1].
>
> Doesn't this mean we are just supporting the Sdtrig extension?

I was looking at where Sdtrig is defined. It turns out this new name
was assigned in a later debug spec and when this patch series was
worked on the Sdtrig extention was not invented yet ...

So the answer is yes, only Sdtrig is supported. Sdext does not make
sense in the QEMU context as it is for the on-chip-debugging.

>
> >
> > Only "Address / Data Match" trigger (type 2) is implemented as of now,
> > which is mainly used for hardware breakpoint and watchpoint. The number
> > of type 2 triggers implemented is 2, which is the number that we can
> > find in the SiFive U54/U74 cores.
> >
> > [1] 
> > https://github.com/riscv/riscv-debug-spec/raw/master/riscv-debug-stable.pdf
> >
> > Signed-off-by: Bin Meng 
> > ---
> >
> > Changes in v3:
> > - drop riscv_trigger_init(), which will be moved to patch #5
> >
> >  target/riscv/cpu.h   |   5 +
> >  target/riscv/debug.h | 108 +
> >  target/riscv/debug.c | 339 +++
> >  target/riscv/meson.build |   1 +
> >  4 files changed, 453 insertions(+)
> >  create mode 100644 target/riscv/debug.h
> >  create mode 100644 target/riscv/debug.c
> >
> > diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
> > index dc10f27093..0f3b3a4219 100644
> > --- a/target/riscv/cpu.h
> > +++ b/target/riscv/cpu.h
> > @@ -98,6 +98,7 @@ typedef struct CPURISCVState CPURISCVState;
> >
> >  #if !defined(CONFIG_USER_ONLY)
> >  #include "pmp.h"
> > +#include "debug.h"
> >  #endif
> >
> >  #define RV_VLEN_MAX 1024
> > @@ -234,6 +235,10 @@ struct CPURISCVState {
> >  pmp_table_t pmp_state;
> >  target_ulong mseccfg;
> >
> > +/* trigger module */
> > +target_ulong trigger_cur;
> > +trigger_type2_t trigger_type2[TRIGGER_TYPE2_NUM];
> > +
> >  /* machine specific rdtime callback */
> >  uint64_t (*rdtime_fn)(uint32_t);
> >  uint32_t rdtime_fn_arg;
> > diff --git a/target/riscv/debug.h b/target/riscv/debug.h
> > new file mode 100644
> > index 00..0a3fda6c72
> > --- /dev/null
> > +++ b/target/riscv/debug.h
> > @@ -0,0 +1,108 @@
> > +/*
> > + * QEMU RISC-V Native Debug Support
> > + *
> > + * Copyright (c) 2022 Wind River Systems, Inc.
> > + *
> > + * Author:
> > + *   Bin Meng 
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms and conditions of the GNU General Public License,
> > + * version 2 or later, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope it will be useful, but WITHOUT
> > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License 
> > for
> > + * more details.
> > + *
> > + * You should have received a copy of the GNU General Public License along 
> > with
> > + * this program.  If not, see .
> > + */
> > +
> > +#ifndef RISCV_DEBUG_H
> > +#define RISCV_DEBUG_H
> > +
> > +/* trigger indexes implemented */
> > +enum {
> > +TRIGGER_TYPE2_IDX_0 = 0,
> > +TRIGGER_TYPE2_IDX_1,
> > +TRIGGER_TYPE2_NUM,
> > +TRIGGER_NUM = TRIGGER_TYPE2_NUM
> > +};
> > +
> > +/* register index of tdata CSRs */
> > +enum {
> > +TDATA1 = 0,
> > +TDATA2,
> > +TDATA3,
> > +TDATA_NUM
> > +};
> > +
> > +typedef enum {
> > +TRIGGER_TYPE_NO_EXIST = 0,  /* trigger does not exist */
> > +TRIGGER_TYPE_AD_MATCH = 2,  /* address/data match trigger */
> > +TRIGGER_TYPE_INST_CNT = 3,  /* instruction count trigger */
> > +TRIGGER_TYPE_INT = 4,   /* interrupt trigger */
> > +TRIGGER_TYPE_EXCP = 5,  /* exception trigger */
> > +TRIGGER_TYPE_AD_MATCH6 = 6, /* new address/data match trigger */
> > +TRIGGER_TYPE_EXT_SRC = 7,   /* external source trigger */
> > +TRIGGER_TYPE_UNAVAIL = 15   /* trigger exists, but unavailable */
> > +} trigger_type_t;
> > +
> > +typedef struct {
> > +target_ulong mcontrol;
> > +target_ulong maddress;
> > +struct CPUBreakpoint *bp;
> > +struct CPUWatchpoint *wp;
> > +} trigger_type2_t;
>
> This is a confusing name
>

I will change it to type2_tigger,

Regards,
Bin

[PATCH experiment 01/16] coroutine: add missing coroutine_fn annotations for CoRwlock functions

[Paolo Bonzini]

CoRwlock can only be taken or released from a coroutine, and it
can yield.  Mark it as coroutine_fn.

Signed-off-by: Paolo Bonzini 
---
 include/qemu/coroutine.h | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index c828a95ee0..da68be5ad2 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -261,7 +261,7 @@ void qemu_co_rwlock_init(CoRwlock *lock);
  * of a parallel writer, control is transferred to the caller of the current
  * coroutine.
  */
-void qemu_co_rwlock_rdlock(CoRwlock *lock);
+void coroutine_fn qemu_co_rwlock_rdlock(CoRwlock *lock);
 
 /**
  * Write Locks the CoRwlock from a reader.  This is a bit more efficient than
@@ -270,7 +270,7 @@ void qemu_co_rwlock_rdlock(CoRwlock *lock);
  * to the caller of the current coroutine; another writer might run while
  * @qemu_co_rwlock_upgrade blocks.
  */
-void qemu_co_rwlock_upgrade(CoRwlock *lock);
+void coroutine_fn qemu_co_rwlock_upgrade(CoRwlock *lock);
 
 /**
  * Downgrades a write-side critic section to a reader.  Downgrading with
@@ -278,20 +278,20 @@ void qemu_co_rwlock_upgrade(CoRwlock *lock);
  * followed by @qemu_co_rwlock_rdlock.  This makes it more efficient, but
  * may also sometimes be necessary for correctness.
  */
-void qemu_co_rwlock_downgrade(CoRwlock *lock);
+void coroutine_fn qemu_co_rwlock_downgrade(CoRwlock *lock);
 
 /**
  * Write Locks the mutex. If the lock cannot be taken immediately because
  * of a parallel reader, control is transferred to the caller of the current
  * coroutine.
  */
-void qemu_co_rwlock_wrlock(CoRwlock *lock);
+void coroutine_fn qemu_co_rwlock_wrlock(CoRwlock *lock);
 
 /**
  * Unlocks the read/write lock and schedules the next coroutine that was
  * waiting for this lock to be run.
  */
-void qemu_co_rwlock_unlock(CoRwlock *lock);
+void coroutine_fn qemu_co_rwlock_unlock(CoRwlock *lock);
 
 typedef struct QemuCoSleep {
 Coroutine *to_wake;
-- 
2.35.1

[PATCH experiment 04/16] coroutine: introduce QemuCoLockable

[Paolo Bonzini]

In preparation for splitting "from coroutine" ("awaitable" in other
languages) and "not from coroutine" functions, remove the CoMutex case
from QemuLockable---thus making qemu_lockable_lock and qemu_lockable_unlock
"not awaitable".

To satisfy the qemu_co_queue_wait use case, introduce QemuCoLockable
which can be used for both QemuMutex (which will trivially never yield)
and CoMutex.  qemu_co_lockable_lock and qemu_co_lockable_unlock are
coroutine_fns.

Signed-off-by: Paolo Bonzini 
---
 include/qemu/co-lockable.h  | 98 +
 include/qemu/coroutine.h|  5 +-
 include/qemu/lockable.h | 13 ++---
 include/qemu/typedefs.h |  1 +
 tests/unit/test-coroutine.c | 10 ++--
 util/qemu-coroutine-lock.c  |  6 +--
 6 files changed, 114 insertions(+), 19 deletions(-)
 create mode 100644 include/qemu/co-lockable.h

diff --git a/include/qemu/co-lockable.h b/include/qemu/co-lockable.h
new file mode 100644
index 00..09f4620017
--- /dev/null
+++ b/include/qemu/co-lockable.h
@@ -0,0 +1,98 @@
+/*
+ * Polymorphic locking functions (aka poor man templates)
+ *
+ * Copyright Red Hat, Inc. 2017, 2018
+ *
+ * Author: Paolo Bonzini 
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_CO_LOCKABLE_H
+#define QEMU_CO_LOCKABLE_H
+
+#include "qemu/coroutine.h"
+#include "qemu/thread.h"
+
+typedef void coroutine_fn QemuCoLockUnlockFunc(void *);
+
+struct QemuCoLockable {
+void *object;
+QemuCoLockUnlockFunc *lock;
+QemuCoLockUnlockFunc *unlock;
+};
+
+static inline __attribute__((__always_inline__)) QemuCoLockable *
+qemu_make_co_lockable(void *x, QemuCoLockable *lockable)
+{
+/*
+ * We cannot test this in a macro, otherwise we get compiler
+ * warnings like "the address of 'm' will always evaluate as 'true'".
+ */
+return x ? lockable : NULL;
+}
+
+static inline __attribute__((__always_inline__)) QemuCoLockable *
+qemu_null_co_lockable(void *x)
+{
+if (x != NULL) {
+qemu_build_not_reached();
+}
+return NULL;
+}
+
+/*
+ * In C, compound literals have the lifetime of an automatic variable.
+ * In C++ it would be different, but then C++ wouldn't need QemuCoLockable
+ * either...
+ */
+#define QMCL_OBJ_(x, name) (&(QemuCoLockable) { \
+.object = (x),  \
+.lock = (QemuCoLockUnlockFunc *) qemu_ ## name ## _lock,\
+.unlock = (QemuCoLockUnlockFunc *) qemu_ ## name ## _unlock \
+})
+
+/**
+ * QEMU_MAKE_CO_LOCKABLE - Make a polymorphic QemuCoLockable
+ *
+ * @x: a lock object (currently one of QemuMutex, CoMutex).
+ *
+ * Returns a QemuCoLockable object that can be passed around
+ * to a function that can operate with locks of any kind, or
+ * NULL if @x is %NULL.
+ *
+ * Note the speci case for void *, so that we may pass "NULL".
+ */
+#define QEMU_MAKE_CO_LOCKABLE(x)\
+_Generic((x), QemuCoLockable *: (x),\
+ void *: qemu_null_co_lockable(x),  \
+ QemuMutex *: qemu_make_co_lockable(x, QMCL_OBJ_(x, mutex)),\
+ CoMutex *: qemu_make_co_lockable(x, QMCL_OBJ_(x, co_mutex)))   \
+
+/**
+ * QEMU_MAKE_CO_LOCKABLE_NONNULL - Make a polymorphic QemuCoLockable
+ *
+ * @x: a lock object (currently one of QemuMutex, QemuRecMutex,
+ * CoMutex, QemuSpin).
+ *
+ * Returns a QemuCoLockable object that can be passed around
+ * to a function that can operate with locks of any kind.
+ */
+#define QEMU_MAKE_CO_LOCKABLE_NONNULL(x)\
+_Generic((x), QemuCoLockable *: (x),\
+  QemuMutex *: QMCL_OBJ_(x, mutex), \
+  CoMutex *: QMCL_OBJ_(x, co_mutex))
+
+static inline void coroutine_fn qemu_co_lockable_lock(QemuCoLockable *x)
+{
+x->lock(x->object);
+}
+
+static inline void coroutine_fn qemu_co_lockable_unlock(QemuCoLockable *x)
+{
+x->unlock(x->object);
+}
+
+#endif
diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index 666f3ba0e0..6f4596fc5b 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -204,8 +204,8 @@ void qemu_co_queue_init(CoQueue *queue);
  * locked again afterwards.
  */
 #define qemu_co_queue_wait(queue, lock) \
-qemu_co_queue_wait_impl(queue, QEMU_MAKE_LOCKABLE(lock))
-void coroutine_fn qemu_co_queue_wait_impl(CoQueue *queue, QemuLockable *lock);
+qemu_co_queue_wait_impl(queue, QEMU_MAKE_CO_LOCKABLE(lock))
+void coroutine_fn qemu_co_queue_wait_impl(CoQueue *queue, QemuCoLockable 
*lock);
 
 /**
  * Removes the next coroutine from the CoQueue, and wake it up.
@@ -342,5 +342,6 @@ void qemu_coroutine_increase_pool_batch_size(unsigned int 
additional_pool_size);
 void qemu_coroutine_decrease_pool_batch_size(unsigned int 
additional_pool_size);

[PATCH experiment 00/16] C++20 coroutine backend

[Paolo Bonzini]

It turns out that going from a prototype C++ implementation of the QEMU
API, to something that could build tests/unit/test-coroutine, was just a
few hours work; and once it compiled, only one line had to be changed
for every test to pass.

Most of the differences between C and C++ already show up here:

- keywords such as "new" (or "class", which I didn't encounter yet)

- _Generic must be replaced by templates and/or overloading (QemuCoLockable
is implemented completely different from QemuLockable, in fact I spent
most of the time on that)

- PRI* functions must be separated with a space from string constants that
precede it

- void* casts must be explicit (g_new takes care of that most of the time,
but not for opaque pointers passed to coroutine).

There are 300 lines of hard-core C++ in the backend and in
coroutine.h.  I tried to comment it as much as possible (this
time I didn't include a big commit message on stackless coroutines
in general) but it still requires some knowledge of the basic C++
coroutine concepts of resumable types, promise types and awaiter types.
https://www.youtube.com/watch?v=ZTqHjjm86Bw is an excellent introduction
and it's where I learnt most of what was needed.

However, there  are no ramifications to actual coroutine code, except
for the template syntax "CoroutineFn" for the function and
the mandatory co_await/co_return keywords... both of which are an
improvement, really: the fact that a single function cannot run either
inside or outside coroutines is checked by the compiler now, because
qemu_coroutine_create accepts a function that returns CoroutineFn.
Therefore I had to disable some more code in util/ and qapi/ that used
qemu_in_coroutine() or coroutine_fn.

Here is the performance comparison of the three backends:

   ucontext   stackless C   stackless C++
/perf/lifecycle0.068 s0.025 s   0.065 s
/perf/nesting  55 s   4.7 s 1.7 s
/perf/yield6.0 s  1.3 s 1.3 s
/perf/cost 8 Mops/s (125ns)   35 ns 1 Mops/s (99 ns)

One important difference is that C++ coroutines allocate frames on the
heap, and that explains why performance is better in /perf/nesting,
which has to do many large memory allocations for the stack in the other
two backends (and also a makecontext/swapcontext in the ucontext case).
C++ coroutines hardly benefit from the coroutine pool; OTOH that also
means the coroutine pool could be removed if we went this way.

I haven't checked why /perf/lifecycle (and therefore /perf/cost; they
are roughly the same test) is so much slower than the handwritten C code.
It's still comparable with the ucontext backend though.

Overall this was ~twice the amount of work of the C experiment, but
that's because the two are very different ways to achieve the same goal:

- the design work was substantially smaller in the C experiment, where
all the backend does is allocate stack frames and do a loop that invokes
a function pointer.  Here the backend has to map between the C++ concepts
and the QEMU API.  In the C case, most of the work was really in the
manual conversion which I had to do one function at a time.

- the remaining work is also completely different: a source-to-source
translator (and only build system work in QEMU) for the C experiment;
making ~100 files compile in C++ for this one (and relatively little
work as far as coroutines are concerned).

This was compiled with GCC 11 only.  Coroutine support was added in
GCC 10, released in 2020, which IIRC is much newer than the most recent
release we support.

Paolo

Paolo Bonzini (17):
  coroutine: add missing coroutine_fn annotations for CoRwlock functions
  coroutine: qemu_coroutine_get_aio_context is not a coroutine_fn
  coroutine: small code cleanup in qemu_co_rwlock_wrlock
  coroutine: introduce QemuCoLockable
  port atomic.h to C++
  use g_new0 instead of g_malloc0
  start porting compiler.h to C++
  tracetool: add extern "C" around generated headers
  start adding extern "C" markers
  add space between liter and string macro
  bump to C++20
  remove "new" keyword from trace-events
  disable some code
  util: introduce C++ stackless coroutine backend
  port QemuCoLockable to C++ coroutines
  port test-coroutine to C++ coroutines

 configure |  48 +-
 include/block/aio.h   |   5 +
 include/fpu/softfloat-types.h |   4 +
 include/qemu/atomic.h |   5 +
 include/qemu/bitops.h |   3 +
 include/qemu/bswap.h  |  10 +-
 include/qemu/co-lockable.h|  93 
 include/qemu/compiler.h   |   4 +
 include/qemu/coroutine.h  | 466 +-
 include/qemu/coroutine_int.h  |   8 +
 include/qemu/host-utils.h |   4 +
 include/qemu/lockable.h

[PATCH experiment 02/16] coroutine: qemu_coroutine_get_aio_context is not a coroutine_fn

[Paolo Bonzini]

Since it operates on a given coroutine, qemu_coroutine_get_aio_context
can be called from outside coroutine context.

This is for example how qio_channel_restart_read uses it.

Signed-off-by: Paolo Bonzini 
---
 include/qemu/coroutine.h | 2 +-
 util/qemu-coroutine.c| 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index da68be5ad2..666f3ba0e0 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -92,7 +92,7 @@ void coroutine_fn qemu_coroutine_yield(void);
 /**
  * Get the AioContext of the given coroutine
  */
-AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co);
+AioContext *qemu_coroutine_get_aio_context(Coroutine *co);
 
 /**
  * Get the currently executing coroutine
diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
index c03b2422ff..9f2bd96fa0 100644
--- a/util/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -200,7 +200,7 @@ bool qemu_coroutine_entered(Coroutine *co)
 return co->caller;
 }
 
-AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co)
+AioContext *qemu_coroutine_get_aio_context(Coroutine *co)
 {
 return co->ctx;
 }
-- 
2.35.1

Re: MAINTAINERS: macOS host support (was: MAINTAINERS: take edk2)

[Daniel P . Berrangé]

On Sat, Mar 12, 2022 at 02:51:21PM +0100, Christian Schoenebeck wrote:
> On Freitag, 11. März 2022 10:26:47 CET Daniel P. Berrangé wrote:
> > On Fri, Mar 11, 2022 at 10:13:24AM +0100, Christian Schoenebeck wrote:
> > > On Donnerstag, 10. März 2022 12:40:06 CET Philippe Mathieu-Daudé wrote:
> > > > +Stefan for overall project resources.
> > > > 
> > > > On 10/3/22 12:07, Daniel P. Berrangé wrote:
> > > > > On Thu, Mar 10, 2022 at 12:00:35PM +0100, Christian Schoenebeck wrote:
> > > > >> On Mittwoch, 9. März 2022 12:44:16 CET Daniel P. Berrangé wrote:
> > > > >>> On Wed, Mar 09, 2022 at 11:40:42AM +0100, Christian Schoenebeck 
> wrote:
> > > >  On Mittwoch, 9. März 2022 11:05:02 CET Philippe Mathieu-Daudé 
> wrote:
> > > > > Not sure what you have in mind. I'm totally new to the
> > > > > macOS/Darwin
> > > > > world, and have no choice but to use it as primary workstation and
> > > > > for CI builds, so I can help with overall testing / maintenance.
> > > > > 
> > > > > Peter, since you take some macOS patches, would you like to
> > > > > maintain
> > > > > this officially? Since I doubt you want to take yet another
> > > > > responsibility, what about having a co-maintained section,
> > > > > including
> > > > > technical expertise from Akihiko / Joelle / Christian? (Cc'ed)
> > > > > 
> > > > > Regards,
> > > >  
> > > >  Also CCing Cameron on this, just in case someone at Apple could
> > > >  spend
> > > >  some
> > > >  slices on QEMU macOS patches in general as well.
> > > >  
> > > >  As for my part: I try to help out more on the macOS front. As
> > > >  there's
> > > >  now
> > > >  macOS host support for 9p I have to start QEMU testing on macOS
> > > >  locally
> > > >  anyway. Too bad that macOS CI tests on Github are no longer
> > > >  available
> > > >  BTW.
> > > > >>> 
> > > > >>> Note QEMU gets macOS CI coverage in GitLab. We use a clever trick by
> > > > >>> which we use 'cirrus-run' from the GitLab job to trigger a build in
> > > > >>> Cirrus CI's macOS builders, and pull the results back when its done.
> > > > >>> 
> > > > >>> Any contributor can get this working on their QEMU fork too, if they
> > > > >>> configure the needed Cirrus CI API token. See the docs in
> > > > >>> 
> > > > >>> .gitlab-ci.d/cirrus/README.rst
> > > > >>> 
> > > > >>> This is enough for build + automated tests.
> > > > >> 
> > > > >> Does this mean that people no longer have to pull their credit card
> > > > >> just
> > > > >> for running CI tests on Gitlab?
> > > > > 
> > > > > Not really. The CC validation is something GitLab have had to force
> > > > > onto all new accounts due to cryptominer abuse of their free shared
> > > > > CI runners :-( If you have VMs somewhere you could theoretically
> > > > > spin up your own CI runners instead of using the shared runners and
> > > > > that could avoid the CC validation need.
> > > > 
> > > > Not that trivial, first you need to figure out the list of dependencies
> > > > GitLab images come with, then you realize you need 50GiB+ of available
> > > > storage a single pipeline (due to all the Docker images pulled / built)
> > > > and you also need a decent internet link otherwise various jobs timeout
> > > > randomly, then you have to wait 20h+ with a quad-core CPU / 16GiB RAM,
> > > 
> > > Considering that CI jobs currently take about 1 hour on Gitlab, which
> > > processor generation are you referring to that would take 20 hours?
> > 
> > You're not taking into account parallelism. The GitLab pipeline takes
> > 1 hour wallclock time, which is not the same as 1 hour CPU time. We
> > probably have 20+ jobs running in parallel on gitlab, as they get
> > farmed out to many machines. If you have only a single machine at your
> > disposal, then you'll have much less prallelism, so overall time can
> > be much longer.
> > 
> > > > and eventually you realize you lost 3 days of your life to not register
> > > > your CC which you'll be forced to give anyway.
> > > 
> > > It's an obstacle. And that keeps people away. Plus the trend seems to be
> > > that free CI services disappear one by one, so I am not so sure that
> > > giving your credit card once solves this issue for good.
> > 
> > The CC requirement there is primarily to act as an identity check
> > on accounts, so they have some mechanism to discourage and/or trace
> > abusive users. You can use it to purchase extra CI time, but they've
> > stated multiple times their intention to continue to grant free CI
> > time to open source projects and their contributors. They are actively
> > discussing their plans with a number of open source project contributors
> > including myself on behalf of QEMU, to better understand our needs. I
> > outlined my current understanding of their intentions here:
> > 
> >  https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg03962.html
> 
> Please send an announcement (in subject

[PATCH experiment 06/16] use g_new0 instead of g_malloc0

[Paolo Bonzini]

Casting to/from void* must be explicit in C++.  g_new0 takes care of that.

Signed-off-by: Paolo Bonzini 
---
 include/qemu/timer.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/qemu/timer.h b/include/qemu/timer.h
index 88ef114689..ee071e07d1 100644
--- a/include/qemu/timer.h
+++ b/include/qemu/timer.h
@@ -520,7 +520,7 @@ static inline QEMUTimer *timer_new_full(QEMUTimerListGroup 
*timer_list_group,
 int scale, int attributes,
 QEMUTimerCB *cb, void *opaque)
 {
-QEMUTimer *ts = g_malloc0(sizeof(QEMUTimer));
+QEMUTimer *ts = g_new0(QEMUTimer, 1);
 timer_init_full(ts, timer_list_group, type, scale, attributes, cb, opaque);
 return ts;
 }
-- 
2.35.1

[PATCH experiment 07/16] start porting compiler.h to C++

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 include/qemu/compiler.h | 4 
 1 file changed, 4 insertions(+)

diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h
index 3baa5e3790..18848f0d49 100644
--- a/include/qemu/compiler.h
+++ b/include/qemu/compiler.h
@@ -72,6 +72,10 @@
 int:(x) ? -1 : 1; \
 }
 
+#ifdef __cplusplus
+#define _Static_assert static_assert
+#endif
+
 #define QEMU_BUILD_BUG_MSG(x, msg) _Static_assert(!(x), msg)
 
 #define QEMU_BUILD_BUG_ON(x) QEMU_BUILD_BUG_MSG(x, "not expecting: " #x)
-- 
2.35.1

[PATCH experiment 11/16] bump to C++20

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 configure   | 4 ++--
 meson.build | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 886000346a..091710ec03 100755
--- a/configure
+++ b/configure
@@ -157,8 +157,8 @@ update_cxxflags() {
 # Set QEMU_CXXFLAGS from QEMU_CFLAGS by filtering out those
 # options which some versions of GCC's C++ compiler complain about
 # because they only make sense for C programs.
-QEMU_CXXFLAGS="-D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS"
-CONFIGURE_CXXFLAGS=$(echo "$CONFIGURE_CFLAGS" | sed 
s/-std=gnu11/-std=gnu++11/)
+QEMU_CXXFLAGS="-D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -fcoroutines"
+CONFIGURE_CXXFLAGS=$(echo "$CONFIGURE_CFLAGS" | sed 
s/-std=gnu11/-std=gnu++20/)
 for arg in $QEMU_CFLAGS; do
 case $arg in
 -Wstrict-prototypes|-Wmissing-prototypes|-Wnested-externs|\
diff --git a/meson.build b/meson.build
index 2d6601467f..810ebb0865 100644
--- a/meson.build
+++ b/meson.build
@@ -1,5 +1,5 @@
 project('qemu', ['c'], meson_version: '>=0.59.3',
-default_options: ['warning_level=1', 'c_std=gnu11', 'cpp_std=gnu++11', 
'b_colorout=auto',
+default_options: ['warning_level=1', 'c_std=gnu11', 'cpp_std=gnu++20', 
'b_colorout=auto',
   'b_staticpic=false', 'stdsplit=false'],
 version: files('VERSION'))
 
-- 
2.35.1

[PATCH experiment 03/16] coroutine: small code cleanup in qemu_co_rwlock_wrlock

[Paolo Bonzini]

qemu_co_rwlock_wrlock stores the current coroutine in a loc variable,
use it instead of calling qemu_coroutine_self() again.

Signed-off-by: Paolo Bonzini 
---
 util/qemu-coroutine-lock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
index 2669403839..490fb32891 100644
--- a/util/qemu-coroutine-lock.c
+++ b/util/qemu-coroutine-lock.c
@@ -436,7 +436,7 @@ void qemu_co_rwlock_wrlock(CoRwlock *lock)
 lock->owners = -1;
 qemu_co_mutex_unlock(&lock->mutex);
 } else {
-CoRwTicket my_ticket = { false, qemu_coroutine_self() };
+CoRwTicket my_ticket = { false, self };
 
 QSIMPLEQ_INSERT_TAIL(&lock->tickets, &my_ticket, next);
 qemu_co_mutex_unlock(&lock->mutex);
-- 
2.35.1

[PATCH experiment 08/16] tracetool: add extern "C" around generated headers

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 scripts/tracetool/format/h.py | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/scripts/tracetool/format/h.py b/scripts/tracetool/format/h.py
index e94f0be7da..2d92fa8bd2 100644
--- a/scripts/tracetool/format/h.py
+++ b/scripts/tracetool/format/h.py
@@ -27,6 +27,9 @@ def generate(events, backend, group):
 '#define TRACE_%s_GENERATED_TRACERS_H' % group.upper(),
 '',
 '#include "%s"' % header,
+'#ifdef __cplusplus',
+'extern "C" {',
+'#endif'
 '')
 
 for e in events:
@@ -100,4 +103,7 @@ def generate(events, backend, group):
 
 backend.generate_end(events, group)
 
-out('#endif /* TRACE_%s_GENERATED_TRACERS_H */' % group.upper())
+out('#ifdef __cplusplus',
+'}',
+'#endif',
+'#endif /* TRACE_%s_GENERATED_TRACERS_H */' % group.upper())
-- 
2.35.1

[PATCH experiment 09/16] start adding extern "C" markers

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 include/block/aio.h   |  5 +
 include/fpu/softfloat-types.h |  4 
 include/qemu/bitops.h |  3 +++
 include/qemu/bswap.h  | 10 +++---
 include/qemu/coroutine.h  |  4 
 include/qemu/host-utils.h |  4 
 include/qemu/notify.h |  4 
 include/qemu/qsp.h|  4 
 include/qemu/thread.h |  4 
 include/qemu/timer.h  |  4 
 10 files changed, 39 insertions(+), 7 deletions(-)

diff --git a/include/block/aio.h b/include/block/aio.h
index 5634173b12..4b21d95f0b 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -17,12 +17,15 @@
 #ifdef CONFIG_LINUX_IO_URING
 #include 
 #endif
+
 #include "qemu/coroutine.h"
 #include "qemu/queue.h"
 #include "qemu/event_notifier.h"
 #include "qemu/thread.h"
 #include "qemu/timer.h"
 
+G_BEGIN_DECLS
+
 typedef struct BlockAIOCB BlockAIOCB;
 typedef void BlockCompletionFunc(void *opaque, int ret);
 
@@ -769,4 +772,6 @@ void aio_context_set_poll_params(AioContext *ctx, int64_t 
max_ns,
 void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch,
 Error **errp);
 
+G_END_DECLS
+
 #endif
diff --git a/include/fpu/softfloat-types.h b/include/fpu/softfloat-types.h
index 8abd9ab4ec..aaf7b0b5fa 100644
--- a/include/fpu/softfloat-types.h
+++ b/include/fpu/softfloat-types.h
@@ -80,6 +80,8 @@ this code that are retained.
 #ifndef SOFTFLOAT_TYPES_H
 #define SOFTFLOAT_TYPES_H
 
+G_BEGIN_DECLS
+
 /*
  * Software IEC/IEEE floating-point types.
  */
@@ -197,4 +199,6 @@ typedef struct float_status {
 bool no_signaling_nans;
 } float_status;
 
+G_END_DECLS
+
 #endif /* SOFTFLOAT_TYPES_H */
diff --git a/include/qemu/bitops.h b/include/qemu/bitops.h
index 03213ce952..677884bead 100644
--- a/include/qemu/bitops.h
+++ b/include/qemu/bitops.h
@@ -12,6 +12,7 @@
 #ifndef BITOPS_H
 #define BITOPS_H
 
+G_BEGIN_DECLS
 
 #include "host-utils.h"
 #include "atomic.h"
@@ -618,4 +619,6 @@ static inline uint64_t half_unshuffle64(uint64_t x)
 return x;
 }
 
+G_END_DECLS
+
 #endif
diff --git a/include/qemu/bswap.h b/include/qemu/bswap.h
index 2d3bb8bbed..439e755ed4 100644
--- a/include/qemu/bswap.h
+++ b/include/qemu/bswap.h
@@ -15,12 +15,10 @@
 #define BSWAP_FROM_FALLBACKS
 #endif /* ! CONFIG_MACHINE_BSWAP_H */
 
-#ifdef __cplusplus
-extern "C" {
-#endif
-
 #include "fpu/softfloat-types.h"
 
+G_BEGIN_DECLS
+
 #ifdef BSWAP_FROM_BYTESWAP
 static inline uint16_t bswap16(uint16_t x)
 {
@@ -508,8 +506,6 @@ DO_STN_LDN_P(be)
 #undef le_bswaps
 #undef be_bswaps
 
-#ifdef __cplusplus
-}
-#endif
+G_END_DECLS
 
 #endif /* BSWAP_H */
diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index 6f4596fc5b..428e97d946 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -18,6 +18,8 @@
 #include "qemu/queue.h"
 #include "qemu/timer.h"
 
+G_BEGIN_DECLS
+
 /**
  * Coroutines are a mechanism for stack switching and can be used for
  * cooperative userspace threading.  These functions provide a simple but
@@ -341,6 +343,8 @@ void qemu_coroutine_increase_pool_batch_size(unsigned int 
additional_pool_size);
  */
 void qemu_coroutine_decrease_pool_batch_size(unsigned int 
additional_pool_size);
 
+G_END_DECLS
+
 #include "qemu/lockable.h"
 #include "qemu/co-lockable.h"
 
diff --git a/include/qemu/host-utils.h b/include/qemu/host-utils.h
index ca979dc6cc..406e593dff 100644
--- a/include/qemu/host-utils.h
+++ b/include/qemu/host-utils.h
@@ -30,6 +30,8 @@
 #ifndef HOST_UTILS_H
 #define HOST_UTILS_H
 
+G_BEGIN_DECLS
+
 #include "qemu/compiler.h"
 #include "qemu/bswap.h"
 
@@ -849,4 +851,6 @@ static inline uint64_t udiv_qrnnd(uint64_t *r, uint64_t n1,
 #endif
 }
 
+G_END_DECLS
+
 #endif
diff --git a/include/qemu/notify.h b/include/qemu/notify.h
index bcfa70fb2e..a8effa39b7 100644
--- a/include/qemu/notify.h
+++ b/include/qemu/notify.h
@@ -14,6 +14,8 @@
 #ifndef QEMU_NOTIFY_H
 #define QEMU_NOTIFY_H
 
+G_BEGIN_DECLS
+
 #include "qemu/queue.h"
 
 typedef struct Notifier Notifier;
@@ -71,4 +73,6 @@ void notifier_with_return_remove(NotifierWithReturn 
*notifier);
 int notifier_with_return_list_notify(NotifierWithReturnList *list,
  void *data);
 
+G_END_DECLS
+
 #endif
diff --git a/include/qemu/qsp.h b/include/qemu/qsp.h
index bf36aabfa8..65389837a1 100644
--- a/include/qemu/qsp.h
+++ b/include/qemu/qsp.h
@@ -16,6 +16,8 @@ enum QSPSortBy {
 QSP_SORT_BY_AVG_WAIT_TIME,
 };
 
+G_BEGIN_DECLS
+
 void qsp_report(size_t max, enum QSPSortBy sort_by,
 bool callsite_coalesce);
 
@@ -24,4 +26,6 @@ void qsp_enable(void);
 void qsp_disable(void);
 void qsp_reset(void);
 
+G_END_DECLS
+
 #endif /* QEMU_QSP_H */
diff --git a/include/qemu/thread.h b/include/qemu/thread.h
index 460568d67d..ec27b7ec58 100644
--- a/include/qemu/thread.h
+++ b/include/qemu/thread.h
@@ -22,6 +22,8 @@ typedef struct QemuThread QemuThread;
 #define QEMU_THREAD_JOINABLE 0
 #define QEMU_THREAD_DETACHED 1
 
+G_BEGIN_DECLS
+

[PATCH experiment 05/16] port atomic.h to C++

[Paolo Bonzini]

The functionality of typeof_strip_qual is provided by the standard library.

Signed-off-by: Paolo Bonzini 
---
 include/qemu/atomic.h | 5 +
 include/qemu/osdep.h  | 1 +
 2 files changed, 6 insertions(+)

diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index 112a29910b..0889a9c5d2 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -26,6 +26,10 @@
  * implicit promotion.  int and larger types, as well as pointers, can be
  * converted to a non-qualified type just by applying a binary operator.
  */
+#ifdef __cplusplus
+#define typeof_strip_qual(expr)
\
+   std::remove_cv::type
+#else
 #define typeof_strip_qual(expr)
\
   typeof(  
\
 __builtin_choose_expr( 
\
@@ -59,6 +63,7 @@
 __builtin_types_compatible_p(typeof(expr), const volatile unsigned 
short), \
 (unsigned short)1, 
\
   (expr)+0))
+#endif
 
 #ifndef __ATOMIC_RELAXED
 #error "Expecting C11 atomic ops"
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index c9ec7830c9..28707bbde3 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -140,6 +140,7 @@ QEMU_EXTERN_C int daemon(int, int);
 #endif
 
 #ifdef __cplusplus
+#include 
 extern "C" {
 #endif
 
-- 
2.35.1

[PATCH experiment 15/16] port QemuCoLockable to C++ coroutines

[Paolo Bonzini]

Convert "T coroutine_fn" annotations to the new type CoroutineFn,
and add co_await as needed.

_Generic is replaced by an overloaded constructor.  C++ also does
not like & on a temporary, so that is replaced by a function
qemu_make_co_lockable_nonnull that hides it from the compiler.

Signed-off-by: Paolo Bonzini 
---
 include/qemu/co-lockable.h | 111 ++---
 1 file changed, 53 insertions(+), 58 deletions(-)

diff --git a/include/qemu/co-lockable.h b/include/qemu/co-lockable.h
index 09f4620017..13e3cc7a69 100644
--- a/include/qemu/co-lockable.h
+++ b/include/qemu/co-lockable.h
@@ -16,83 +16,78 @@
 #include "qemu/coroutine.h"
 #include "qemu/thread.h"
 
-typedef void coroutine_fn QemuCoLockUnlockFunc(void *);
+typedef CoroutineFn QemuCoLockUnlockFunc(void *);
+
+extern CoroutineFn qemu_mutex_co_lock(QemuMutex *m);
+extern CoroutineFn qemu_mutex_co_unlock(QemuMutex *m);
 
 struct QemuCoLockable {
 void *object;
 QemuCoLockUnlockFunc *lock;
 QemuCoLockUnlockFunc *unlock;
+
+QemuCoLockable() :
+object{NULL},
+lock{(QemuCoLockUnlockFunc *) NULL},
+unlock{(QemuCoLockUnlockFunc *) NULL} {}
+QemuCoLockable(QemuMutex *x) :
+object{x},
+lock{(QemuCoLockUnlockFunc *) qemu_mutex_co_lock},
+unlock{(QemuCoLockUnlockFunc *) qemu_mutex_co_unlock} {}
+QemuCoLockable(CoMutex *x) :
+object{x},
+lock{(QemuCoLockUnlockFunc *) qemu_co_mutex_lock},
+unlock{(QemuCoLockUnlockFunc *) qemu_co_mutex_unlock} {}
 };
 
-static inline __attribute__((__always_inline__)) QemuCoLockable *
-qemu_make_co_lockable(void *x, QemuCoLockable *lockable)
+template
+static inline QemuCoLockable qcml_obj_nonnull_(T *x)
 {
-/*
- * We cannot test this in a macro, otherwise we get compiler
- * warnings like "the address of 'm' will always evaluate as 'true'".
- */
-return x ? lockable : NULL;
+return QemuCoLockable{x};
 }
 
-static inline __attribute__((__always_inline__)) QemuCoLockable *
-qemu_null_co_lockable(void *x)
+static inline QemuCoLockable const 
*qemu_make_co_lockable_nonnull(QemuCoLockable const &x)
 {
+return &x;
+}
+
+template
+static inline QemuCoLockable qcml_obj_(T *x)
+{
+return QemuCoLockable{x};
+}
+extern void build_not_reached();
+
+template<> inline
+QemuCoLockable qcml_obj_(void *x)
+{
+#ifdef __OPTIMIZE__
 if (x != NULL) {
-qemu_build_not_reached();
+build_not_reached();
 }
-return NULL;
+#endif
+return QemuCoLockable{};
 }
 
-/*
- * In C, compound literals have the lifetime of an automatic variable.
- * In C++ it would be different, but then C++ wouldn't need QemuCoLockable
- * either...
- */
-#define QMCL_OBJ_(x, name) (&(QemuCoLockable) { \
-.object = (x),  \
-.lock = (QemuCoLockUnlockFunc *) qemu_ ## name ## _lock,\
-.unlock = (QemuCoLockUnlockFunc *) qemu_ ## name ## _unlock \
-})
-
-/**
- * QEMU_MAKE_CO_LOCKABLE - Make a polymorphic QemuCoLockable
- *
- * @x: a lock object (currently one of QemuMutex, CoMutex).
- *
- * Returns a QemuCoLockable object that can be passed around
- * to a function that can operate with locks of any kind, or
- * NULL if @x is %NULL.
- *
- * Note the speci case for void *, so that we may pass "NULL".
- */
-#define QEMU_MAKE_CO_LOCKABLE(x)\
-_Generic((x), QemuCoLockable *: (x),\
- void *: qemu_null_co_lockable(x),  \
- QemuMutex *: qemu_make_co_lockable(x, QMCL_OBJ_(x, mutex)),\
- CoMutex *: qemu_make_co_lockable(x, QMCL_OBJ_(x, co_mutex)))   \
-
-/**
- * QEMU_MAKE_CO_LOCKABLE_NONNULL - Make a polymorphic QemuCoLockable
- *
- * @x: a lock object (currently one of QemuMutex, QemuRecMutex,
- * CoMutex, QemuSpin).
- *
- * Returns a QemuCoLockable object that can be passed around
- * to a function that can operate with locks of any kind.
- */
-#define QEMU_MAKE_CO_LOCKABLE_NONNULL(x)\
-_Generic((x), QemuCoLockable *: (x),\
-  QemuMutex *: QMCL_OBJ_(x, mutex), \
-  CoMutex *: QMCL_OBJ_(x, co_mutex))
-
-static inline void coroutine_fn qemu_co_lockable_lock(QemuCoLockable *x)
+static inline QemuCoLockable const *qemu_make_co_lockable(QemuCoLockable const 
&x)
 {
-x->lock(x->object);
+if (x.object)
+return &x;
+else
+return NULL;
 }
 
-static inline void coroutine_fn qemu_co_lockable_unlock(QemuCoLockable *x)
+#define QEMU_MAKE_CO_LOCKABLE_NONNULL(x) 
qemu_make_co_lockable_nonnull(qcml_obj_nonnull_(x))
+#define QEMU_MAKE_CO_LOCKABLE(x) qemu_make_co_lockable(qcml_obj_(x))
+
+static inline CoroutineFn qemu_co_lockable_lock(const QemuCoLockable *x)
 {
-x->unlock(x->object);
+co_await x->lock(x->object);
+}
+
+static inl

[PATCH experiment 16/16] port test-coroutine to C++ coroutines

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 tests/unit/meson.build|   6 +-
 .../{test-coroutine.c => test-coroutine.cc}   | 140 +-
 2 files changed, 76 insertions(+), 70 deletions(-)
 rename tests/unit/{test-coroutine.c => test-coroutine.cc} (81%)

diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index 4ca5fdb699..675b5323dd 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -168,7 +168,11 @@ slow_tests = {
 }
 
 foreach test_name, extra: tests
-  src = [test_name + '.c']
+  if test_name == 'test-coroutine'
+src = [test_name + '.cc']
+  else
+src = [test_name + '.c']
+  endif
   deps = [qemuutil]
   if extra.length() > 0
 # use a sourceset to quickly separate sources and deps
diff --git a/tests/unit/test-coroutine.c b/tests/unit/test-coroutine.cc
similarity index 81%
rename from tests/unit/test-coroutine.c
rename to tests/unit/test-coroutine.cc
index c230c2fa6e..8f9ddc50da 100644
--- a/tests/unit/test-coroutine.c
+++ b/tests/unit/test-coroutine.cc
@@ -16,14 +16,14 @@
 #include "qemu/coroutine_int.h"
 #include "qemu/lockable.h"
 
-#if 0
 /*
  * Check that qemu_in_coroutine() works
  */
 
-static void coroutine_fn verify_in_coroutine(void *opaque)
+static CoroutineFn verify_in_coroutine(void *opaque)
 {
 g_assert(qemu_in_coroutine());
+co_return;
 }
 
 static void test_in_coroutine(void)
@@ -40,10 +40,11 @@ static void test_in_coroutine(void)
  * Check that qemu_coroutine_self() works
  */
 
-static void coroutine_fn verify_self(void *opaque)
+static CoroutineFn verify_self(void *opaque)
 {
-Coroutine **p_co = opaque;
+Coroutine **p_co = (Coroutine **)opaque;
 g_assert(qemu_coroutine_self() == *p_co);
+co_return;
 }
 
 static void test_self(void)
@@ -58,20 +59,20 @@ static void test_self(void)
  * Check that qemu_coroutine_entered() works
  */
 
-static void coroutine_fn verify_entered_step_2(void *opaque)
+static CoroutineFn verify_entered_step_2(void *opaque)
 {
 Coroutine *caller = (Coroutine *)opaque;
 
 g_assert(qemu_coroutine_entered(caller));
 g_assert(qemu_coroutine_entered(qemu_coroutine_self()));
-qemu_coroutine_yield();
+co_await qemu_coroutine_yield();
 
 /* Once more to check it still works after yielding */
 g_assert(qemu_coroutine_entered(caller));
 g_assert(qemu_coroutine_entered(qemu_coroutine_self()));
 }
 
-static void coroutine_fn verify_entered_step_1(void *opaque)
+static CoroutineFn verify_entered_step_1(void *opaque)
 {
 Coroutine *self = qemu_coroutine_self();
 Coroutine *coroutine;
@@ -83,6 +84,7 @@ static void coroutine_fn verify_entered_step_1(void *opaque)
 qemu_coroutine_enter(coroutine);
 g_assert(!qemu_coroutine_entered(coroutine));
 qemu_coroutine_enter(coroutine);
+co_return;
 }
 
 static void test_entered(void)
@@ -104,9 +106,9 @@ typedef struct {
 unsigned int max;   /* maximum level of nesting */
 } NestData;
 
-static void coroutine_fn nest(void *opaque)
+static CoroutineFn nest(void *opaque)
 {
-NestData *nd = opaque;
+NestData *nd = (NestData *)opaque;
 
 nd->n_enter++;
 
@@ -118,6 +120,7 @@ static void coroutine_fn nest(void *opaque)
 }
 
 nd->n_return++;
+co_return;
 }
 
 static void test_nesting(void)
@@ -141,13 +144,13 @@ static void test_nesting(void)
  * Check that yield/enter transfer control correctly
  */
 
-static void coroutine_fn yield_5_times(void *opaque)
+static CoroutineFn yield_5_times(void *opaque)
 {
-bool *done = opaque;
+bool *done = (bool *)opaque;
 int i;
 
 for (i = 0; i < 5; i++) {
-qemu_coroutine_yield();
+co_await qemu_coroutine_yield();
 }
 *done = true;
 }
@@ -166,15 +169,16 @@ static void test_yield(void)
 g_assert_cmpint(i, ==, 5); /* coroutine must yield 5 times */
 }
 
-static void coroutine_fn c2_fn(void *opaque)
+static CoroutineFn c2_fn(void *opaque)
 {
-qemu_coroutine_yield();
+co_await qemu_coroutine_yield();
 }
 
-static void coroutine_fn c1_fn(void *opaque)
+static CoroutineFn c1_fn(void *opaque)
 {
-Coroutine *c2 = opaque;
+Coroutine *c2 = (Coroutine *)opaque;
 qemu_coroutine_enter(c2);
+co_return;
 }
 
 static void test_no_dangling_access(void)
@@ -200,34 +204,35 @@ static void test_no_dangling_access(void)
 static bool locked;
 static int done;
 
-static void coroutine_fn mutex_fn(void *opaque)
+static CoroutineFn mutex_fn(void *opaque)
 {
-CoMutex *m = opaque;
-qemu_co_mutex_lock(m);
+CoMutex *m = (CoMutex *)opaque;
+co_await qemu_co_mutex_lock(m);
 assert(!locked);
 locked = true;
-qemu_coroutine_yield();
+co_await qemu_coroutine_yield();
 locked = false;
-qemu_co_mutex_unlock(m);
+co_await qemu_co_mutex_unlock(m);
 done++;
 }
 
-static void coroutine_fn lockable_fn(void *opaque)
+static CoroutineFn lockable_fn(void *opaque)
 {
-QemuCoLockable *x = opaque;
-qemu_co_lockable_lock(x);
+QemuCoLockable *x = (QemuCoLockable

[PATCH experiment 14/16] util: introduce C++ stackless coroutine backend

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 configure |  44 +-
 include/qemu/coroutine.h  | 445 +-
 include/qemu/coroutine_int.h  |   8 +
 util/coroutine-stackless.cc   | 145 ++
 util/meson.build  |   6 +-
 ...oroutine-lock.c => qemu-coroutine-lock.cc} |  70 +--
 ...outine-sleep.c => qemu-coroutine-sleep.cc} |  12 +-
 util/{qemu-coroutine.c => qemu-coroutine.cc}  |  16 -
 8 files changed, 530 insertions(+), 216 deletions(-)
 create mode 100644 util/coroutine-stackless.cc
 rename util/{qemu-coroutine-lock.c => qemu-coroutine-lock.cc} (87%)
 rename util/{qemu-coroutine-sleep.c => qemu-coroutine-sleep.cc} (89%)
 rename util/{qemu-coroutine.c => qemu-coroutine.cc} (94%)

diff --git a/configure b/configure
index 091710ec03..c02b5edcba 100755
--- a/configure
+++ b/configure
@@ -1220,8 +1220,6 @@ Advanced options (experts only):
   --with-trace-file=NAME   Full PATH,NAME of file to store traces
Default:trace-
   --cpu=CPUBuild for host CPU [$cpu]
-  --with-coroutine=BACKEND coroutine backend. Supported options:
-   ucontext, sigaltstack, windows
   --enable-gcovenable test coverage analysis with gcov
   --tls-priority   default TLS protocol/cipher priority string
   --enable-plugins
@@ -1242,7 +1240,7 @@ cat << EOF
   debug-info  debugging information
   lto Enable Link-Time Optimization.
   safe-stack  SafeStack Stack Smash Protection. Depends on
-  clang/llvm >= 3.7 and requires coroutine backend ucontext.
+  clang/llvm >= 3.7
   rdmaEnable RDMA-based migration
   pvrdma  Enable PVRDMA support
   vhost-net   vhost-net kernel acceleration support
@@ -2338,39 +2336,7 @@ EOF
   fi
 fi
 
-if test "$coroutine" = ""; then
-  if test "$mingw32" = "yes"; then
-coroutine=win32
-  elif test "$ucontext_works" = "yes"; then
-coroutine=ucontext
-  else
-coroutine=sigaltstack
-  fi
-else
-  case $coroutine in
-  windows)
-if test "$mingw32" != "yes"; then
-  error_exit "'windows' coroutine backend only valid for Windows"
-fi
-# Unfortunately the user visible backend name doesn't match the
-# coroutine-*.c filename for this case, so we have to adjust it here.
-coroutine=win32
-;;
-  ucontext)
-if test "$ucontext_works" != "yes"; then
-  feature_not_found "ucontext"
-fi
-;;
-  sigaltstack)
-if test "$mingw32" = "yes"; then
-  error_exit "only the 'windows' coroutine backend is valid for Windows"
-fi
-;;
-  *)
-error_exit "unknown coroutine backend $coroutine"
-;;
-  esac
-fi
+coroutine=stackless
 
 ##
 # SafeStack
@@ -2395,9 +2361,6 @@ EOF
   else
 error_exit "SafeStack not supported by your compiler"
   fi
-  if test "$coroutine" != "ucontext"; then
-error_exit "SafeStack is only supported by the coroutine backend ucontext"
-  fi
 else
 cat > $TMPC << EOF
 int main(int argc, char *argv[])
@@ -2427,9 +2390,6 @@ else # "$safe_stack" = ""
 safe_stack="no"
   else
 safe_stack="yes"
-if test "$coroutine" != "ucontext"; then
-  error_exit "SafeStack is only supported by the coroutine backend 
ucontext"
-fi
   fi
 fi
 fi
diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index ac9891502e..0f89fbafa0 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -48,25 +48,6 @@ G_BEGIN_DECLS
 
 typedef struct Coroutine Coroutine;
 
-/**
- * Coroutine entry point
- *
- * When the coroutine is entered for the first time, opaque is passed in as an
- * argument.
- *
- * When this function returns, the coroutine is destroyed automatically and
- * execution continues in the caller who last entered the coroutine.
- */
-typedef void coroutine_fn CoroutineEntry(void *opaque);
-
-/**
- * Create a new coroutine
- *
- * Use qemu_coroutine_enter() to actually transfer control to the coroutine.
- * The opaque argument is passed as the argument to the entry point.
- */
-Coroutine *qemu_coroutine_create(CoroutineEntry *entry, void *opaque);
-
 /**
  * Transfer control to a coroutine
  */
@@ -83,14 +64,6 @@ void qemu_coroutine_enter_if_inactive(Coroutine *co);
  */
 void qemu_aio_coroutine_enter(AioContext *ctx, Coroutine *co);
 
-/**
- * Transfer control back to a coroutine's caller
- *
- * This function does not return until the coroutine is re-entered using
- * qemu_coroutine_enter().
- */
-void coroutine_fn qemu_coroutine_yield(void);
-
 /**
  * Get the AioContext of the given coroutine
  */
@@ -157,18 +130,6 @@ struct CoMutex {
  */
 void qemu_co_mutex_init(CoMutex *mutex);
 
-/**
- * Locks the mutex. If the lock cannot be taken immediately, control is
- * transferred to the caller of the current coroutine.
- */
-void coroutine_fn qemu_co_mutex_lock(CoMutex *mutex);
-
-/**
- * Unlocks

[PATCH experiment 12/16] remove "new" keyword from trace-events

[Paolo Bonzini]

This is invalid in C++.

Signed-off-by: Paolo Bonzini 
---
 util/trace-events | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/util/trace-events b/util/trace-events
index 5bc718eff7..9e23c11d11 100644
--- a/util/trace-events
+++ b/util/trace-events
@@ -3,8 +3,8 @@
 # aio-posix.c
 run_poll_handlers_begin(void *ctx, int64_t max_ns, int64_t timeout) "ctx %p 
max_ns %" PRId64 " timeout %" PRId64
 run_poll_handlers_end(void *ctx, bool progress, int64_t timeout) "ctx %p 
progress %d new timeout %" PRId64
-poll_shrink(void *ctx, int64_t old, int64_t new) "ctx %p old %" PRId64" new %" 
PRId64
-poll_grow(void *ctx, int64_t old, int64_t new) "ctx %p old %" PRId64" new %" 
PRId64
+poll_shrink(void *ctx, int64_t prev, int64_t curr) "ctx %p old %" PRId64" new 
%" PRId64
+poll_grow(void *ctx, int64_t prev, int64_t curr) "ctx %p old %" PRId64" new %" 
PRId64
 poll_add(void *ctx, void *node, int fd, unsigned revents) "ctx %p node %p fd 
%d revents 0x%x"
 poll_remove(void *ctx, void *node, int fd) "ctx %p node %p fd %d"
 
@@ -57,13 +57,13 @@ hbitmap_reset(void *hb, uint64_t start, uint64_t count, 
uint64_t sbit, uint64_t
 hbitmap_set(void *hb, uint64_t start, uint64_t count, uint64_t sbit, uint64_t 
ebit) "hb %p items %" PRIu64",%" PRIu64" bits %" PRIu64"..%" PRIu64
 
 # lockcnt.c
-lockcnt_fast_path_attempt(const void *lockcnt, int expected, int new) "lockcnt 
%p fast path %d->%d"
-lockcnt_fast_path_success(const void *lockcnt, int expected, int new) "lockcnt 
%p fast path %d->%d succeeded"
-lockcnt_unlock_attempt(const void *lockcnt, int expected, int new) "lockcnt %p 
unlock %d->%d"
-lockcnt_unlock_success(const void *lockcnt, int expected, int new) "lockcnt %p 
unlock %d->%d succeeded"
-lockcnt_futex_wait_prepare(const void *lockcnt, int expected, int new) 
"lockcnt %p preparing slow path %d->%d"
+lockcnt_fast_path_attempt(const void *lockcnt, int expected, int newval) 
"lockcnt %p fast path %d->%d"
+lockcnt_fast_path_success(const void *lockcnt, int expected, int newval) 
"lockcnt %p fast path %d->%d succeeded"
+lockcnt_unlock_attempt(const void *lockcnt, int expected, int newval) "lockcnt 
%p unlock %d->%d"
+lockcnt_unlock_success(const void *lockcnt, int expected, int newval) "lockcnt 
%p unlock %d->%d succeeded"
+lockcnt_futex_wait_prepare(const void *lockcnt, int expected, int newval) 
"lockcnt %p preparing slow path %d->%d"
 lockcnt_futex_wait(const void *lockcnt, int val) "lockcnt %p waiting on %d"
-lockcnt_futex_wait_resume(const void *lockcnt, int new) "lockcnt %p after 
wait: %d"
+lockcnt_futex_wait_resume(const void *lockcnt, int newval) "lockcnt %p after 
wait: %d"
 lockcnt_futex_wake(const void *lockcnt) "lockcnt %p waking up one waiter"
 
 # qemu-sockets.c
-- 
2.35.1

[PATCH experiment 10/16] add space between liter and string macro

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 util/trace-events | 28 ++--
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/util/trace-events b/util/trace-events
index c8f53d7d9f..5bc718eff7 100644
--- a/util/trace-events
+++ b/util/trace-events
@@ -1,10 +1,10 @@
 # See docs/devel/tracing.rst for syntax documentation.
 
 # aio-posix.c
-run_poll_handlers_begin(void *ctx, int64_t max_ns, int64_t timeout) "ctx %p 
max_ns %"PRId64 " timeout %"PRId64
-run_poll_handlers_end(void *ctx, bool progress, int64_t timeout) "ctx %p 
progress %d new timeout %"PRId64
-poll_shrink(void *ctx, int64_t old, int64_t new) "ctx %p old %"PRId64" new 
%"PRId64
-poll_grow(void *ctx, int64_t old, int64_t new) "ctx %p old %"PRId64" new 
%"PRId64
+run_poll_handlers_begin(void *ctx, int64_t max_ns, int64_t timeout) "ctx %p 
max_ns %" PRId64 " timeout %" PRId64
+run_poll_handlers_end(void *ctx, bool progress, int64_t timeout) "ctx %p 
progress %d new timeout %" PRId64
+poll_shrink(void *ctx, int64_t old, int64_t new) "ctx %p old %" PRId64" new %" 
PRId64
+poll_grow(void *ctx, int64_t old, int64_t new) "ctx %p old %" PRId64" new %" 
PRId64
 poll_add(void *ctx, void *node, int fd, unsigned revents) "ctx %p node %p fd 
%d revents 0x%x"
 poll_remove(void *ctx, void *node, int fd) "ctx %p node %p fd %d"
 
@@ -52,9 +52,9 @@ qemu_vfree(void *ptr) "ptr %p"
 qemu_anon_ram_free(void *ptr, size_t size) "ptr %p size %zu"
 
 # hbitmap.c
-hbitmap_iter_skip_words(const void *hb, void *hbi, uint64_t pos, unsigned long 
cur) "hb %p hbi %p pos %"PRId64" cur 0x%lx"
-hbitmap_reset(void *hb, uint64_t start, uint64_t count, uint64_t sbit, 
uint64_t ebit) "hb %p items %"PRIu64",%"PRIu64" bits %"PRIu64"..%"PRIu64
-hbitmap_set(void *hb, uint64_t start, uint64_t count, uint64_t sbit, uint64_t 
ebit) "hb %p items %"PRIu64",%"PRIu64" bits %"PRIu64"..%"PRIu64
+hbitmap_iter_skip_words(const void *hb, void *hbi, uint64_t pos, unsigned long 
cur) "hb %p hbi %p pos %" PRId64" cur 0x%lx"
+hbitmap_reset(void *hb, uint64_t start, uint64_t count, uint64_t sbit, 
uint64_t ebit) "hb %p items %" PRIu64",%" PRIu64" bits %" PRIu64"..%" PRIu64
+hbitmap_set(void *hb, uint64_t start, uint64_t count, uint64_t sbit, uint64_t 
ebit) "hb %p items %" PRIu64",%" PRIu64" bits %" PRIu64"..%" PRIu64
 
 # lockcnt.c
 lockcnt_fast_path_attempt(const void *lockcnt, int expected, int new) "lockcnt 
%p fast path %d->%d"
@@ -82,15 +82,15 @@ qemu_vfio_ram_block_added(void *s, void *p, size_t size) "s 
%p host %p size 0x%z
 qemu_vfio_ram_block_removed(void *s, void *p, size_t size) "s %p host %p size 
0x%zx"
 qemu_vfio_dump_mapping(void *host, uint64_t iova, size_t size) "vfio mapping 
%p to iova 0x%08" PRIx64 " size 0x%zx"
 qemu_vfio_find_mapping(void *s, void *p) "s %p host %p"
-qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t 
iova) "s %p host %p size 0x%zx index %d iova 0x%"PRIx64
-qemu_vfio_do_mapping(void *s, void *host, uint64_t iova, size_t size) "s %p 
host %p <-> iova 0x%"PRIx64 " size 0x%zx"
+qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t 
iova) "s %p host %p size 0x%zx index %d iova 0x%" PRIx64
+qemu_vfio_do_mapping(void *s, void *host, uint64_t iova, size_t size) "s %p 
host %p <-> iova 0x%" PRIx64 " size 0x%zx"
 qemu_vfio_dma_map(void *s, void *host, size_t size, bool temporary, uint64_t 
*iova) "s %p host %p size 0x%zx temporary %d &iova %p"
-qemu_vfio_dma_mapped(void *s, void *host, uint64_t iova, size_t size) "s %p 
host %p <-> iova 0x%"PRIx64" size 0x%zx"
+qemu_vfio_dma_mapped(void *s, void *host, uint64_t iova, size_t size) "s %p 
host %p <-> iova 0x%" PRIx64" size 0x%zx"
 qemu_vfio_dma_unmap(void *s, void *host) "s %p host %p"
-qemu_vfio_pci_read_config(void *buf, int ofs, int size, uint64_t region_ofs, 
uint64_t region_size) "read cfg ptr %p ofs 0x%x size 0x%x (region addr 
0x%"PRIx64" size 0x%"PRIx64")"
-qemu_vfio_pci_write_config(void *buf, int ofs, int size, uint64_t region_ofs, 
uint64_t region_size) "write cfg ptr %p ofs 0x%x size 0x%x (region addr 
0x%"PRIx64" size 0x%"PRIx64")"
-qemu_vfio_region_info(const char *desc, uint64_t region_ofs, uint64_t 
region_size, uint32_t cap_offset) "region '%s' addr 0x%"PRIx64" size 
0x%"PRIx64" cap_ofs 0x%"PRIx32
-qemu_vfio_pci_map_bar(int index, uint64_t region_ofs, uint64_t region_size, 
int ofs, void *host) "map region bar#%d addr 0x%"PRIx64" size 0x%"PRIx64" ofs 
0x%x host %p"
+qemu_vfio_pci_read_config(void *buf, int ofs, int size, uint64_t region_ofs, 
uint64_t region_size) "read cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%" 
PRIx64" size 0x%" PRIx64")"
+qemu_vfio_pci_write_config(void *buf, int ofs, int size, uint64_t region_ofs, 
uint64_t region_size) "write cfg ptr %p ofs 0x%x size 0x%x (region addr 0x%" 
PRIx64" size 0x%" PRIx64")"
+qemu_vfio_region_info(const char *desc, uint64_t region_ofs, uint64_t 
region_size, uint32_t cap_offset) "region '%s' addr 0x%" PRIx64" size 0x%" 
PRIx64" cap_ofs 0x%" PRIx32
+qemu_vfio_pci_map_bar(int index, uint

[PATCH experiment 13/16] disable some code

[Paolo Bonzini]

Disable a lot of code that I can't be bothered to convert right now.

Signed-off-by: Paolo Bonzini 
---
 include/qemu/coroutine.h|  2 ++
 qapi/qmp-dispatch.c |  2 ++
 tests/unit/meson.build  |  2 +-
 tests/unit/test-coroutine.c |  6 ++
 util/async.c|  2 ++
 util/meson.build| 10 +-
 util/qemu-coroutine-lock.c  |  2 ++
 util/qemu-coroutine-sleep.c |  2 ++
 util/thread-pool.c  |  2 ++
 9 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index 428e97d946..ac9891502e 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -200,6 +200,7 @@ typedef struct CoQueue {
  */
 void qemu_co_queue_init(CoQueue *queue);
 
+#if 0
 /**
  * Adds the current coroutine to the CoQueue and transfers control to the
  * caller of the coroutine.  The mutex is unlocked during the wait and
@@ -208,6 +209,7 @@ void qemu_co_queue_init(CoQueue *queue);
 #define qemu_co_queue_wait(queue, lock) \
 qemu_co_queue_wait_impl(queue, QEMU_MAKE_CO_LOCKABLE(lock))
 void coroutine_fn qemu_co_queue_wait_impl(CoQueue *queue, QemuCoLockable 
*lock);
+#endif
 
 /**
  * Removes the next coroutine from the CoQueue, and wake it up.
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index d378bccac7..6a4633c133 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -235,7 +235,9 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject 
*request,
 };
 aio_bh_schedule_oneshot(qemu_get_aio_context(), do_qmp_dispatch_bh,
 &data);
+#if 0
 qemu_coroutine_yield();
+#endif
 }
 qobject_unref(args);
 if (err) {
diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index 96b295263e..4ca5fdb699 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -61,7 +61,7 @@ endif
 
 if have_block
   tests += {
-'test-coroutine': [testblock],
+'test-coroutine': [],
 'test-aio': [testblock],
 'test-aio-multithread': [testblock],
 'test-throttle': [testblock],
diff --git a/tests/unit/test-coroutine.c b/tests/unit/test-coroutine.c
index 82e22db070..c230c2fa6e 100644
--- a/tests/unit/test-coroutine.c
+++ b/tests/unit/test-coroutine.c
@@ -16,6 +16,7 @@
 #include "qemu/coroutine_int.h"
 #include "qemu/lockable.h"
 
+#if 0
 /*
  * Check that qemu_in_coroutine() works
  */
@@ -638,11 +639,13 @@ static void perf_cost(void)
duration, ops,
(unsigned long)(10.0 * duration / maxcycles));
 }
+#endif
 
 int main(int argc, char **argv)
 {
 g_test_init(&argc, &argv, NULL);
 
+#if 0
 /* This test assumes there is a freelist and marks freed coroutine memory
  * with a sentinel value.  If there is no freelist this would legitimately
  * crash, so skip it.
@@ -650,7 +653,9 @@ int main(int argc, char **argv)
 if (CONFIG_COROUTINE_POOL) {
 g_test_add_func("/basic/no-dangling-access", test_no_dangling_access);
 }
+#endif
 
+#if 0
 g_test_add_func("/basic/lifecycle", test_lifecycle);
 g_test_add_func("/basic/yield", test_yield);
 g_test_add_func("/basic/nesting", test_nesting);
@@ -669,5 +674,6 @@ int main(int argc, char **argv)
 g_test_add_func("/perf/function-call", perf_baseline);
 g_test_add_func("/perf/cost", perf_cost);
 }
+#endif
 return g_test_run();
 }
diff --git a/util/async.c b/util/async.c
index 2ea1172f3e..95a9e0f95f 100644
--- a/util/async.c
+++ b/util/async.c
@@ -595,6 +595,7 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
 aio_context_unref(ctx);
 }
 
+#if 0
 typedef struct AioCoRescheduleSelf {
 Coroutine *co;
 AioContext *new_ctx;
@@ -624,6 +625,7 @@ void coroutine_fn aio_co_reschedule_self(AioContext 
*new_ctx)
 qemu_coroutine_yield();
 }
 }
+#endif
 
 void aio_co_wake(struct Coroutine *co)
 {
diff --git a/util/meson.build b/util/meson.build
index f6ee74ad0c..30949cd481 100644
--- a/util/meson.build
+++ b/util/meson.build
@@ -76,13 +76,13 @@ if have_block
   util_ss.add(files('lockcnt.c'))
   util_ss.add(files('main-loop.c'))
   util_ss.add(files('nvdimm-utils.c'))
-  util_ss.add(files('qemu-coroutine.c', 'qemu-coroutine-lock.c', 
'qemu-coroutine-io.c'))
-  util_ss.add(when: 'CONFIG_LINUX', if_true: [
-files('vhost-user-server.c'), vhost_user
-  ])
+  util_ss.add(files('qemu-coroutine.c', 'qemu-coroutine-lock.c')) # 
'qemu-coroutine-io.c'
+# util_ss.add(when: 'CONFIG_LINUX', if_true: [
+#   files('vhost-user-server.c'), vhost_user
+# ])
   util_ss.add(files('block-helpers.c'))
   util_ss.add(files('qemu-coroutine-sleep.c'))
-  util_ss.add(files('qemu-co-shared-resource.c'))
+# util_ss.add(files('qemu-co-shared-resource.c'))
   util_ss.add(files('thread-pool.c', 'qemu-timer.c'))
   util_ss.add(files('readline.c'))
   util_ss.add(files('throttle.c'))
diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
index 3f12b53a31..d6c0565ba5 100644
--- a/util/qemu-coro

Re: [PATCH v2] block-qdict: Fix -Werror=maybe-uninitialized build failure

[Markus Armbruster]

Murilo Opsfelder Araujo  writes:

> Building QEMU on Fedora 37 (Rawhide Prerelease) ppc64le failed with the
> following error:
>
> $ ../configure --prefix=/usr/local/qemu-disabletcg 
> --target-list=ppc-softmmu,ppc64-softmmu --disable-tcg --disable-linux-user
> ...
> $ make -j$(nproc)
> ...
> FAILED: libqemuutil.a.p/qobject_block-qdict.c.o
> cc -m64 -mlittle-endian -Ilibqemuutil.a.p -I. -I.. 
> -Isubprojects/libvhost-user -I../subprojects/libvhost-user -Iqapi -Itrace 
> -Iui -Iui/shader -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include 
> -I/usr/include/sysprof-4 -I/usr/include/lib
> mount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 
> -I/usr/include/p11-kit-1 -I/usr/include/pixman-1 -fdiagnostics-color=auto 
> -Wall -Winvalid-pch -Werror -std=gnu11 -O2 -g -isystem 
> /root/qemu/linux-headers -isystem linux-headers -iquote
>  . -iquote /root/qemu -iquote /root/qemu/include -iquote 
> /root/qemu/disas/libvixl -pthread -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 
> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
> -Wredundant-decls -Wundef -Wwrite
> -strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv 
> -Wold-style-declaration -Wold-style-definition -Wtype-limits 
> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body 
> -Wnested-externs -Wendif-label
> s -Wexpansion-to-defined -Wimplicit-fallthrough=2 
> -Wno-missing-include-dirs -Wno-shift-negative-value -Wno-psabi 
> -fstack-protector-strong -fPIE -MD -MQ 
> libqemuutil.a.p/qobject_block-qdict.c.o -MF 
> libqemuutil.a.p/qobject_block-qdict.c.o.d -
> o libqemuutil.a.p/qobject_block-qdict.c.o -c ../qobject/block-qdict.c
> In file included from /root/qemu/include/qapi/qmp/qdict.h:16,
>  from /root/qemu/include/block/qdict.h:13,
>  from ../qobject/block-qdict.c:11:
> /root/qemu/include/qapi/qmp/qobject.h: In function ‘qdict_array_split’:
> /root/qemu/include/qapi/qmp/qobject.h:49:17: error: ‘subqdict’ may be 
> used uninitialized [-Werror=maybe-uninitialized]
>49 | typeof(obj) _obj = (obj);   \
>   | ^~~~
> ../qobject/block-qdict.c:227:16: note: ‘subqdict’ declared here
>   227 | QDict *subqdict;
>   |^~~~
> cc1: all warnings being treated as errors
>
> Fix build failure by expanding the ternary operation.
> Tested with `make check-unit` (the check-block-qdict test passed).
>
> Signed-off-by: Murilo Opsfelder Araujo 
> Cc: Kevin Wolf 
> Cc: Hanna Reitz 
> Cc: Markus Armbruster 
> ---
> v1: https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg03224.html
>
>  qobject/block-qdict.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
> index 1487cc5dd8..4a83bda2c3 100644
> --- a/qobject/block-qdict.c
> +++ b/qobject/block-qdict.c
> @@ -251,12 +251,12 @@ void qdict_array_split(QDict *src, QList **dst)
>  if (is_subqdict) {
>  qdict_extract_subqdict(src, &subqdict, prefix);
>  assert(qdict_size(subqdict) > 0);
> +qlist_append_obj(*dst, QOBJECT(subqdict));
>  } else {
>  qobject_ref(subqobj);
>  qdict_del(src, indexstr);
> +qlist_append_obj(*dst, subqobj);
>  }
> -
> -qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
>  }
>  }

Reviewed-by: Markus Armbruster 

Re: [RESEND PATCH v3 5/7] target/riscv: csr: Hook debug CSR read/write

[Bin Meng]

On Wed, Jan 19, 2022 at 11:06 AM Alistair Francis  wrote:
>
> On Wed, Jan 5, 2022 at 1:15 PM Bin Meng  wrote:
> >
> > From: Bin Meng 
> >
> > This adds debug CSR read/write support to the RISC-V CSR RW table.
> >
> > Signed-off-by: Bin Meng 
> > ---
> >
> > Changes in v3:
> > - add riscv_trigger_init(), moved from patch #1 to this patch
> >
> >  target/riscv/debug.h |  2 ++
> >  target/riscv/cpu.c   |  6 +
> >  target/riscv/csr.c   | 57 
> >  target/riscv/debug.c | 27 +
> >  4 files changed, 92 insertions(+)
> >
> > diff --git a/target/riscv/debug.h b/target/riscv/debug.h
> > index d0f63e2414..f4da2db35d 100644
> > --- a/target/riscv/debug.h
> > +++ b/target/riscv/debug.h
> > @@ -109,4 +109,6 @@ void riscv_cpu_debug_excp_handler(CPUState *cs);
> >  bool riscv_cpu_debug_check_breakpoint(CPUState *cs);
> >  bool riscv_cpu_debug_check_watchpoint(CPUState *cs, CPUWatchpoint *wp);
> >
> > +void riscv_trigger_init(CPURISCVState *env);
> > +
> >  #endif /* RISCV_DEBUG_H */
> > diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
> > index d36c31ce9a..17dcc3c14f 100644
> > --- a/target/riscv/cpu.c
> > +++ b/target/riscv/cpu.c
> > @@ -575,6 +575,12 @@ static void riscv_cpu_realize(DeviceState *dev, Error 
> > **errp)
> >
> >  riscv_cpu_register_gdb_regs_for_features(cs);
> >
> > +#ifndef CONFIG_USER_ONLY
> > +if (riscv_feature(env, RISCV_FEATURE_DEBUG)) {
> > +riscv_trigger_init(env);
> > +}
> > +#endif
> > +
> >  qemu_init_vcpu(cs);
> >  cpu_reset(cs);
> >
> > diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> > index 146447eac5..189b9cc8c6 100644
> > --- a/target/riscv/csr.c
> > +++ b/target/riscv/csr.c
> > @@ -220,6 +220,15 @@ static RISCVException epmp(CPURISCVState *env, int 
> > csrno)
> >
> >  return RISCV_EXCP_ILLEGAL_INST;
> >  }
> > +
> > +static RISCVException debug(CPURISCVState *env, int csrno)
> > +{
> > +if (riscv_feature(env, RISCV_FEATURE_DEBUG)) {
> > +return RISCV_EXCP_NONE;
> > +}
> > +
> > +return RISCV_EXCP_ILLEGAL_INST;
> > +}
> >  #endif
> >
> >  /* User Floating-Point CSRs */
> > @@ -1464,6 +1473,48 @@ static RISCVException write_pmpaddr(CPURISCVState 
> > *env, int csrno,
> >  return RISCV_EXCP_NONE;
> >  }
> >
> > +static RISCVException read_tselect(CPURISCVState *env, int csrno,
> > +   target_ulong *val)
> > +{
> > +*val = tselect_csr_read(env);
> > +return RISCV_EXCP_NONE;
> > +}
> > +
> > +static RISCVException write_tselect(CPURISCVState *env, int csrno,
> > +target_ulong val)
> > +{
> > +tselect_csr_write(env, val);
> > +return RISCV_EXCP_NONE;
> > +}
> > +
> > +static RISCVException read_tdata(CPURISCVState *env, int csrno,
> > + target_ulong *val)
> > +{
> > +/* return 0 in tdata1 to end the trigger enumeration */
> > +if (env->trigger_cur >= TRIGGER_NUM && csrno == CSR_TDATA1) {
> > +*val = 0;
> > +return RISCV_EXCP_NONE;
> > +}
> > +
> > +if (!tdata_available(env, csrno - CSR_TDATA1)) {
> > +return RISCV_EXCP_ILLEGAL_INST;
> > +}
> > +
> > +*val = tdata_csr_read(env, csrno - CSR_TDATA1);
> > +return RISCV_EXCP_NONE;
> > +}
> > +
> > +static RISCVException write_tdata(CPURISCVState *env, int csrno,
> > +  target_ulong val)
> > +{
> > +if (!tdata_available(env, csrno - CSR_TDATA1)) {
> > +return RISCV_EXCP_ILLEGAL_INST;
> > +}
> > +
> > +tdata_csr_write(env, csrno - CSR_TDATA1, val);
> > +return RISCV_EXCP_NONE;
> > +}
> > +
> >  /*
> >   * Functions to access Pointer Masking feature registers
> >   * We have to check if current priv lvl could modify
> > @@ -1962,6 +2013,12 @@ riscv_csr_operations csr_ops[CSR_TABLE_SIZE] = {
> >  [CSR_PMPADDR14] =  { "pmpaddr14", pmp, read_pmpaddr, write_pmpaddr },
> >  [CSR_PMPADDR15] =  { "pmpaddr15", pmp, read_pmpaddr, write_pmpaddr },
> >
> > +/* Debug CSRs */
> > +[CSR_TSELECT]   =  { "tselect", debug, read_tselect, write_tselect },
> > +[CSR_TDATA1]=  { "tdata1",  debug, read_tdata,   write_tdata   },
> > +[CSR_TDATA2]=  { "tdata2",  debug, read_tdata,   write_tdata   },
> > +[CSR_TDATA3]=  { "tdata3",  debug, read_tdata,   write_tdata   },
> > +
> >  /* User Pointer Masking */
> >  [CSR_UMTE]={ "umte",pointer_masking, read_umte,
> > write_umte},
> >  [CSR_UPMMASK] ={ "upmmask", pointer_masking, read_upmmask, 
> > write_upmmask },
> > diff --git a/target/riscv/debug.c b/target/riscv/debug.c
> > index 7760c4611f..041a0d3a89 100644
> > --- a/target/riscv/debug.c
> > +++ b/target/riscv/debug.c
> > @@ -412,3 +412,30 @@ bool riscv_cpu_debug_check_watchpoint(CPUState *cs, 
> > CPUWatchpoint *wp)
> >
> >  return false;
> >  }
> > +
> > +void riscv_trigger_init(CPURISCVState *env)
> > +{
> > +target_ulong type2 = trigger_

Re: [PATCH v2] tests/avocado: starts PhoneServer upfront

[Thomas Huth]

On 11/03/2022 18.11, Beraldo Leal wrote:

Race conditions can happen with the current code, because the port that
was available might not be anymore by the time the server is started.

By setting the port to 0, PhoneServer it will use the OS default
behavior to get a free port, then we save this information so we can
later configure the guest.

Suggested-by: Daniel P. Berrangé 
Signed-off-by: Beraldo Leal 
---
  tests/avocado/avocado_qemu/__init__.py | 13 +++--
  1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/tests/avocado/avocado_qemu/__init__.py 
b/tests/avocado/avocado_qemu/__init__.py
index 9b056b5ce5..ac85e36a4d 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -18,7 +18,7 @@
  import uuid
  
  import avocado

-from avocado.utils import cloudinit, datadrainer, network, process, ssh, 
vmimage
+from avocado.utils import cloudinit, datadrainer, process, ssh, vmimage
  from avocado.utils.path import find_command
  
  #: The QEMU build root directory.  It may also be the source directory

@@ -602,9 +602,6 @@ def prepare_cloudinit(self, ssh_pubkey=None):
  self.log.info('Preparing cloudinit image')
  try:
  cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
-self.phone_home_port = network.find_free_port()
-if not self.phone_home_port:
-self.cancel('Failed to get a free port')
  pubkey_content = None
  if ssh_pubkey:
  with open(ssh_pubkey) as pubkey:
@@ -614,7 +611,7 @@ def prepare_cloudinit(self, ssh_pubkey=None):
password=self.password,
# QEMU's hard coded usermode router address
phone_home_host='10.0.2.2',
-  phone_home_port=self.phone_home_port,
+  phone_home_port=self.phone_server.server_port,
authorized_key=pubkey_content)
  except Exception:
  self.cancel('Failed to prepare the cloudinit image')
@@ -625,6 +622,8 @@ def set_up_boot(self):
  self.vm.add_args('-drive', 'file=%s' % path)
  
  def set_up_cloudinit(self, ssh_pubkey=None):

+self.phone_server = cloudinit.PhoneHomeServer(('0.0.0.0', 0),
+  self.name)
  cloudinit_iso = self.prepare_cloudinit(ssh_pubkey)
  self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
  
@@ -635,7 +634,9 @@ def launch_and_wait(self, set_up_ssh_connection=True):

   
logger=self.log.getChild('console'))
  console_drainer.start()
  self.log.info('VM launched, waiting for boot confirmation from guest')
-cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), 
self.name)
+while not self.phone_server.instance_phoned_back:
+self.phone_server.handle_request()
+
  if set_up_ssh_connection:
  self.log.info('Setting up the SSH connection')
  self.ssh_connect(self.username, self.ssh_key)


Thank you very much, this indeed fixes my problem with find_free_port() in 
the related tests:


Tested-by: Thomas Huth 

Re: [PATCH v5 0/5] user creatable pnv-phb4 devices

[Cédric Le Goater]

On 3/10/22 19:49, Thomas Huth wrote:

On 11/01/2022 14.10, Daniel Henrique Barboza wrote:

Hi,

This version implements Cedric's review suggestions from v4. No
drastic design changes were made.

Changes from v4:
- patches 1,3,5: unchanged
- patch 2:
   * renamed function to pnv_phb4_xscom_realize()
   * pnv4_phb4_xscom_realize() is now called at the end of phb4_realize()
- patch 4:
   * changed pnv_phb4_get_stack signature to use chip and phb
   * added a new helper called pnv_pec_stk_default_phb_realize() to
realize the default phb when running with defaults
- v4 link: https://lists.gnu.org/archive/html/qemu-devel/2022-01/msg02148.html

Daniel Henrique Barboza (5):
   ppc/pnv: set phb4 properties in stk_realize()
   ppc/pnv: move PHB4 XSCOM init to phb4_realize()
   ppc/pnv: turn 'phb' into a pointer in struct PnvPhb4PecStack
   ppc/pnv: Introduce user creatable pnv-phb4 devices
   ppc/pnv: turn pnv_phb4_update_regions() into static


It's now possible to crash QEMU with the pnv-phb4 device:

$ ./qemu-system-ppc64 -nographic -M powernv9 -device pnv-phb4
Unexpected error in object_property_try_add() at 
../../devel/qemu/qom/object.c:1229:
qemu-system-ppc64: -device pnv-phb4: attempt to add duplicate property 
'pnv-phb4[0]' to object (type 'power9_v2.0-pnv-chip')
Aborted (core dumped)

Any ideas how to fix this?


This was introduced by :

  commit 6e7b96750359 ("ppc/pnv: fix default PHB4 QOM hierarchy")

It could be fixed with :

@@ -1598,15 +1598,15 @@ static void pnv_phb4_realize(DeviceState
 error_propagate(errp, local_err);
 return;
 }
-}
 
-/* Reparent the PHB to the chip to build the device tree */

-pnv_chip_parent_fixup(chip, OBJECT(phb), phb->phb_id);
+/* Reparent the PHB to the chip to build the device tree */
+pnv_chip_parent_fixup(chip, OBJECT(phb), phb->phb_id);
 
-s = qdev_get_parent_bus(DEVICE(chip));

-if (!qdev_set_parent_bus(DEVICE(phb), s, &local_err)) {
-error_propagate(errp, local_err);
-return;
+s = qdev_get_parent_bus(DEVICE(chip));
+if (!qdev_set_parent_bus(DEVICE(phb), s, &local_err)) {
+error_propagate(errp, local_err);
+return;
+}
 }
 
 /* Set the "big_phb" flag */



but I am not sure we want to keep user-created PHB* devices.

Thanks,

C.

Re: [PATCH 5/5] s390x: pv: Add dump support

[Janosch Frank]

On 3/12/22 15:52, Richard Henderson wrote:

On 3/10/22 03:25, Janosch Frank wrote:

Sometimes dumping a guest from the outside is the only way to get the
data that is needed. This can be the case if a dumping mechanism like
KDUMP hasn't been configured or data needs to be fetched at a specific
point. Dumping a protected guest from the outside without help from
fw/hw doesn't yield sufficient data to be useful. Hence we now
introduce PV dump support.

The PV dump support works by integrating the firmware into the dump
process. New Ultravisor calls are used to initiate the dump process,
dump cpu data, dump memory state and lastly complete the dump process.
The UV calls are exposed by KVM via the new KVM_PV_DUMP command and
its subcommands. The guest's data is fully encrypted and can only be
decrypted by the entity that owns the customer communication key for
the dumped guest. Also dumping needs to be allowed via a flag in the
SE header.

On the QEMU side of things we store the PV dump data in the newly
introduced architecture ELF sections (storage state and completion
data) and the cpu notes (for cpu dump data).

Users can use a tool to convert the encrypted QEMU dump to an
unencrypted one.

Signed-off-by: Janosch Frank 
---
   dump/dump.c  |  27 -
   include/elf.h|   1 +
   include/sysemu/dump.h|  27 +
   target/s390x/arch_dump.c | 247 ++-
   4 files changed, 244 insertions(+), 58 deletions(-)

diff --git a/dump/dump.c b/dump/dump.c
index a9d9ea514d..4b7d333866 100644
--- a/dump/dump.c
+++ b/dump/dump.c
@@ -708,33 +708,6 @@ static void dump_begin(DumpState *s, Error **errp)
   write_elf_notes(s, errp);
   }
   
-static int get_next_block(DumpState *s, GuestPhysBlock *block)

-{
-while (1) {
-block = QTAILQ_NEXT(block, next);
-if (!block) {
-/* no more block */
-return 1;
-}
-
-s->start = 0;
-s->next_block = block;
-if (s->has_filter) {
-if (block->target_start >= s->begin + s->length ||
-block->target_end <= s->begin) {
-/* This block is out of the range */
-continue;
-}
-
-if (s->begin > block->target_start) {
-s->start = s->begin - block->target_start;
-}
-}
-
-return 0;
-}
-}


No reason to make this inline, surely.  Just export it.
Anyway, this should be it's own patch.


Yeah, will do.




@@ -76,9 +77,12 @@ typedef struct noteStruct {
   uint64_t todcmp;
   uint32_t todpreg;
   uint64_t ctrs[16];
+uint8_t dynamic;  /* Real size comes from PV info interface */


This needs a better comment.  Possibly

  uint8_t dynamic[1];  /* Would be a flexible array member, if that was 
legal inside a
union. Real size comes from PV info interface. */

Alternately, rewrite everything to work with

typedef struct noteStruct {
  Elf64_Nhdr hdr;
  char name[8];
  uint8_t contents[];
} QEMU_PACKED Note;

which is a better description of what's going on in this file anyway.


I'll have a look at that



r~

Re: [PATCH 3/3] tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

[David Hildenbrand]

On 14.03.22 09:30, Christian Borntraeger wrote:
> 
> 
> Am 11.03.22 um 21:32 schrieb Richard Henderson:
>> On 3/11/22 10:49, Ilya Leoshkevich wrote:
>>> +    size_t length = 0x10006;
>>> +    unsigned char *buf;
>>> +    int i;
>>> +
>>> +    buf = mmap(NULL, length, PROT_READ | PROT_WRITE | PROT_EXEC,
>>> +   MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>>> +    assert(buf != MAP_FAILED);
>>
>> I'm thinking exit success here, as such a large allocation may well fail 
>> depending on the host.
> 
> What about using MAP_NORESERVE ?

+1


-- 
Thanks,

David / dhildenb

Re: [PATCH v2 3/3] tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

[David Hildenbrand]

On 12.03.22 10:25, Ilya Leoshkevich wrote:
> Add a small test in order to prevent regressions.
> 
> Signed-off-by: Ilya Leoshkevich 
> ---
>  tests/tcg/s390x/Makefile.target|  1 +
>  tests/tcg/s390x/branch-relative-long.c | 60 ++
>  2 files changed, 61 insertions(+)
>  create mode 100644 tests/tcg/s390x/branch-relative-long.c
> 
> diff --git a/tests/tcg/s390x/Makefile.target b/tests/tcg/s390x/Makefile.target
> index 257c568c58..fd34b130f7 100644
> --- a/tests/tcg/s390x/Makefile.target
> +++ b/tests/tcg/s390x/Makefile.target
> @@ -15,6 +15,7 @@ TESTS+=mvc
>  TESTS+=shift
>  TESTS+=trap
>  TESTS+=signals-s390x
> +TESTS+=branch-relative-long
>  
>  ifneq ($(HAVE_GDB_BIN),)
>  GDB_SCRIPT=$(SRC_PATH)/tests/guest-debug/run-test.py
> diff --git a/tests/tcg/s390x/branch-relative-long.c 
> b/tests/tcg/s390x/branch-relative-long.c
> new file mode 100644
> index 00..c6f3f2db6d
> --- /dev/null
> +++ b/tests/tcg/s390x/branch-relative-long.c
> @@ -0,0 +1,60 @@
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +#define DEFINE_ASM(_name, _code) \
> +extern const char _name[]; \
> +extern const char _name ## _end[]; \
> +asm(".globl " #_name "\n" \
> +#_name ":\n" \
> +"" _code "\n" \
> +".globl " #_name "_end\n" \
> +#_name "_end:\n");
> +
> +DEFINE_ASM(br_r14, "br %r14");
> +DEFINE_ASM(brasl_r0, "brasl %r0,.-0x1");
> +DEFINE_ASM(brcl_0xf, "brcl 0xf,.-0x1");
> +
> +struct test {
> +const char *code;
> +const char *code_end;
> +};
> +
> +static const struct test tests[] = {
> +{
> +.code = brasl_r0,
> +.code_end = brasl_r0_end,
> +},
> +{
> +.code = brcl_0xf,
> +.code_end = brcl_0xf_end,
> +},
> +};
> +
> +int main(void)
> +{
> +size_t length = 0x10006;
> +unsigned char *buf;
> +void (*code)(void);
> +size_t i;
> +
> +buf = mmap(NULL, length, PROT_READ | PROT_WRITE | PROT_EXEC,
> +   MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

With MAP_NORESERVE

Reviewed-by: David Hildenbrand 

Thanks!

-- 
Thanks,

David / dhildenb

Re: [PATCH] x86: q35: require split irqchip for large CPU count

[Igor Mammedov]

On Fri, 11 Mar 2022 14:58:41 +
David Woodhouse  wrote:

> On Fri, 2022-03-11 at 09:39 -0500, Igor Mammedov wrote:
> > if VM is started with:
> > 
> >-enable-kvm -smp 256
> > 
> > without specifying 'split' irqchip, VM might eventually boot
> > but no more than 255 CPUs will be operational and following
> > error messages in guest could be observed:
> >...
> >smpboot: native_cpu_up: bad cpu 256
> >...
> > It's a regression introduced by [1], which removed dependency
> > on intremap=on that were implicitly requiring 'split' irqchip
> > and forgot to check for 'split' irqchip.
> > Instead of letting VM boot a broken VM, error out and tell
> > user how to fix CLI.  
> 
> Hm, wasn't that already fixed in the patches I posted in December?
It might be, could you point to the commit/series that fixed it.

Regardless of that, fixing it in recent kernels doesn't help
as still supported kernels are still affected by it.

If there is a way to detect that fix, I can add to q35 a compat
property and an extra logic to enable kernel-irqchip if fix is present.
Otherwise the fix does not exist until minimum supported kernel
version reaches version where it was fixed.

[PATCH v3 1/3] s390x/tcg: Fix BRASL with a large negative offset

[Ilya Leoshkevich]

When RI2 is 0x8000, qemu enters an infinite loop instead of jumping
backwards. Fix by adding a missing cast, like in in2_ri2().

Fixes: 8ac33cdb8bfb ("Convert BRANCH AND SAVE")
Reviewed-by: David Hildenbrand 
Reviewed-by: Richard Henderson 
Signed-off-by: Ilya Leoshkevich 
---
 target/s390x/tcg/translate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c
index 904b51542f..41c8696185 100644
--- a/target/s390x/tcg/translate.c
+++ b/target/s390x/tcg/translate.c
@@ -1597,7 +1597,7 @@ static DisasJumpType op_bal(DisasContext *s, DisasOps *o)
 static DisasJumpType op_basi(DisasContext *s, DisasOps *o)
 {
 pc_to_link_info(o->out, s, s->pc_tmp);
-return help_goto_direct(s, s->base.pc_next + 2 * get_field(s, i2));
+return help_goto_direct(s, s->base.pc_next + (int64_t)get_field(s, i2) * 
2);
 }
 
 static DisasJumpType op_bc(DisasContext *s, DisasOps *o)
-- 
2.35.1

[PATCH v3 0/3] Fix BRASL and BRCL with large negative offsets

[Ilya Leoshkevich]

Hi,

I noticed that sometimes jumping backwards leads to crashes or hangs.
The problem is a missing cast.
Patches 1 and 2 fix the problem, patch 3 adds a test.


v1: https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg03356.html
v1 -> v2:
- Skip the test if mmap() fails (Richard).
- Replace test opcodes with inline asm (David). Since we now want to
  skip the test if the code cannot be mapped (e.g. on a 31-bit host),
  we shouldn't be asking the loader to map the code right away. So
  the mmap() approach stays.

v2: https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg03387.html
v2 -> v3:
- Use MAP_NORESERVE (Christian / David).
- Unhardcode 0x10006 (Richard).

Best regards,
Ilya

Ilya Leoshkevich (3):
  s390x/tcg: Fix BRASL with a large negative offset
  s390x/tcg: Fix BRCL with a large negative offset
  tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

 target/s390x/tcg/translate.c   |  4 +-
 tests/tcg/s390x/Makefile.target|  1 +
 tests/tcg/s390x/branch-relative-long.c | 68 ++
 3 files changed, 71 insertions(+), 2 deletions(-)
 create mode 100644 tests/tcg/s390x/branch-relative-long.c

-- 
2.35.1

[PATCH v3 3/3] tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

[Ilya Leoshkevich]

Add a small test in order to prevent regressions.

Reviewed-by: Richard Henderson 
Reviewed-by: David Hildenbrand 
Signed-off-by: Ilya Leoshkevich 
---
 tests/tcg/s390x/Makefile.target|  1 +
 tests/tcg/s390x/branch-relative-long.c | 68 ++
 2 files changed, 69 insertions(+)
 create mode 100644 tests/tcg/s390x/branch-relative-long.c

diff --git a/tests/tcg/s390x/Makefile.target b/tests/tcg/s390x/Makefile.target
index 257c568c58..fd34b130f7 100644
--- a/tests/tcg/s390x/Makefile.target
+++ b/tests/tcg/s390x/Makefile.target
@@ -15,6 +15,7 @@ TESTS+=mvc
 TESTS+=shift
 TESTS+=trap
 TESTS+=signals-s390x
+TESTS+=branch-relative-long
 
 ifneq ($(HAVE_GDB_BIN),)
 GDB_SCRIPT=$(SRC_PATH)/tests/guest-debug/run-test.py
diff --git a/tests/tcg/s390x/branch-relative-long.c 
b/tests/tcg/s390x/branch-relative-long.c
new file mode 100644
index 00..94219afcad
--- /dev/null
+++ b/tests/tcg/s390x/branch-relative-long.c
@@ -0,0 +1,68 @@
+#include 
+#include 
+#include 
+#include 
+
+#define DEFINE_ASM(_name, _code) \
+extern const char _name[]; \
+extern const char _name ## _end[]; \
+asm(".globl " #_name "\n" \
+#_name ":\n" \
+"" _code "\n" \
+".globl " #_name "_end\n" \
+#_name "_end:\n");
+
+DEFINE_ASM(br_r14, "br %r14");
+DEFINE_ASM(brasl_r0, "brasl %r0,.-0x1");
+DEFINE_ASM(brcl_0xf, "brcl 0xf,.-0x1");
+
+struct test {
+const char *code;
+const char *code_end;
+};
+
+static const struct test tests[] = {
+{
+.code = brasl_r0,
+.code_end = brasl_r0_end,
+},
+{
+.code = brcl_0xf,
+.code_end = brcl_0xf_end,
+},
+};
+
+int main(void)
+{
+unsigned char *buf;
+size_t length = 0;
+size_t i;
+
+for (i = 0; i < sizeof(tests) / sizeof(tests[0]); i++) {
+size_t test_length = 0x1 + (tests[i].code_end - tests[i].code);
+
+if (test_length > length) {
+length = test_length;
+}
+}
+
+buf = mmap(NULL, length, PROT_READ | PROT_WRITE | PROT_EXEC,
+   MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
+if (buf == MAP_FAILED) {
+perror("SKIP: mmap() failed");
+return 0;
+}
+
+memcpy(buf, br_r14, br_r14_end - br_r14);
+for (i = 0; i < sizeof(tests) / sizeof(tests[0]); i++) {
+void (*code)(void) = (void *)(buf + 0x1);
+
+memcpy(code, tests[i].code, tests[i].code_end - tests[i].code);
+code();
+memset(code, 0, tests[i].code_end - tests[i].code);
+}
+
+munmap(buf, length);
+
+return 0;
+}
-- 
2.35.1

[PATCH v2 0/3] rSTify a few more docs; move them to QEMU Git

[Kashyap Chamarthy]

This series rST-ifies:

  - security-process[1]
  - MailingLists[2]
  - GettingStartedDevelopers[3]

The 'security-process' page is from the QEMU web and is moved to
docs/devel/ in QEMU Git.  This is based on Paolo's feedback here[4].
The next two docs are converted from the Wiki.

[1] https://www.qemu.org/contribute/security-process
[2] https://wiki.qemu.org/Contribute/MailingLists
[3] https://wiki.qemu.org/Documentation/GettingStartedDevelopers
[4] https://lists.nongnu.org/archive/html/qemu-devel/2021-11/msg04002.html

Kashyap Chamarthy (3):
  docs: rSTify "security-process" page; move it to QEMU Git
  docs: rSTify MailingLists wiki; move it to QEMU Git
  docs: rSTify GettingStartedDevelopers wiki; move it to QEMU Git

 docs/devel/getting-started-developers.rst | 200 ++
 docs/devel/index.rst  |   3 +
 docs/devel/mailing-lists.rst  |  53 ++
 docs/devel/security-process.rst   | 190 
 4 files changed, 446 insertions(+)
 create mode 100644 docs/devel/getting-started-developers.rst
 create mode 100644 docs/devel/mailing-lists.rst
 create mode 100644 docs/devel/security-process.rst

-- 
2.33.1

[PATCH v2 1/3] docs: rSTify "security-process" page; move it to QEMU Git

[Kashyap Chamarthy]

This is based on Paolo's suggestion[1] that the 'security-process'[2]
page being a candidate for docs/devel.

Converted from Markdown to rST using:

$> pandoc -f markdown -t rst security-process.md \
-o security-process.rst

It's a 1-1 conversion (I double-checked to the best I could).  I've also
checked that the hyperlinks work correctly post-conversion.

[1] https://lists.nongnu.org/archive/html/qemu-devel/2021-11/msg04002.html
[2] https://www.qemu.org/contribute/security-process

Suggested-by: Paolo Bonzini 
Signed-off-by: Kashyap Chamarthy 
Reviewed-by: Peter Maydell 
---
 docs/devel/index.rst|   1 +
 docs/devel/security-process.rst | 190 
 2 files changed, 191 insertions(+)
 create mode 100644 docs/devel/security-process.rst

diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index afd937535e..424eff9294 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -48,3 +48,4 @@ modifying QEMU's source code.
trivial-patches
submitting-a-patch
submitting-a-pull-request
+   security-process
diff --git a/docs/devel/security-process.rst b/docs/devel/security-process.rst
new file mode 100644
index 00..cc1000fe43
--- /dev/null
+++ b/docs/devel/security-process.rst
@@ -0,0 +1,190 @@
+.. _security-process:
+
+Security Process
+
+
+Please report any suspected security issue in QEMU to the security
+mailing list at:
+
+-  ` 
`__
+
+To report an issue via `GPG `__ encrypted email,
+please send it to the Red Hat Product Security team at:
+
+-  ` 
`__
+
+**Note:** after the triage, encrypted issue details shall be sent to the
+upstream ‘qemu-security’ mailing list for archival purposes.
+
+How to report an issue
+--
+
+-  Please include as many details as possible in the issue report. Ex:
+
+   -  QEMU version, upstream commit/tag
+   -  Host & Guest architecture x86/Arm/PPC, 32/64 bit etc.
+   -  Affected code area/snippets
+   -  Stack traces, crash details
+   -  Malicious inputs/reproducer steps etc.
+   -  Any configurations/settings required to trigger the issue.
+
+-  Please share the QEMU command line used to invoke a guest VM.
+
+-  Please specify whom to acknowledge for reporting this issue.
+
+How we respond
+~~
+
+-  Process of handling security issues comprises following steps:
+
+   0) **Acknowledge:**
+
+   -  A non-automated response email is sent to the reporter(s) to
+  acknowledge the reception of the report. (*60 day’s counter starts
+  here*)
+
+   1) **Triage:**
+
+   -  Examine the issue details and confirm whether the issue is genuine
+   -  Validate if it can be misused for malicious purposes
+   -  Determine its worst case impact and severity
+  [Low/Moderate/Important/Critical]
+
+   2) **Response:**
+
+   -  Negotiate embargo timeline (if required, depending on severity)
+   -  Request a `CVE `__ and open an
+  upstream `bug `__
+   -  Create an upstream fix patch annotated with
+
+  -  CVE-ID
+  -  Link to an upstream bugzilla
+  -  Reported-by, Tested-by etc. tags
+
+   -  Once the patch is merged, close the upstream bug with a link to
+  the commit
+
+  -  Fixed in:
+
+-  Above security lists are operated by select analysts, maintainers
+   and/or representatives from downstream communities.
+
+-  List members follow a **responsible disclosure** policy. Any
+   non-public information you share about security issues, is kept
+   confidential within members of the QEMU security team and a minimal
+   supporting staff in their affiliated companies. Such information will
+   not be disclosed to third party organisations/individuals without
+   prior permission from the reporter(s).
+
+-  We aim to process security issues within maximum of **60 days**. That
+   is not to say that issues will remain private for 60 days, nope.
+   After the triaging step above
+
+   -  If severity of the issue is sufficiently low, an upstream public
+  bug will be created immediately.
+   -  If severity of the issue requires co-ordinated disclosure at a
+  future date, then the embargo process below is followed, and
+  upstream bug will be opened at the end of the embargo period.
+
+   This will allow upstream contributors to create, test and track fix
+   patch(es).
+
+Publication embargo
+~~~
+
+-  If a security issue is reported that is not already public and its
+   severity requires coordinated disclosure, then an embargo date will
+   be set and communicated to the reporter(s).
+
+-  Embargo periods will be negotiated by mutual agreement between
+   reporter(s), members of the security list and other relevant parties
+   to the problem. The preferred embargo period is upto `2
+   weeks 

[PATCH v2 2/3] docs: rSTify MailingLists wiki; move it to QEMU Git

[Kashyap Chamarthy]

This document is referred to from the GettingStartedDevelopers wiki
which will be rSTified in a follow-up commit.

Converted from Mediawiki to rST using:

$> pandoc -f Mediawiki -t rst MailingLists.wiki
-o mailing-lists.rst

It's a 1-1 conversion (I double-checked to the best I could).  I've also
checked that the hyperlinks work correctly post-conversion.

Signed-off-by: Kashyap Chamarthy 
---
 docs/devel/index.rst |  1 +
 docs/devel/mailing-lists.rst | 53 
 2 files changed, 54 insertions(+)
 create mode 100644 docs/devel/mailing-lists.rst

diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 424eff9294..fb9d9f3a80 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -12,6 +12,7 @@ modifying QEMU's source code.
 
code-of-conduct
conflict-resolution
+   mailing-lists
build-system
style
kconfig
diff --git a/docs/devel/mailing-lists.rst b/docs/devel/mailing-lists.rst
new file mode 100644
index 00..53dcbfb007
--- /dev/null
+++ b/docs/devel/mailing-lists.rst
@@ -0,0 +1,53 @@
+.. _mailing-lists:
+
+Mailing lists
+=
+
+-  `QEMU developers mailing
+   list `__
+-  `QEMU stable mailing
+   list `__
+-  `QEMU trivial patch mailing
+   list `__
+-  `QEMU users mailing
+   list `__
+
+.. _subsystem_specific_lists:
+
+Subsystem Specific Lists
+
+
+These exist to make it a little easier to follow subsystem specific
+patches. You should however continue to CC qemu-devel so your series
+gets wide visibility.
+
+-  `QEMU ARM mailing
+   list `__
+-  `QEMU block devices mailing
+   list `__
+-  `QEMU PowerPC mailing
+   list `__
+-  `QEMU RISC-V mailing
+   list `__
+-  `QEMU s390x mailing
+   list `__
+
+If a subsystem maintainer thinks that a new mailing list for their
+subsystem would make life easier, we're happy to create one -- mail
+qemu-devel to suggest it (ideally cc'ing the people listed as Savannah
+project admins in our `AdminContacts `__ page, as they
+are the ones with the ability to make the change).
+
+If you are a Savannah project admin, you may want the `technical notes
+on how to create and configure a new
+list `__.
+
+.. _access_via_lore.kernel.org:
+
+Access via lore.kernel.org
+--
+
+The qemu-devel mailing list is also archived via
+`public-inbox `__ on
+https://lore.kernel.org/qemu-devel/ and accessible via NNTP at
+nntp.lore.kernel.org (newsgroup org.nongnu.qemu-devel).
-- 
2.33.1

[PATCH v2 3/3] docs: rSTify GettingStartedDevelopers wiki; move it to QEMU Git

[Kashyap Chamarthy]

Converted the wiki[1] from Markdown to rST using:

$> pandoc -f Mediawiki -t rst getting-started-developers.wiki
-o getting-started-developers.rst

It's a 1-1 conversion (I double-checked to the best I could).  I've also
checked that the hyperlinks work correctly post-conversion.

[1] https://wiki.qemu.org/Documentation/GettingStartedDevelopers

Signed-off-by: Kashyap Chamarthy 
---
 docs/devel/getting-started-developers.rst | 200 ++
 docs/devel/index.rst  |   1 +
 2 files changed, 201 insertions(+)
 create mode 100644 docs/devel/getting-started-developers.rst

diff --git a/docs/devel/getting-started-developers.rst 
b/docs/devel/getting-started-developers.rst
new file mode 100644
index 00..1299e1dfee
--- /dev/null
+++ b/docs/devel/getting-started-developers.rst
@@ -0,0 +1,200 @@
+.. _getting-started-developers:
+
+Getting Started for Developers
+==
+
+You want to contribute code, documentation or patches to QEMU?
+
+Then...
+
+-  ... you should probably first join the :ref:`mailing-lists`.
+
+   -  Mailing lists are moderated. Non-subscribers may post, so list
+  policy is reply-to-all to ensure original poster is included.
+   -  Be prepared for upwards of one thousand messages per week if you
+  subscribe.
+   -  First-time posts (whether subscribed or not) are subject to a
+  moderation delay until a human can whitelist your email address.
+
+-  Also check out the `patch
+   submission 
`__
+   page for some hints on mail contributions.
+
+Wiki
+
+
+-  To create an account in the QEMU wiki, you must ask on the mailing
+   list for someone else to do it on your behalf (self-creation is
+   prohibited to cut down on spam accounts).
+-  Start with reading the QEMU wiki.
+-  Contribute to the QEMU wiki by adding new topics or improving and
+   expanding existing topics. It should help you and others in the
+   future.
+
+Documentation
+-
+
+-  Continue with reading the `existing documentation `__
+   and `Contributions Guide `__.
+
+   Be prepared that all written documentation might be invalid - either
+   because it is too old or because it was never correct. And it is
+   never complete...
+
+-  If you find bugs in the documentation then fix them and send patches
+   to the mailing list. See
+   `Contribute/ReportABug `__.
+-  If you find problems in the wiki, then fix them if you can, or add
+   notes to either the applicable page or the Discussion page.
+-  Depending on how much computer architecture / hardware background you
+   have, it may help to read some general books. Suggestions include:
+
+   -  "Computers as Components, Second Edition: Principles of Embedded
+  Computing System Design", Wayne Wolf, ISBN-13: 978-0123743978
+
+Code
+
+
+-  Get the code. If you want to be a developer, you almost certainly
+   want to be building from git (see the
+   `Download `__ page for
+   the right tree).
+-  Compile the code. Here are some instructions how to do this:
+
+   -  `QEMU on Linux hosts `__
+   -  `QEMU on OS X (macOS) hosts `__
+   -  `QEMU on Windows hosts `__
+
+-  Run the QEMU system and user mode emulation for different targets
+   (x86, mips, powerpc, ...). Images can be obtained from the
+   `Testing `__ page.
+-  QEMU has a lot of different parts (hardware device emulation, target
+   emulation, code generation for different hosts, configuration, ...).
+
+   -  Choose an interesting part and concentrate on it for some time and
+  read the code. Its going to take some effort, so try to find
+  something that you are really interested in - something that will
+  be a least a little bit fun for you.
+   -  It will be easier if you choose a part of the code that has an
+  active / responsive maintainer (since this gives you someone to
+  discuss things with).
+
+-  If you find bugs in the code, then fix them and send a patch to the
+   mailing list (see `patch submission
+   process `__)
+
+   -  Patches need to go the mailing list, and possibly also to a
+  specific maintainer (read the MAINTAINERS text file in the top of
+  the source code tree).
+   -  Read the HACKING and CODING_STYLE text files (in the top of the
+  source code tree) before writing the patch
+   -  Run your patch through the . See
+  
http://blog.vmsplice.net/2011/03/how-to-automatically-run-checkpatchpl.html
+  for how to hook it into git.
+   -  For very small, simple changes, you can do it as a single patch.
+  If your change is more complex, you need to break it into smaller,
+  separate patches (which together form a set of patches, or a
+  patchset). Each step in the patch process can rely on previous
+  patches, but not later patches - otherwise "git bisect" will
+ 

[PATCH v3 2/3] s390x/tcg: Fix BRCL with a large negative offset

[Ilya Leoshkevich]

When RI2 is 0x8000, qemu enters an infinite loop instead of jumping
backwards. Fix by adding a missing cast, like in in2_ri2().

Fixes: 7233f2ed1717 ("target-s390: Convert BRANCH ON CONDITION")
Reviewed-by: David Hildenbrand 
Reviewed-by: Richard Henderson 
Signed-off-by: Ilya Leoshkevich 
---
 target/s390x/tcg/translate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c
index 41c8696185..5acfc0ff9b 100644
--- a/target/s390x/tcg/translate.c
+++ b/target/s390x/tcg/translate.c
@@ -1201,7 +1201,7 @@ static DisasJumpType help_branch(DisasContext *s, 
DisasCompare *c,
  bool is_imm, int imm, TCGv_i64 cdest)
 {
 DisasJumpType ret;
-uint64_t dest = s->base.pc_next + 2 * imm;
+uint64_t dest = s->base.pc_next + (int64_t)imm * 2;
 TCGLabel *lab;
 
 /* Take care of the special cases first.  */
-- 
2.35.1

Re: [PATCH experiment 06/16] use g_new0 instead of g_malloc0

[Markus Armbruster]

Paolo Bonzini  writes:

> Casting to/from void* must be explicit in C++.  g_new0 takes care of that.
>
> Signed-off-by: Paolo Bonzini 
> ---
>  include/qemu/timer.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/qemu/timer.h b/include/qemu/timer.h
> index 88ef114689..ee071e07d1 100644
> --- a/include/qemu/timer.h
> +++ b/include/qemu/timer.h
> @@ -520,7 +520,7 @@ static inline QEMUTimer 
> *timer_new_full(QEMUTimerListGroup *timer_list_group,
>  int scale, int attributes,
>  QEMUTimerCB *cb, void *opaque)
>  {
> -QEMUTimer *ts = g_malloc0(sizeof(QEMUTimer));
> +QEMUTimer *ts = g_new0(QEMUTimer, 1);
>  timer_init_full(ts, timer_list_group, type, scale, attributes, cb, 
> opaque);
>  return ts;
>  }

Looks like a rerun of commit b45c03f585's Coccinelle script is due.
I'll take care of it.

Re: [PATCH v1 1/1] hw: aspeed_scu: Add AST2600 hpll calculation function

[Cédric Le Goater]

Hello Steven,

On 3/14/22 10:54, Steven Lee wrote:

AST2600's HPLL register offset and bit definition are different from
AST2500. Add a hpll calculation function for ast2600 and modify apb frequency
calculation function based on SCU200 register description in ast2600v11.pdf.


It looks good. A few minor comments on the modeling.
 

Signed-off-by: Steven Lee 
---
  hw/misc/aspeed_scu.c | 43 
  include/hw/misc/aspeed_scu.h | 17 ++
  2 files changed, 56 insertions(+), 4 deletions(-)

diff --git a/hw/misc/aspeed_scu.c b/hw/misc/aspeed_scu.c
index d06e179a6e..3b11e98d66 100644
--- a/hw/misc/aspeed_scu.c
+++ b/hw/misc/aspeed_scu.c
@@ -205,6 +205,8 @@ static const uint32_t ast2500_a1_resets[ASPEED_SCU_NR_REGS] 
= {
   [BMC_DEV_ID]  = 0x2402U
  };
  
+static uint32_t aspeed_2600_scu_calc_hpll(AspeedSCUState *s, uint32_t hpll_reg);

+
  static uint32_t aspeed_scu_get_random(void)
  {
  uint32_t num;
@@ -215,9 +217,19 @@ static uint32_t aspeed_scu_get_random(void)
  uint32_t aspeed_scu_get_apb_freq(AspeedSCUState *s)
  {
  AspeedSCUClass *asc = ASPEED_SCU_GET_CLASS(s);
-uint32_t hpll = asc->calc_hpll(s, s->regs[HPLL_PARAM]);
+uint32_t hpll, hpll_reg, clk_sel_reg;
+
+if (asc->calc_hpll == aspeed_2600_scu_calc_hpll) {


That's indeed one way to distinguish the AST2600 from the previous SoCs.
I would prefer to introduce a new APB freq class handler to deal with
the differences in the AST2600. aspeed_scu_get_apb_freq() would become :

uint32_t aspeed_scu_get_apb_freq(AspeedSCUState *s)
{
return ASPEED_SCU_GET_CLASS(s)->get_apb(s);
}

The current aspeed_scu_get_apb_freq() would become the AST2400 and AST2500
handler and you would have to introduce a new one for the AST2600.


+hpll_reg = s->regs[AST2600_HPLL_PARAM];
+clk_sel_reg = s->regs[AST2600_CLK_SEL];
+} else {
+hpll_reg = s->regs[HPLL_PARAM];
+clk_sel_reg = s->regs[CLK_SEL];
+}
+
+hpll = asc->calc_hpll(s, hpll_reg);
  
-return hpll / (SCU_CLK_GET_PCLK_DIV(s->regs[CLK_SEL]) + 1)

+return hpll / (SCU_CLK_GET_PCLK_DIV(clk_sel_reg) + 1)
  / asc->apb_divider;>   }
  
@@ -357,7 +369,10 @@ static const MemoryRegionOps aspeed_ast2500_scu_ops = {
  
  static uint32_t aspeed_scu_get_clkin(AspeedSCUState *s)

  {
-if (s->hw_strap1 & SCU_HW_STRAP_CLK_25M_IN) {
+AspeedSCUClass *asc = ASPEED_SCU_GET_CLASS(s);
+
+if (s->hw_strap1 & SCU_HW_STRAP_CLK_25M_IN ||
+asc->calc_hpll == aspeed_2600_scu_calc_hpll) {


Indeed, the AST2600 CLKIN is always 25Mhz. Instead of testing ->calc_hpll,
I would introduce a class attribute, something like 'bool is_25Mhz'.

This change should be in a second patch though.

Thanks,

C.


  return 2500;
  } else if (s->hw_strap1 & SCU_HW_STRAP_CLK_48M_IN) {
  return 4800;
@@ -426,6 +441,26 @@ static uint32_t aspeed_2500_scu_calc_hpll(AspeedSCUState 
*s, uint32_t hpll_reg)
  return clkin * multiplier;
  }
  
+static uint32_t aspeed_2600_scu_calc_hpll(AspeedSCUState *s, uint32_t hpll_reg)

+{
+uint32_t multiplier = 1;
+uint32_t clkin = aspeed_scu_get_clkin(s);
+
+if (hpll_reg & SCU_AST2600_H_PLL_OFF) {
+return 0;
+}
+
+if (!(hpll_reg & SCU_H_PLL_BYPASS_EN)) {
+uint32_t p = (hpll_reg >> 19) & 0xf;
+uint32_t n = (hpll_reg >> 13) & 0x3f;
+uint32_t m = hpll_reg & 0x1fff;
+
+multiplier = ((m + 1) / (n + 1)) / (p + 1);
+}
+
+return clkin * multiplier;
+}
+
  static void aspeed_scu_reset(DeviceState *dev)
  {
  AspeedSCUState *s = ASPEED_SCU(dev);
@@ -716,7 +751,7 @@ static void aspeed_2600_scu_class_init(ObjectClass *klass, 
void *data)
  dc->desc = "ASPEED 2600 System Control Unit";
  dc->reset = aspeed_ast2600_scu_reset;
  asc->resets = ast2600_a3_resets;
-asc->calc_hpll = aspeed_2500_scu_calc_hpll; /* No change since AST2500 */
+asc->calc_hpll = aspeed_2600_scu_calc_hpll;
  asc->apb_divider = 4;
  asc->nr_regs = ASPEED_AST2600_SCU_NR_REGS;
  asc->ops = &aspeed_ast2600_scu_ops;
diff --git a/include/hw/misc/aspeed_scu.h b/include/hw/misc/aspeed_scu.h
index c14aff2bcb..91c500c5bc 100644
--- a/include/hw/misc/aspeed_scu.h
+++ b/include/hw/misc/aspeed_scu.h
@@ -316,4 +316,21 @@ uint32_t aspeed_scu_get_apb_freq(AspeedSCUState *s);
  SCU_HW_STRAP_VGA_SIZE_SET(VGA_16M_DRAM) |   \
  SCU_AST2500_HW_STRAP_RESERVED1)
  
+/* SCU200   H-PLL Parameter Register (for Aspeed AST2600 SOC)

+ *
+ *  28:26  H-PLL Parameters
+ *  25 Enable H-PLL reset
+ *  24 Enable H-PLL bypass mode
+ *  23 Turn off H-PLL
+ *  22:19  H-PLL Post Divider (P)
+ *  18:13   H-PLL Numerator (M)
+ *  12:0H-PLL Denumerator (N)
+ *
+ *  (Output frequency) = CLKIN(25MHz) * [(M+1) / (N+1)] / (P+1)
+ *
+ * The default frequency is 1200Mhz when CLKIN = 25MHz
+ */
+#define SCU_AST2600_H_PLL_BYPASS_EN(0x1

Re: [PATCH v3 0/3] Fix BRASL and BRCL with large negative offsets

[Thomas Huth]

On 14/03/2022 11.42, Ilya Leoshkevich wrote:

Hi,

I noticed that sometimes jumping backwards leads to crashes or hangs.
The problem is a missing cast.
Patches 1 and 2 fix the problem, patch 3 adds a test.


v1: https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg03356.html
v1 -> v2:
- Skip the test if mmap() fails (Richard).
- Replace test opcodes with inline asm (David). Since we now want to
   skip the test if the code cannot be mapped (e.g. on a 31-bit host),
   we shouldn't be asking the loader to map the code right away. So
   the mmap() approach stays.

v2: https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg03387.html
v2 -> v3:
- Use MAP_NORESERVE (Christian / David).
- Unhardcode 0x10006 (Richard).

Best regards,
Ilya

Ilya Leoshkevich (3):
   s390x/tcg: Fix BRASL with a large negative offset
   s390x/tcg: Fix BRCL with a large negative offset
   tests/tcg/s390x: Test BRASL and BRCL with large negative offsets

  target/s390x/tcg/translate.c   |  4 +-
  tests/tcg/s390x/Makefile.target|  1 +
  tests/tcg/s390x/branch-relative-long.c | 68 ++
  3 files changed, 71 insertions(+), 2 deletions(-)
  create mode 100644 tests/tcg/s390x/branch-relative-long.c


Thanks, queued to my s390x-next branch now:

https://gitlab.com/thuth/qemu/-/commits/s390x-next/

 Thomas

Re: [PATCH 0/3] tcg/s390x vector fixes

[Thomas Huth]

On 10/03/2022 21.27, Richard Henderson wrote:

These 3 issues were found by running risu on arm neon test cases.
In the meantime, Thomas encountered one of the same with the new
tests of vectorized sha512.


Thanks! If you don't mind (e.g. if you don't have any other TCG patches 
pending), I can take these through my s390x-next tree (I'm planning another 
pull request before the rc0 hard freeze tomorrow):


 https://gitlab.com/thuth/qemu/-/commits/s390x-next/

 Thomas

[PULL 05/12] ui/console: move console compatibility check to dcl_display_console()

[marcandre . lureau]

From: Marc-André Lureau 

The current checks are done at registration time only. However, if a DCL
has no specific console specified, it may be switched dynamically with
console_select() later on.

Let's move the checks when displaychangelistener_display_console() is
called, which includes registration time and remains fatal if the
specified console is incompatible.

Note: we may want to display the compatibility error to the DCL, this is
left for a future improvement.

Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/console.c | 19 +--
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/ui/console.c b/ui/console.c
index c9318552871b..d3ecbb215736 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -148,6 +148,8 @@ static DisplayState *get_alloc_displaystate(void);
 static void text_console_update_cursor_timer(void);
 static void text_console_update_cursor(void *opaque);
 static bool displaychangelistener_has_dmabuf(DisplayChangeListener *dcl);
+static bool console_compatible_with(QemuConsole *con,
+DisplayChangeListener *dcl, Error **errp);
 
 static void gui_update(void *opaque)
 {
@@ -1057,13 +1059,14 @@ static void console_putchar(QemuConsole *s, int ch)
 }
 
 static void displaychangelistener_display_console(DisplayChangeListener *dcl,
-  QemuConsole *con)
+  QemuConsole *con,
+  Error **errp)
 {
 static const char nodev[] =
 "This VM has no graphic display device.";
 static DisplaySurface *dummy;
 
-if (!con) {
+if (!con || !console_compatible_with(con, dcl, errp)) {
 if (!dcl->ops->dpy_gfx_switch) {
 return;
 }
@@ -1114,7 +1117,7 @@ void console_select(unsigned int index)
 if (dcl->con != NULL) {
 continue;
 }
-displaychangelistener_display_console(dcl, s);
+displaychangelistener_display_console(dcl, s, NULL);
 }
 }
 if (ds->have_text) {
@@ -1475,8 +1478,8 @@ static bool 
displaychangelistener_has_dmabuf(DisplayChangeListener *dcl)
 return false;
 }
 
-static bool dpy_compatible_with(QemuConsole *con,
-DisplayChangeListener *dcl, Error **errp)
+static bool console_compatible_with(QemuConsole *con,
+DisplayChangeListener *dcl, Error **errp)
 {
 int flags;
 
@@ -1522,10 +1525,6 @@ void 
register_displaychangelistener(DisplayChangeListener *dcl)
 
 assert(!dcl->ds);
 
-if (dcl->con) {
-dpy_compatible_with(dcl->con, dcl, &error_fatal);
-}
-
 trace_displaychangelistener_register(dcl, dcl->ops->dpy_name);
 dcl->ds = get_alloc_displaystate();
 QLIST_INSERT_HEAD(&dcl->ds->listeners, dcl, next);
@@ -1536,7 +1535,7 @@ void register_displaychangelistener(DisplayChangeListener 
*dcl)
 } else {
 con = active_console;
 }
-displaychangelistener_display_console(dcl, con);
+displaychangelistener_display_console(dcl, con, dcl->con ? &error_fatal : 
NULL);
 text_console_update_cursor(NULL);
 }
 
-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 00/12] dbus console fixes

[marcandre . lureau]

From: Marc-André Lureau 

The following changes since commit 1416688c53be6535be755b44c15fb2eb9defd20f:

  Merge remote-tracking branch 
'remotes/mcayland/tags/q800-updates-for-7.0-20220309' into staging (2022-03-10 
13:16:37 +)

are available in the Git repository at:

  g...@gitlab.com:marcandre.lureau/qemu.git tags/dbus-pull-request

for you to fetch changes up to c923967ca32600aa4e8ab817565ebf5415b4e195:

  ui/console: call gfx_switch() even if the current scanout is GL (2022-03-14 
15:16:29 +0400)


GL & D-Bus display related fixes

Hi,

Here are pending fixes related to D-Bus and GL, most of them reported thanks to
Akihiko Odaki.



Marc-André Lureau (12):
  ui/console: move check for compatible GL context
  ui/console: move dcl compatiblity check to a callback
  ui/console: egl-headless is compatible with non-gl listeners
  ui/dbus: associate the DBusDisplayConsole listener with the given
console
  ui/console: move console compatibility check to dcl_display_console()
  ui/shader: fix potential leak of shader on error
  ui/shader: free associated programs
  ui/console: add a dpy_gfx_switch callback helper
  ui/console: optionally update after gfx switch
  ui/dbus: fix texture sharing
  ui/dbus: do not send 2d scanout until gfx_update
  ui/console: call gfx_switch() even if the current scanout is GL

 include/ui/console.h |  19 +---
 ui/dbus.h|   3 ++
 ui/console.c | 102 ++-
 ui/dbus-console.c|  27 ++--
 ui/dbus-listener.c   |  48 +---
 ui/dbus.c|  35 ++-
 ui/egl-headless.c|  17 +++-
 ui/gtk.c |  18 +++-
 ui/sdl2.c|   9 +++-
 ui/shader.c  |   9 +++-
 ui/spice-display.c   |   9 +++-
 11 files changed, 200 insertions(+), 96 deletions(-)

-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 09/12] ui/console: optionally update after gfx switch

[marcandre . lureau]

From: Marc-André Lureau 

When switching to the dummy surface, we should also call gfx_update.
But when using GL, we shouldn't call it.

By making it an argument to displaychangelistener_gfx_switch(), it will
be explicit, and cannot be forgotten that easily.

Fixes: commit ebced091 ("console: save current scanout details")
Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/console.c | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/ui/console.c b/ui/console.c
index 102fcf0a5068..06ba82db61c9 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1059,11 +1059,18 @@ static void console_putchar(QemuConsole *s, int ch)
 }
 
 static void displaychangelistener_gfx_switch(DisplayChangeListener *dcl,
- struct DisplaySurface 
*new_surface)
+ struct DisplaySurface 
*new_surface,
+ bool update)
 {
 if (dcl->ops->dpy_gfx_switch) {
 dcl->ops->dpy_gfx_switch(dcl, new_surface);
 }
+
+if (update && dcl->ops->dpy_gfx_update) {
+dcl->ops->dpy_gfx_update(dcl, 0, 0,
+ surface_width(new_surface),
+ surface_height(new_surface));
+}
 }
 
 
@@ -1079,7 +1086,7 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
 if (!dummy) {
 dummy = qemu_create_placeholder_surface(640, 480, nodev);
 }
-displaychangelistener_gfx_switch(dcl, dummy);
+displaychangelistener_gfx_switch(dcl, dummy, TRUE);
 return;
 }
 
@@ -1098,12 +1105,8 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
  con->scanout.texture.width,
  con->scanout.texture.height);
 } else if (con->scanout.kind == SCANOUT_SURFACE) {
-displaychangelistener_gfx_switch(dcl, con->surface);
+displaychangelistener_gfx_switch(dcl, con->surface, TRUE);
 }
-
-dcl->ops->dpy_gfx_update(dcl, 0, 0,
- qemu_console_get_width(con, 0),
- qemu_console_get_height(con, 0));
 }
 
 void console_select(unsigned int index)
@@ -1682,7 +1685,7 @@ void dpy_gfx_replace_surface(QemuConsole *con,
 if (con != (dcl->con ? dcl->con : active_console)) {
 continue;
 }
-displaychangelistener_gfx_switch(dcl, surface);
+displaychangelistener_gfx_switch(dcl, surface, FALSE);
 }
 qemu_free_displaysurface(old_surface);
 }
-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 02/12] ui/console: move dcl compatiblity check to a callback

[marcandre . lureau]

From: Marc-André Lureau 

As expected from the "compatible_dcl" comment, a simple comparison of
ops isn't enough. The following patch will fix a regression introduced
by this limited check by extending the compatibility callback for
egl-headless.

For now, this patch simply replaces the the "compatible_dcl" ops pointer
with a "dpy_gl_ctx_is_compatible_ctx" callback.

Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 include/ui/console.h |  9 ++---
 ui/console.c |  3 ++-
 ui/dbus.c|  9 -
 ui/egl-headless.c|  9 -
 ui/gtk.c | 18 --
 ui/sdl2.c|  9 -
 ui/spice-display.c   |  9 -
 7 files changed, 52 insertions(+), 14 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index f590819880b5..18a10c0b7db0 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -282,13 +282,8 @@ struct DisplayChangeListener {
 };
 
 typedef struct DisplayGLCtxOps {
-/*
- * We only check if the GLCtx is compatible with a DCL via ops. A natural
- * evolution of this would be a callback to check some runtime requirements
- * and allow various DCL kinds.
- */
-const DisplayChangeListenerOps *compatible_dcl;
-
+bool (*dpy_gl_ctx_is_compatible_dcl)(DisplayGLCtx *dgc,
+ DisplayChangeListener *dcl);
 QEMUGLContext (*dpy_gl_ctx_create)(DisplayGLCtx *dgc,
QEMUGLParams *params);
 void (*dpy_gl_ctx_destroy)(DisplayGLCtx *dgc,
diff --git a/ui/console.c b/ui/console.c
index 57e431d9e609..c9318552871b 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1482,7 +1482,8 @@ static bool dpy_compatible_with(QemuConsole *con,
 
 flags = con->hw_ops->get_flags ? con->hw_ops->get_flags(con->hw) : 0;
 
-if (console_has_gl(con) && con->gl->ops->compatible_dcl != dcl->ops) {
+if (console_has_gl(con) &&
+!con->gl->ops->dpy_gl_ctx_is_compatible_dcl(con->gl, dcl)) {
 error_setg(errp, "Display %s is incompatible with the GL context",
dcl->ops->dpy_name);
 return false;
diff --git a/ui/dbus.c b/ui/dbus.c
index 0074424c1fed..f00a44421cf7 100644
--- a/ui/dbus.c
+++ b/ui/dbus.c
@@ -48,8 +48,15 @@ static QEMUGLContext dbus_create_context(DisplayGLCtx *dgc,
 return qemu_egl_create_context(dgc, params);
 }
 
+static bool
+dbus_is_compatible_dcl(DisplayGLCtx *dgc,
+   DisplayChangeListener *dcl)
+{
+return dcl->ops == &dbus_gl_dcl_ops;
+}
+
 static const DisplayGLCtxOps dbus_gl_ops = {
-.compatible_dcl  = &dbus_gl_dcl_ops,
+.dpy_gl_ctx_is_compatible_dcl = dbus_is_compatible_dcl,
 .dpy_gl_ctx_create   = dbus_create_context,
 .dpy_gl_ctx_destroy  = qemu_egl_destroy_context,
 .dpy_gl_ctx_make_current = qemu_egl_make_context_current,
diff --git a/ui/egl-headless.c b/ui/egl-headless.c
index 94082a9da951..9aff115280bc 100644
--- a/ui/egl-headless.c
+++ b/ui/egl-headless.c
@@ -166,8 +166,15 @@ static const DisplayChangeListenerOps egl_ops = {
 .dpy_gl_update   = egl_scanout_flush,
 };
 
+static bool
+egl_is_compatible_dcl(DisplayGLCtx *dgc,
+  DisplayChangeListener *dcl)
+{
+return dcl->ops == &egl_ops;
+}
+
 static const DisplayGLCtxOps eglctx_ops = {
-.compatible_dcl  = &egl_ops,
+.dpy_gl_ctx_is_compatible_dcl = egl_is_compatible_dcl,
 .dpy_gl_ctx_create   = egl_create_context,
 .dpy_gl_ctx_destroy  = qemu_egl_destroy_context,
 .dpy_gl_ctx_make_current = qemu_egl_make_context_current,
diff --git a/ui/gtk.c b/ui/gtk.c
index a8567b9ddc8f..1b24a67d7964 100644
--- a/ui/gtk.c
+++ b/ui/gtk.c
@@ -614,8 +614,15 @@ static const DisplayChangeListenerOps dcl_gl_area_ops = {
 .dpy_has_dmabuf  = gd_has_dmabuf,
 };
 
+static bool
+gd_gl_area_is_compatible_dcl(DisplayGLCtx *dgc,
+ DisplayChangeListener *dcl)
+{
+return dcl->ops == &dcl_gl_area_ops;
+}
+
 static const DisplayGLCtxOps gl_area_ctx_ops = {
-.compatible_dcl  = &dcl_gl_area_ops,
+.dpy_gl_ctx_is_compatible_dcl = gd_gl_area_is_compatible_dcl,
 .dpy_gl_ctx_create   = gd_gl_area_create_context,
 .dpy_gl_ctx_destroy  = gd_gl_area_destroy_context,
 .dpy_gl_ctx_make_current = gd_gl_area_make_current,
@@ -641,8 +648,15 @@ static const DisplayChangeListenerOps dcl_egl_ops = {
 .dpy_has_dmabuf  = gd_has_dmabuf,
 };
 
+static bool
+gd_egl_is_compatible_dcl(DisplayGLCtx *dgc,
+ DisplayChangeListener *dcl)
+{
+return dcl->ops == &dcl_egl_ops;
+}
+
 static const DisplayGLCtxOps egl_ctx_ops = {
-.compatible_dcl  = &dcl_egl_ops,
+.dpy_gl_ctx_is_compatible_dcl = gd_egl_is_compatible_dcl,
 .dpy_gl_ctx_create   = gd_egl_create_context,
 .dpy_gl_ctx_destroy  = qemu_egl_destroy_context,
 .dpy_gl_ctx_make_current = gd_egl_make_current,
diff --git a/ui/sdl2.c b/u

[PULL 01/12] ui/console: move check for compatible GL context

[marcandre . lureau]

From: Marc-André Lureau 

Move GL context compatibility check in dpy_compatible_with(), and use
recommended error reporting.

Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/console.c | 21 ++---
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/ui/console.c b/ui/console.c
index 365a2c14b809..57e431d9e609 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1482,6 +1482,12 @@ static bool dpy_compatible_with(QemuConsole *con,
 
 flags = con->hw_ops->get_flags ? con->hw_ops->get_flags(con->hw) : 0;
 
+if (console_has_gl(con) && con->gl->ops->compatible_dcl != dcl->ops) {
+error_setg(errp, "Display %s is incompatible with the GL context",
+   dcl->ops->dpy_name);
+return false;
+}
+
 if (flags & GRAPHIC_FLAGS_GL &&
 !console_has_gl(con)) {
 error_setg(errp, "The console requires a GL context.");
@@ -1509,27 +1515,12 @@ void qemu_console_set_display_gl_ctx(QemuConsole *con, 
DisplayGLCtx *gl)
 con->gl = gl;
 }
 
-static bool dpy_gl_compatible_with(QemuConsole *con, DisplayChangeListener 
*dcl)
-{
-if (!con->gl) {
-return true;
-}
-
-return con->gl->ops->compatible_dcl == dcl->ops;
-}
-
 void register_displaychangelistener(DisplayChangeListener *dcl)
 {
 QemuConsole *con;
 
 assert(!dcl->ds);
 
-if (dcl->con && !dpy_gl_compatible_with(dcl->con, dcl)) {
-error_report("Display %s is incompatible with the GL context",
- dcl->ops->dpy_name);
-exit(1);
-}
-
 if (dcl->con) {
 dpy_compatible_with(dcl->con, dcl, &error_fatal);
 }
-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 10/12] ui/dbus: fix texture sharing

[marcandre . lureau]

From: Marc-André Lureau 

The DBus listener naively create, update and destroy textures without
taking into account other listeners. The texture were shared, but
texture update was unnecessarily duplicated.

Teach DisplayGLCtx to do optionally shared texture handling. This is
only implemented for DBus display at this point, however the same
infrastructure could potentially be used for other future combinations.

Reported-by: Akihiko Odaki 
Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 include/ui/console.h | 10 ++
 ui/console.c | 26 ++
 ui/dbus-listener.c   | 11 ---
 ui/dbus.c| 26 ++
 4 files changed, 62 insertions(+), 11 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 18a10c0b7db0..0f84861933e1 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -290,10 +290,20 @@ typedef struct DisplayGLCtxOps {
QEMUGLContext ctx);
 int (*dpy_gl_ctx_make_current)(DisplayGLCtx *dgc,
QEMUGLContext ctx);
+void (*dpy_gl_ctx_create_texture)(DisplayGLCtx *dgc,
+  DisplaySurface *surface);
+void (*dpy_gl_ctx_destroy_texture)(DisplayGLCtx *dgc,
+  DisplaySurface *surface);
+void (*dpy_gl_ctx_update_texture)(DisplayGLCtx *dgc,
+  DisplaySurface *surface,
+  int x, int y, int w, int h);
 } DisplayGLCtxOps;
 
 struct DisplayGLCtx {
 const DisplayGLCtxOps *ops;
+#ifdef CONFIG_OPENGL
+QemuGLShader *gls; /* optional shared shader */
+#endif
 };
 
 DisplayState *init_displaystate(void);
diff --git a/ui/console.c b/ui/console.c
index 06ba82db61c9..5bfecea4549e 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1073,6 +1073,27 @@ static void 
displaychangelistener_gfx_switch(DisplayChangeListener *dcl,
 }
 }
 
+static void dpy_gfx_create_texture(QemuConsole *con, DisplaySurface *surface)
+{
+if (con->gl && con->gl->ops->dpy_gl_ctx_create_texture) {
+con->gl->ops->dpy_gl_ctx_create_texture(con->gl, surface);
+}
+}
+
+static void dpy_gfx_destroy_texture(QemuConsole *con, DisplaySurface *surface)
+{
+if (con->gl && con->gl->ops->dpy_gl_ctx_destroy_texture) {
+con->gl->ops->dpy_gl_ctx_destroy_texture(con->gl, surface);
+}
+}
+
+static void dpy_gfx_update_texture(QemuConsole *con, DisplaySurface *surface,
+   int x, int y, int w, int h)
+{
+if (con->gl && con->gl->ops->dpy_gl_ctx_update_texture) {
+con->gl->ops->dpy_gl_ctx_update_texture(con->gl, surface, x, y, w, h);
+}
+}
 
 static void displaychangelistener_display_console(DisplayChangeListener *dcl,
   QemuConsole *con,
@@ -1085,6 +1106,7 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
 if (!con || !console_compatible_with(con, dcl, errp)) {
 if (!dummy) {
 dummy = qemu_create_placeholder_surface(640, 480, nodev);
+dpy_gfx_create_texture(con, dummy);
 }
 displaychangelistener_gfx_switch(dcl, dummy, TRUE);
 return;
@@ -1105,6 +1127,7 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
  con->scanout.texture.width,
  con->scanout.texture.height);
 } else if (con->scanout.kind == SCANOUT_SURFACE) {
+dpy_gfx_create_texture(con, con->surface);
 displaychangelistener_gfx_switch(dcl, con->surface, TRUE);
 }
 }
@@ -1637,6 +1660,7 @@ void dpy_gfx_update(QemuConsole *con, int x, int y, int 
w, int h)
 if (!qemu_console_is_visible(con)) {
 return;
 }
+dpy_gfx_update_texture(con, con->surface, x, y, w, h);
 QLIST_FOREACH(dcl, &s->listeners, next) {
 if (con != (dcl->con ? dcl->con : active_console)) {
 continue;
@@ -1681,12 +1705,14 @@ void dpy_gfx_replace_surface(QemuConsole *con,
 
 con->scanout.kind = SCANOUT_SURFACE;
 con->surface = surface;
+dpy_gfx_create_texture(con, surface);
 QLIST_FOREACH(dcl, &s->listeners, next) {
 if (con != (dcl->con ? dcl->con : active_console)) {
 continue;
 }
 displaychangelistener_gfx_switch(dcl, surface, FALSE);
 }
+dpy_gfx_destroy_texture(con, old_surface);
 qemu_free_displaysurface(old_surface);
 }
 
diff --git a/ui/dbus-listener.c b/ui/dbus-listener.c
index 81c119b13a2c..a287edd2fc15 100644
--- a/ui/dbus-listener.c
+++ b/ui/dbus-listener.c
@@ -42,7 +42,6 @@ struct _DBusDisplayListener {
 
 DisplayChangeListener dcl;
 DisplaySurface *ds;
-QemuGLShader *gls;
 int gl_updates;
 };
 
@@ -240,10 +239,6 @@ static void dbus_gl_gfx_update(DisplayChangeListener *dcl,
 {
 DBusDisplayListener *ddl = container_of(dcl, DBusDispl

[PULL 06/12] ui/shader: fix potential leak of shader on error

[marcandre . lureau]

From: Marc-André Lureau 

Value of 0 for program and shaders are silently ignored and indicate error.

Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/shader.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/ui/shader.c b/ui/shader.c
index e8b8d321b7c7..4c80fc831f68 100644
--- a/ui/shader.c
+++ b/ui/shader.c
@@ -130,15 +130,17 @@ static GLuint qemu_gl_create_link_program(GLuint vert, 
GLuint frag)
 static GLuint qemu_gl_create_compile_link_program(const GLchar *vert_src,
   const GLchar *frag_src)
 {
-GLuint vert_shader, frag_shader, program;
+GLuint vert_shader, frag_shader, program = 0;
 
 vert_shader = qemu_gl_create_compile_shader(GL_VERTEX_SHADER, vert_src);
 frag_shader = qemu_gl_create_compile_shader(GL_FRAGMENT_SHADER, frag_src);
 if (!vert_shader || !frag_shader) {
-return 0;
+goto end;
 }
 
 program = qemu_gl_create_link_program(vert_shader, frag_shader);
+
+end:
 glDeleteShader(vert_shader);
 glDeleteShader(frag_shader);
 
-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 03/12] ui/console: egl-headless is compatible with non-gl listeners

[marcandre . lureau]

From: Marc-André Lureau 

Fix a regression introduced by commit 5e79d516e ("ui: split the GL
context in a different object").

Reported-by: Akihiko Odaki 
Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/egl-headless.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/ui/egl-headless.c b/ui/egl-headless.c
index 9aff115280bc..7a30fd977765 100644
--- a/ui/egl-headless.c
+++ b/ui/egl-headless.c
@@ -170,6 +170,14 @@ static bool
 egl_is_compatible_dcl(DisplayGLCtx *dgc,
   DisplayChangeListener *dcl)
 {
+if (!dcl->ops->dpy_gl_update) {
+/*
+ * egl-headless is compatible with all 2d listeners, as it blits the GL
+ * updates on the 2d console surface.
+ */
+return true;
+}
+
 return dcl->ops == &egl_ops;
 }
 
-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 11/12] ui/dbus: do not send 2d scanout until gfx_update

[marcandre . lureau]

From: Marc-André Lureau 

gfx_switch() is called to set the new_surface, not necessarily to
display it. It should be displayed after gfx_update(). Send the whole
scanout only in this case.

Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/dbus-listener.c | 37 -
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/ui/dbus-listener.c b/ui/dbus-listener.c
index a287edd2fc15..f9fc8eda519a 100644
--- a/ui/dbus-listener.c
+++ b/ui/dbus-listener.c
@@ -255,6 +255,26 @@ static void dbus_gfx_update(DisplayChangeListener *dcl,
 
 trace_dbus_update(x, y, w, h);
 
+if (x == 0 && y == 0 && w == surface_width(ddl->ds) && h == 
surface_height(ddl->ds)) {
+v_data = g_variant_new_from_data(
+G_VARIANT_TYPE("ay"),
+surface_data(ddl->ds),
+surface_stride(ddl->ds) * surface_height(ddl->ds),
+TRUE,
+(GDestroyNotify)pixman_image_unref,
+pixman_image_ref(ddl->ds->image));
+qemu_dbus_display1_listener_call_scanout(
+ddl->proxy,
+surface_width(ddl->ds),
+surface_height(ddl->ds),
+surface_stride(ddl->ds),
+surface_format(ddl->ds),
+v_data,
+G_DBUS_CALL_FLAGS_NONE,
+DBUS_DEFAULT_TIMEOUT, NULL, NULL, NULL);
+return;
+}
+
 /* make a copy, since gvariant only handles linear data */
 img = pixman_image_create_bits(surface_format(ddl->ds),
w, h, NULL, stride);
@@ -295,29 +315,12 @@ static void dbus_gfx_switch(DisplayChangeListener *dcl,
 struct DisplaySurface *new_surface)
 {
 DBusDisplayListener *ddl = container_of(dcl, DBusDisplayListener, dcl);
-GVariant *v_data = NULL;
 
 ddl->ds = new_surface;
 if (!ddl->ds) {
 /* why not call disable instead? */
 return;
 }
-
-v_data = g_variant_new_from_data(
-G_VARIANT_TYPE("ay"),
-surface_data(ddl->ds),
-surface_stride(ddl->ds) * surface_height(ddl->ds),
-TRUE,
-(GDestroyNotify)pixman_image_unref,
-pixman_image_ref(ddl->ds->image));
-qemu_dbus_display1_listener_call_scanout(ddl->proxy,
-surface_width(ddl->ds),
-surface_height(ddl->ds),
-surface_stride(ddl->ds),
-surface_format(ddl->ds),
-v_data,
-G_DBUS_CALL_FLAGS_NONE,
-DBUS_DEFAULT_TIMEOUT, NULL, NULL, NULL);
 }
 
 static void dbus_mouse_set(DisplayChangeListener *dcl,
-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 04/12] ui/dbus: associate the DBusDisplayConsole listener with the given console

[marcandre . lureau]

From: Marc-André Lureau 

DBusDisplayConsole is specific to a given QemuConsole.

Fixes: commit 142ca628 ("ui: add a D-Bus display backend")
Reported-by: Akihiko Odaki 
Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/dbus.h |  3 +++
 ui/dbus-console.c | 27 +--
 ui/dbus.c |  2 +-
 3 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/ui/dbus.h b/ui/dbus.h
index 64c77cab4441..5f5c1f759c9b 100644
--- a/ui/dbus.h
+++ b/ui/dbus.h
@@ -79,6 +79,9 @@ dbus_display_console_new(DBusDisplay *display, QemuConsole 
*con);
 int
 dbus_display_console_get_index(DBusDisplayConsole *ddc);
 
+
+extern const DisplayChangeListenerOps dbus_console_dcl_ops;
+
 #define DBUS_DISPLAY_TYPE_LISTENER dbus_display_listener_get_type()
 G_DECLARE_FINAL_TYPE(DBusDisplayListener,
  dbus_display_listener,
diff --git a/ui/dbus-console.c b/ui/dbus-console.c
index e062f721d761..898a4ac8a5ba 100644
--- a/ui/dbus-console.c
+++ b/ui/dbus-console.c
@@ -36,7 +36,6 @@ struct _DBusDisplayConsole {
 DisplayChangeListener dcl;
 
 DBusDisplay *display;
-QemuConsole *con;
 GHashTable *listeners;
 QemuDBusDisplay1Console *iface;
 
@@ -118,7 +117,7 @@ dbus_gl_scanout_update(DisplayChangeListener *dcl,
 {
 }
 
-static const DisplayChangeListenerOps dbus_console_dcl_ops = {
+const DisplayChangeListenerOps dbus_console_dcl_ops = {
 .dpy_name= "dbus-console",
 .dpy_gfx_switch  = dbus_gfx_switch,
 .dpy_gfx_update  = dbus_gfx_update,
@@ -191,7 +190,7 @@ dbus_console_set_ui_info(DBusDisplayConsole *ddc,
 .height = arg_height,
 };
 
-if (!dpy_ui_info_supported(ddc->con)) {
+if (!dpy_ui_info_supported(ddc->dcl.con)) {
 g_dbus_method_invocation_return_error(invocation,
   DBUS_DISPLAY_ERROR,
   DBUS_DISPLAY_ERROR_UNSUPPORTED,
@@ -199,7 +198,7 @@ dbus_console_set_ui_info(DBusDisplayConsole *ddc,
 return DBUS_METHOD_INVOCATION_HANDLED;
 }
 
-dpy_set_ui_info(ddc->con, &info, false);
+dpy_set_ui_info(ddc->dcl.con, &info, false);
 qemu_dbus_display1_console_complete_set_uiinfo(ddc->iface, invocation);
 return DBUS_METHOD_INVOCATION_HANDLED;
 }
@@ -335,8 +334,8 @@ dbus_mouse_rel_motion(DBusDisplayConsole *ddc,
 return DBUS_METHOD_INVOCATION_HANDLED;
 }
 
-qemu_input_queue_rel(ddc->con, INPUT_AXIS_X, dx);
-qemu_input_queue_rel(ddc->con, INPUT_AXIS_Y, dy);
+qemu_input_queue_rel(ddc->dcl.con, INPUT_AXIS_X, dx);
+qemu_input_queue_rel(ddc->dcl.con, INPUT_AXIS_Y, dy);
 qemu_input_event_sync();
 
 qemu_dbus_display1_mouse_complete_rel_motion(ddc->iface_mouse,
@@ -362,8 +361,8 @@ dbus_mouse_set_pos(DBusDisplayConsole *ddc,
 return DBUS_METHOD_INVOCATION_HANDLED;
 }
 
-width = qemu_console_get_width(ddc->con, 0);
-height = qemu_console_get_height(ddc->con, 0);
+width = qemu_console_get_width(ddc->dcl.con, 0);
+height = qemu_console_get_height(ddc->dcl.con, 0);
 if (x >= width || y >= height) {
 g_dbus_method_invocation_return_error(
 invocation, DBUS_DISPLAY_ERROR,
@@ -371,8 +370,8 @@ dbus_mouse_set_pos(DBusDisplayConsole *ddc,
 "Invalid mouse position");
 return DBUS_METHOD_INVOCATION_HANDLED;
 }
-qemu_input_queue_abs(ddc->con, INPUT_AXIS_X, x, 0, width);
-qemu_input_queue_abs(ddc->con, INPUT_AXIS_Y, y, 0, height);
+qemu_input_queue_abs(ddc->dcl.con, INPUT_AXIS_X, x, 0, width);
+qemu_input_queue_abs(ddc->dcl.con, INPUT_AXIS_Y, y, 0, height);
 qemu_input_event_sync();
 
 qemu_dbus_display1_mouse_complete_set_abs_position(ddc->iface_mouse,
@@ -388,7 +387,7 @@ dbus_mouse_press(DBusDisplayConsole *ddc,
 {
 trace_dbus_mouse_press(button);
 
-qemu_input_queue_btn(ddc->con, button, true);
+qemu_input_queue_btn(ddc->dcl.con, button, true);
 qemu_input_event_sync();
 
 qemu_dbus_display1_mouse_complete_press(ddc->iface_mouse, invocation);
@@ -403,7 +402,7 @@ dbus_mouse_release(DBusDisplayConsole *ddc,
 {
 trace_dbus_mouse_release(button);
 
-qemu_input_queue_btn(ddc->con, button, false);
+qemu_input_queue_btn(ddc->dcl.con, button, false);
 qemu_input_event_sync();
 
 qemu_dbus_display1_mouse_complete_release(ddc->iface_mouse, invocation);
@@ -424,7 +423,7 @@ dbus_mouse_mode_change(Notifier *notify, void *data)
 
 int dbus_display_console_get_index(DBusDisplayConsole *ddc)
 {
-return qemu_console_get_index(ddc->con);
+return qemu_console_get_index(ddc->dcl.con);
 }
 
 DBusDisplayConsole *
@@ -446,7 +445,7 @@ dbus_display_console_new(DBusDisplay *display, QemuConsole 
*con)
 "g-object-path", path,
 NULL);
 ddc->display = display;
-ddc->con = con;
+ddc->dcl.con = con;
 /* handle errors, and skip non graphics? */
 qemu_console_fill_device_address(
  

[PULL 12/12] ui/console: call gfx_switch() even if the current scanout is GL

[marcandre . lureau]

From: Marc-André Lureau 

egl-headless depends on the backing surface to be set before texture are
set and updated. Display it (update=true) iff the current scanout kind
is SURFACE.

Reported-by: Akihiko Odaki 
Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/console.c | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/ui/console.c b/ui/console.c
index 5bfecea4549e..16a0b0909ba2 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1112,6 +1112,10 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
 return;
 }
 
+dpy_gfx_create_texture(con, con->surface);
+displaychangelistener_gfx_switch(dcl, con->surface,
+ con->scanout.kind == SCANOUT_SURFACE);
+
 if (con->scanout.kind == SCANOUT_DMABUF &&
 displaychangelistener_has_dmabuf(dcl)) {
 dcl->ops->dpy_gl_scanout_dmabuf(dcl, con->scanout.dmabuf);
@@ -1126,9 +1130,6 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
  con->scanout.texture.y,
  con->scanout.texture.width,
  con->scanout.texture.height);
-} else if (con->scanout.kind == SCANOUT_SURFACE) {
-dpy_gfx_create_texture(con, con->surface);
-displaychangelistener_gfx_switch(dcl, con->surface, TRUE);
 }
 }
 
-- 
2.35.1.273.ge6ebfd0e8cbb

[PULL 08/12] ui/console: add a dpy_gfx_switch callback helper

[marcandre . lureau]

From: Marc-André Lureau 

Slight code improvement.

Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/console.c | 23 +--
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/ui/console.c b/ui/console.c
index d3ecbb215736..102fcf0a5068 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1058,6 +1058,15 @@ static void console_putchar(QemuConsole *s, int ch)
 }
 }
 
+static void displaychangelistener_gfx_switch(DisplayChangeListener *dcl,
+ struct DisplaySurface 
*new_surface)
+{
+if (dcl->ops->dpy_gfx_switch) {
+dcl->ops->dpy_gfx_switch(dcl, new_surface);
+}
+}
+
+
 static void displaychangelistener_display_console(DisplayChangeListener *dcl,
   QemuConsole *con,
   Error **errp)
@@ -1067,13 +1076,10 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
 static DisplaySurface *dummy;
 
 if (!con || !console_compatible_with(con, dcl, errp)) {
-if (!dcl->ops->dpy_gfx_switch) {
-return;
-}
 if (!dummy) {
 dummy = qemu_create_placeholder_surface(640, 480, nodev);
 }
-dcl->ops->dpy_gfx_switch(dcl, dummy);
+displaychangelistener_gfx_switch(dcl, dummy);
 return;
 }
 
@@ -1091,9 +1097,8 @@ static void 
displaychangelistener_display_console(DisplayChangeListener *dcl,
  con->scanout.texture.y,
  con->scanout.texture.width,
  con->scanout.texture.height);
-} else if (con->scanout.kind == SCANOUT_SURFACE &&
-   dcl->ops->dpy_gfx_switch) {
-dcl->ops->dpy_gfx_switch(dcl, con->surface);
+} else if (con->scanout.kind == SCANOUT_SURFACE) {
+displaychangelistener_gfx_switch(dcl, con->surface);
 }
 
 dcl->ops->dpy_gfx_update(dcl, 0, 0,
@@ -1677,9 +1682,7 @@ void dpy_gfx_replace_surface(QemuConsole *con,
 if (con != (dcl->con ? dcl->con : active_console)) {
 continue;
 }
-if (dcl->ops->dpy_gfx_switch) {
-dcl->ops->dpy_gfx_switch(dcl, surface);
-}
+displaychangelistener_gfx_switch(dcl, surface);
 }
 qemu_free_displaysurface(old_surface);
 }
-- 
2.35.1.273.ge6ebfd0e8cbb

Re: [PATCH] x86: q35: require split irqchip for large CPU count

[David Woodhouse]

On Mon, 2022-03-14 at 11:35 +0100, Igor Mammedov wrote:
> On Fri, 11 Mar 2022 14:58:41 +
> David Woodhouse <
> dw...@infradead.org
> > wrote:
> 
> > On Fri, 2022-03-11 at 09:39 -0500, Igor Mammedov wrote:
> > > if VM is started with:
> > > 
> > >-enable-kvm -smp 256
> > > 
> > > without specifying 'split' irqchip, VM might eventually boot
> > > but no more than 255 CPUs will be operational and following
> > > error messages in guest could be observed:
> > >...
> > >smpboot: native_cpu_up: bad cpu 256
> > >...
> > > It's a regression introduced by [1], which removed dependency
> > > on intremap=on that were implicitly requiring 'split' irqchip
> > > and forgot to check for 'split' irqchip.
> > > Instead of letting VM boot a broken VM, error out and tell
> > > user how to fix CLI.  
> > 
> > Hm, wasn't that already fixed in the patches I posted in December?
> 
> It might be, could you point to the commit/series that fixed it.

https://lore.kernel.org/all/20211209220840.14889-1-dw...@infradead.org/
is the patch I was thinking of, but although that moves the check to a
more useful place and fixes the X2APIC check, it *doesn't* include the
fix you're making; it's still using kvm_irqchip_in_kernel().

I can change that and repost the series, which is still sitting (with
fixed Reviewed-By/Acked-By attributions that I screwed up last time) in
https://git.infradead.org/users/dwmw2/qemu.git

> Regardless of that, fixing it in recent kernels doesn't help
> as still supported kernels are still affected by it.
> 
> If there is a way to detect that fix, I can add to q35 a compat
> property and an extra logic to enable kernel-irqchip if fix is present.
> Otherwise the fix does not exist until minimum supported kernel
> version reaches version where it was fixed.

Hm, I'm not sure I follow here. Do you mean recent versions of *qemu*
when you say 'kernels'? 

I'm not even sure I agree with the observation that qemu should error
out here. The guest boots fine and the guest can even *use* all the
CPUs. IPIs etc. will all work fine. The only thing that doesn't work is
delivering *external* interrupts to CPUs above 254.

Ultimately, this is the *guest's* problem. Some operating systems can
cope; some can't.

The fact that *Linux* has a fundamental assumption that *all* CPUs can
receive all interrupts and that affinity can't be limited in hardware,
is a Linux problem. I tried to fix it once but it was distinctly non-
trivial and eventually I gave up and took a different approach.
https://lore.kernel.org/linux-iommu/87lfgj59mp@nanos.tec.linutronix.de/T/

But even if we 'fix' the check as you suggest to bail out and refuse to
boot a certain configuration because Linux guest wouldn't be able to
fully utilize it... Even if we boot with the split IRQ chip and the 15-
bit MSI enlightenment, we're still in the same position. Some guests
will be able to use it; some won't.

In fact, there are operating systems that don't even know about X2APIC.

Why should qemu refuse to even start up?


smime.p7s
Description: S/MIME cryptographic signature

[PATCH v2 10/10] tests/qemu-iotests/030: test_stream_parallel should use auto_finalize=False

[Emanuele Giuseppe Esposito]

First, use run_job() instead of the current logic to run the
stream job. Then, use auto_finalize=False to be sure that
the job is not automatically deleted once it is done.

In this way, if the job finishes before we want, it is not
finalized yet so the other commands can still execute
without failing. run_job() will then take care of calling
job-finalize.

Signed-off-by: Emanuele Giuseppe Esposito 
Suggested-by: Kevin Wolf 
---
 tests/qemu-iotests/030 | 12 +++-
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
index 567bf1da67..951c28878c 100755
--- a/tests/qemu-iotests/030
+++ b/tests/qemu-iotests/030
@@ -248,7 +248,7 @@ class TestParallelOps(iotests.QMPTestCase):
 pending_jobs.append(job_id)
 result = self.vm.qmp('block-stream', device=node_name,
  job_id=job_id, bottom=f'node{i-1}',
- speed=1024)
+ speed=1024, auto_finalize=False)
 self.assert_qmp(result, 'return', {})
 
 # Do this in reverse: After unthrottling them, some jobs may finish
@@ -264,14 +264,8 @@ class TestParallelOps(iotests.QMPTestCase):
 result = self.vm.qmp('block-job-set-speed', device=job, speed=0)
 self.assert_qmp(result, 'return', {})
 
-# Wait for all jobs to be finished.
-while len(pending_jobs) > 0:
-for event in self.vm.get_qmp_events(wait=True):
-if event['event'] == 'BLOCK_JOB_COMPLETED':
-job_id = self.dictpath(event, 'data/device')
-self.assertTrue(job_id in pending_jobs)
-self.assert_qmp_absent(event, 'data/error')
-pending_jobs.remove(job_id)
+for job in pending_jobs:
+self.vm.run_job(job=job, auto_finalize=False)
 
 self.assert_no_active_block_jobs()
 self.vm.shutdown()
-- 
2.31.1

[PATCH v2 09/10] child_job_drained_poll: override polling condition only when in home thread

[Emanuele Giuseppe Esposito]

drv->drained_poll() is only implemented in mirror, and allows
it to drain from within the coroutine. The mirror implementation uses
in_drain flag to recognize when it is draining from coroutine,
and consequently avoid deadlocking (wait the poll condition in
child_job_drained_poll to wait for itself).

The problem is that this flag is dangerous, because it breaks
bdrv_drained_begin() invariants: once drained_begin ends, all
jobs, in_flight requests, and anything running in the iothread
are blocked.

This can be broken in such way:
iothread(mirror): s->in_drain = true; // mirror.c:1112
main loop: bdrv_drained_begin(mirror_bs);
/*
 * drained_begin wait for bdrv_drain_poll_top_level() condition,
 * that translates in child_job_drained_poll() for jobs, but
 * mirror implements drv->drained_poll() so it returns
 * !!in_flight_requests, which his 0 (assertion in mirror.c:1105).
 */
main loop: thinks iothread is stopped and is modifying the graph...
iothread(mirror): *continues*, as nothing is stopping it
iothread(mirror): bdrv_drained_begin(bs);
/* draining reads the graph while it is modified!! */
main loop: done modifying the graph...

In order to fix this, we can simply allow drv->drained_poll()
to be called only by the iothread, and not the main loop.
We distinguish it by using in_aio_context_home_thread(), that
returns false if @ctx is not the same as the thread that runs it.

Co-Developed-by: Paolo Bonzini 
Signed-off-by: Emanuele Giuseppe Esposito 
---
 blockjob.c | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/blockjob.c b/blockjob.c
index 4868453d74..14a919b3cc 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -110,7 +110,9 @@ static bool child_job_drained_poll(BdrvChild *c)
 BlockJob *bjob = c->opaque;
 Job *job = &bjob->job;
 const BlockJobDriver *drv = block_job_driver(bjob);
+AioContext *ctx;
 
+ctx = job->aio_context;
 /* An inactive or completed job doesn't have any pending requests. Jobs
  * with !job->busy are either already paused or have a pause point after
  * being reentered, so no job driver code will run before they pause. */
@@ -118,9 +120,14 @@ static bool child_job_drained_poll(BdrvChild *c)
 return false;
 }
 
-/* Otherwise, assume that it isn't fully stopped yet, but allow the job to
- * override this assumption. */
-if (drv->drained_poll) {
+/*
+ * Otherwise, assume that it isn't fully stopped yet, but allow the job to
+ * override this assumption, if the drain is being performed in the
+ * iothread. We need to check that the caller is the home thread because
+ * it could otherwise lead the main loop to exit polling while the job
+ * has not paused yet.
+ */
+if (in_aio_context_home_thread(ctx) && drv->drained_poll) {
 return drv->drained_poll(bjob);
 } else {
 return true;
-- 
2.31.1

[PATCH v2 07/10] test-bdrv-drain.c: remove test_detach_by_parent_cb()

[Emanuele Giuseppe Esposito]

This test uses a callback of an I/O function (blk_aio_preadv)
to modify the graph, using bdrv_attach_child.
This is simply not allowed anymore. I/O cannot change the graph.

Before "block/io.c: make bdrv_do_drained_begin_quiesce static
and introduce bdrv_drained_begin_no_poll", the test would simply
be at risk of failure, because if bdrv_replace_child_noperm()
(called to modify the graph) would call a drain,
then one callback of .drained_begin() is bdrv_do_drained_begin_quiesce,
that specifically asserts that we are not in a coroutine.

Now that we fixed the behavior, the drain will invoke a bh in the
main loop, so we don't have such problem. However, this test is still
illegal and fails because we forbid graph changes from I/O paths.

Once we add the required subtree_drains to protect
bdrv_replace_child_noperm(), the key problem in this test is in:

acb = blk_aio_preadv(blk, 0, &qiov, 0, detach_by_parent_aio_cb, NULL);
/* Drain and check the expected result */
bdrv_subtree_drained_begin(parent_b);

because the detach_by_parent_aio_cb calls detach_indirect_bh(), that
modifies the graph and is invoked during bdrv_subtree_drained_begin().
The call stack is the following:
1. blk_aio_preadv() creates a coroutine, increments in_flight counter
and enters the coroutine running blk_aio_read_entry()
2. blk_aio_read_entry() performs the read and then schedules a bh to
   complete (blk_aio_complete)
3. at this point, subtree_drained_begin() kicks in and waits for all
   in_flight requests, polling
4. polling allows the bh to be scheduled, so blk_aio_complete runs
5. blk_aio_complete *first* invokes the callback
   (detach_by_parent_aio_cb) and then decrements the in_flight counter
6. Here we have the problem: detach_by_parent_aio_cb modifies the graph,
   so both bdrv_unref_child() and bdrv_attach_child() will have
   subtree_drains inside. And this causes a deadlock, because the
   nested drain will wait for in_flight counter to go to zero, which
   is only happening once the drain itself finishes.

Different story is test_detach_by_driver_cb(): in this case,
detach_by_parent_aio_cb() does not call detach_indirect_bh(),
but it is instead called as a bh running in the main loop by
detach_by_driver_cb_drained_begin(), the callback for
.drained_begin().

This test was added in 231281ab42 and part of the series
"Drain fixes and cleanups, part 3"
https://lists.nongnu.org/archive/html/qemu-block/2018-05/msg01132.html
but as explained above I believe that it is not valid anymore, and
can be discarded.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Stefan Hajnoczi 
---
 tests/unit/test-bdrv-drain.c | 46 +---
 1 file changed, 11 insertions(+), 35 deletions(-)

diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c
index f750ddfc4e..fa0d86209a 100644
--- a/tests/unit/test-bdrv-drain.c
+++ b/tests/unit/test-bdrv-drain.c
@@ -1294,7 +1294,6 @@ struct detach_by_parent_data {
 BdrvChild *child_b;
 BlockDriverState *c;
 BdrvChild *child_c;
-bool by_parent_cb;
 };
 static struct detach_by_parent_data detach_by_parent_data;
 
@@ -1312,12 +1311,7 @@ static void detach_indirect_bh(void *opaque)
 
 static void detach_by_parent_aio_cb(void *opaque, int ret)
 {
-struct detach_by_parent_data *data = &detach_by_parent_data;
-
 g_assert_cmpint(ret, ==, 0);
-if (data->by_parent_cb) {
-detach_indirect_bh(data);
-}
 }
 
 static BdrvChildClass detach_by_driver_cb_class;
@@ -1339,31 +1333,24 @@ static void detach_by_driver_cb_drained_begin(BdrvChild 
*child)
  *\ /   \
  * A B C
  *
- * by_parent_cb == true:  Test that parent callbacks don't poll
- *
- * PA has a pending write request whose callback changes the child nodes of
- * PB: It removes B and adds C instead. The subtree of PB is drained, which
- * will indirectly drain the write request, too.
- *
- * by_parent_cb == false: Test that bdrv_drain_invoke() doesn't poll
+ * Test that bdrv_drain_invoke() doesn't poll
  *
  * PA's BdrvChildClass has a .drained_begin callback that schedules a BH
  * that does the same graph change. If bdrv_drain_invoke() calls it, the
  * state is messed up, but if it is only polled in the single
  * BDRV_POLL_WHILE() at the end of the drain, this should work fine.
  */
-static void test_detach_indirect(bool by_parent_cb)
+static void test_detach_indirect(void)
 {
 BlockBackend *blk;
 BlockDriverState *parent_a, *parent_b, *a, *b, *c;
 BdrvChild *child_a, *child_b;
 BlockAIOCB *acb;
+BDRVTestState *s;
 
 QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, NULL, 0);
 
-if (!by_parent_cb) {
-detach_by_driver_cb_class = child_of_bds;
-}
+detach_by_driver_cb_class = child_of_bds;
 
 /* Create all involved nodes */
 parent_a = bdrv_new_open_driver(&bdrv_test, "parent-a", BDRV_O_RDWR,
@@ -1382,10 +1369,8 @@ static void test_detach_indirect(bool by_parent_cb)
 
 /* If we want to 

[PATCH v2 05/10] block.c: bdrv_replace_child_noperm: first call ->attach(), and then add child

[Emanuele Giuseppe Esposito]

Doing the opposite can make adding the child node to a non-drained node,
as apply_subtree_drain is only done in ->attach() and thus make
assert_bdrv_graph_writable fail.

This can happen for example during a transaction rollback (test 245,
test_io_with_graph_changes):
1. a node is removed from the graph, thus it is undrained
2. then something happens, and we need to roll back the transactions
   through tran_abort()
3. at this point, the current code would first attach the undrained node
   to the graph via QLIST_INSERT_HEAD, and then call ->attach() that
   will take care of restoring the drain with apply_subtree_drain(),
   leaving the node undrained between the two operations.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 block.c | 20 +++-
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/block.c b/block.c
index d870ba5393..c6a550f9c6 100644
--- a/block.c
+++ b/block.c
@@ -1434,6 +1434,11 @@ static void bdrv_inherited_options(BdrvChildRole role, 
bool parent_is_format,
 *child_flags = flags;
 }
 
+/*
+ * Add the child node to child->opaque->children list,
+ * and then apply the drain to the whole child subtree,
+ * so that the drain count matches with the parent.
+ */
 static void bdrv_child_cb_attach(BdrvChild *child)
 {
 BlockDriverState *bs = child->opaque;
@@ -2889,8 +2894,6 @@ static void bdrv_replace_child_noperm(BdrvChild **childp,
 }
 
 if (new_bs) {
-assert_bdrv_graph_writable(new_bs);
-QLIST_INSERT_HEAD(&new_bs->parents, child, next_parent);
 
 /*
  * Detaching the old node may have led to the new node's
@@ -2901,12 +2904,19 @@ static void bdrv_replace_child_noperm(BdrvChild 
**childp,
 assert(new_bs->quiesce_counter <= new_bs_quiesce_counter);
 drain_saldo += new_bs->quiesce_counter - new_bs_quiesce_counter;
 
-/* Attach only after starting new drained sections, so that recursive
- * drain sections coming from @child don't get an extra .drained_begin
- * callback. */
+/*
+ * First call ->attach() cb.
+ * In child_of_bds case, add child to the parent
+ * (child->opaque) ->children list and if
+ * necessary add missing drains in the child subtree.
+ */
 if (child->klass->attach) {
 child->klass->attach(child);
 }
+
+/* Then add child to new_bs->parents list */
+assert_bdrv_graph_writable(new_bs);
+QLIST_INSERT_HEAD(&new_bs->parents, child, next_parent);
 }
 
 /*
-- 
2.31.1

[PATCH v2 02/10] bdrv_parent_drained_begin_single: handle calls from coroutine context

[Emanuele Giuseppe Esposito]

bdrv_parent_drained_begin_single() is also called by
bdrv_replace_child_noperm(). The latter is often called
from coroutines, for example in bdrv_co_create_opts() callbacks.

This can potentially create deadlocks, because if the drain_saldo
in bdrv_replace_child_noperm is > 0, the coroutine will start
polling using BDRV_POLL_WHILE. Right now this does not seem
to happen, but if additional drains are used in future,
this will be much more likely to happen.

Fix the problem by doing something very similar to
bdrv_do_drained_begin(): if in coroutine, schedule a bh
to execute the drain in the main loop, and enter the coroutine
only once it is done.

Just as the other drains, check the coroutine case only when
effectively polling.

As a consequence of this, remove the coroutine assertion in
bdrv_do_drained_begin_quiesce. We are never polling in that case.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 block/io.c | 98 +-
 1 file changed, 97 insertions(+), 1 deletion(-)

diff --git a/block/io.c b/block/io.c
index 4a3e8d037d..e446782ae0 100644
--- a/block/io.c
+++ b/block/io.c
@@ -67,10 +67,101 @@ static void 
bdrv_parent_drained_end_single_no_poll(BdrvChild *c,
 }
 }
 
+typedef struct {
+Coroutine *co;
+BdrvChild *child;
+bool done;
+bool begin;
+bool poll;
+} BdrvCoDrainParentData;
+
+static void bdrv_co_drain_parent_bh_cb(void *opaque)
+{
+BdrvCoDrainParentData *data = opaque;
+Coroutine *co = data->co;
+BdrvChild *child = data->child;
+BlockDriverState *bs = child->bs;
+AioContext *ctx = bdrv_get_aio_context(bs);
+
+if (bs) {
+aio_context_acquire(ctx);
+bdrv_dec_in_flight(bs);
+}
+
+if (data->begin) {
+bdrv_parent_drained_begin_single(child, data->poll);
+} else {
+assert(!data->poll);
+bdrv_parent_drained_end_single(child);
+}
+
+if (bs) {
+aio_context_release(ctx);
+}
+
+data->done = true;
+aio_co_wake(co);
+}
+
+static void coroutine_fn bdrv_co_yield_to_drain_parent(BdrvChild *c,
+   bool begin, bool poll)
+{
+BdrvCoDrainParentData data;
+Coroutine *self = qemu_coroutine_self();
+BlockDriverState *bs = c->bs;
+AioContext *ctx = bdrv_get_aio_context(bs);
+AioContext *co_ctx = qemu_coroutine_get_aio_context(self);
+
+/* Calling bdrv_drain() from a BH ensures the current coroutine yields and
+ * other coroutines run if they were queued by aio_co_enter(). */
+
+assert(qemu_in_coroutine());
+data = (BdrvCoDrainParentData) {
+.co = self,
+.child = c,
+.done = false,
+.begin = begin,
+.poll = poll,
+};
+
+if (bs) {
+bdrv_inc_in_flight(bs);
+}
+
+/*
+ * Temporarily drop the lock across yield or we would get deadlocks.
+ * bdrv_co_yield_to_drain_parent() reaquires the lock as needed.
+ *
+ * When we yield below, the lock for the current context will be
+ * released, so if this is actually the lock that protects bs, don't drop
+ * it a second time.
+ */
+if (ctx != co_ctx) {
+aio_context_release(ctx);
+}
+replay_bh_schedule_oneshot_event(ctx, bdrv_co_drain_parent_bh_cb, &data);
+
+qemu_coroutine_yield();
+/* If we are resumed from some other event (such as an aio completion or a
+ * timer callback), it is a bug in the caller that should be fixed. */
+assert(data.done);
+
+/* Reaquire the AioContext of bs if we dropped it */
+if (ctx != co_ctx) {
+aio_context_acquire(ctx);
+}
+}
+
 void bdrv_parent_drained_end_single(BdrvChild *c)
 {
 int drained_end_counter = 0;
 IO_OR_GS_CODE();
+
+if (qemu_in_coroutine()) {
+bdrv_co_yield_to_drain_parent(c, false, false);
+return;
+}
+
 bdrv_parent_drained_end_single_no_poll(c, &drained_end_counter);
 BDRV_POLL_WHILE(c->bs, qatomic_read(&drained_end_counter) > 0);
 }
@@ -116,6 +207,12 @@ static bool bdrv_parent_drained_poll(BlockDriverState *bs, 
BdrvChild *ignore,
 void bdrv_parent_drained_begin_single(BdrvChild *c, bool poll)
 {
 IO_OR_GS_CODE();
+
+if (poll && qemu_in_coroutine()) {
+bdrv_co_yield_to_drain_parent(c, true, poll);
+return;
+}
+
 c->parent_quiesce_counter++;
 if (c->klass->drained_begin) {
 c->klass->drained_begin(c);
@@ -430,7 +527,6 @@ void bdrv_do_drained_begin_quiesce(BlockDriverState *bs,
BdrvChild *parent, bool ignore_bds_parents)
 {
 IO_OR_GS_CODE();
-assert(!qemu_in_coroutine());
 
 /* Stop things in parent-to-child order */
 if (qatomic_fetch_inc(&bs->quiesce_counter) == 0) {
-- 
2.31.1

[PATCH v2 06/10] test-bdrv-drain.c: adapt test to support additional subtree drains

[Emanuele Giuseppe Esposito]

There will be a problem in this test when we will add
subtree drains in bdrv_replace_child_noperm:

test_detach_indirect is only interested in observing the first
call to .drained_begin. In the original test, there was only a single
subtree drain; however, with additional drains introduced in
bdrv_replace_child_noperm(), the test callback would be called too early
and/or multiple times.
Override the callback only when we actually want to use it, and put back
the original after it's been invoked.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 tests/unit/test-bdrv-drain.c | 15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c
index 36be84ae55..f750ddfc4e 100644
--- a/tests/unit/test-bdrv-drain.c
+++ b/tests/unit/test-bdrv-drain.c
@@ -1320,15 +1320,18 @@ static void detach_by_parent_aio_cb(void *opaque, int 
ret)
 }
 }
 
+static BdrvChildClass detach_by_driver_cb_class;
+
 static void detach_by_driver_cb_drained_begin(BdrvChild *child)
 {
+/* restore .drained_begin cb, we don't need it anymore. */
+detach_by_driver_cb_class.drained_begin = child_of_bds.drained_begin;
+
 aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
 detach_indirect_bh, &detach_by_parent_data);
 child_of_bds.drained_begin(child);
 }
 
-static BdrvChildClass detach_by_driver_cb_class;
-
 /*
  * Initial graph:
  *
@@ -1360,8 +1363,6 @@ static void test_detach_indirect(bool by_parent_cb)
 
 if (!by_parent_cb) {
 detach_by_driver_cb_class = child_of_bds;
-detach_by_driver_cb_class.drained_begin =
-detach_by_driver_cb_drained_begin;
 }
 
 /* Create all involved nodes */
@@ -1419,6 +1420,12 @@ static void test_detach_indirect(bool by_parent_cb)
 acb = blk_aio_preadv(blk, 0, &qiov, 0, detach_by_parent_aio_cb, NULL);
 g_assert(acb != NULL);
 
+if (!by_parent_cb) {
+/* set .drained_begin cb to run only in the following drain. */
+detach_by_driver_cb_class.drained_begin =
+detach_by_driver_cb_drained_begin;
+}
+
 /* Drain and check the expected result */
 bdrv_subtree_drained_begin(parent_b);
 
-- 
2.31.1

[PATCH] ppc/pnv: Remove user-created PHB{3,4,5} devices

[Cédric Le Goater]

On a real system with POWER{8,9,10} processors, PHBs are sub-units of
the processor, they can be deactivated by firmware but not plugged in
or out like a PCI adapter on a slot. Nevertheless, having user-created
PHBs in QEMU seemed to be a good idea for testing purposes :

 1. having a limited set of PHBs speedups boot time.
 2. it is useful to be able to mimic a partially broken topology you
some time have to deal with during bring-up.

PowerNV is also used for distro install tests and having libvirt
support eases these tasks. libvirt prefers to run the machine with
-nodefaults to be sure not to drag unexpected devices which would need
to be defined in the domain file without being specified on the QEMU
command line. For this reason :

 3. -nodefaults should not include default PHBs

User-created PHB{3,4,5} devices satisfied all these needs but reality
proves to be a bit more complex, internally when modeling such
devices, and externally when dealing with the user interface.

Req 1. and 2. can be simply addressed differently with a machine option:
"phb-mask=", which QEMU would use to enable/disable PHB device
nodes when creating the device tree.

For Req 3., we need to make sure we are taking the right approach. It
seems that we should expose a new type of user-created PHB device, a
generic virtualized one, that libvirt would use and not one depending
on the processor revision. This needs more thinking.

For now, remove user-created PHB{3,4,5} devices. All the cleanups we
did are not lost and they will be useful for the next steps.

Fixes: 5bc67b052b51 ("ppc/pnv: Introduce user creatable pnv-phb4 devices")
Fixes: 1f6a88fffc75 ("ppc/pnv: Introduce support for user created PHB3 devices")
Signed-off-by: Cédric Le Goater 
---
 include/hw/ppc/pnv.h   |  1 -
 hw/pci-host/pnv_phb3.c | 33 ++--
 hw/pci-host/pnv_phb4.c | 62 ++
 hw/pci-host/pnv_phb4_pec.c |  7 ++---
 hw/ppc/pnv.c   | 25 +--
 5 files changed, 10 insertions(+), 118 deletions(-)

diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index 1e34ddd502d8..86cb7d7f971b 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -190,7 +190,6 @@ DECLARE_INSTANCE_CHECKER(PnvChip, PNV_CHIP_POWER10,
 
 PowerPCCPU *pnv_chip_find_cpu(PnvChip *chip, uint32_t pir);
 void pnv_phb_attach_root_port(PCIHostState *pci, const char *name);
-void pnv_chip_parent_fixup(PnvChip *chip, Object *obj, int index);
 
 #define TYPE_PNV_MACHINE   MACHINE_TYPE_NAME("powernv")
 typedef struct PnvMachineClass PnvMachineClass;
diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
index aafd46b635e8..ac801ac83582 100644
--- a/hw/pci-host/pnv_phb3.c
+++ b/hw/pci-host/pnv_phb3.c
@@ -994,30 +994,6 @@ static void pnv_phb3_realize(DeviceState *dev, Error 
**errp)
 PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
 int i;
 
-/* User created devices */
-if (!phb->chip) {
-Error *local_err = NULL;
-BusState *s;
-
-phb->chip = pnv_get_chip(pnv, phb->chip_id);
-if (!phb->chip) {
-error_setg(errp, "invalid chip id: %d", phb->chip_id);
-return;
-}
-
-/*
- * Reparent user created devices to the chip to build
- * correctly the device tree.
- */
-pnv_chip_parent_fixup(phb->chip, OBJECT(phb), phb->phb_id);
-
-s = qdev_get_parent_bus(DEVICE(phb->chip));
-if (!qdev_set_parent_bus(DEVICE(phb), s, &local_err)) {
-error_propagate(errp, local_err);
-return;
-}
-}
-
 if (phb->phb_id >= PNV_CHIP_GET_CLASS(phb->chip)->num_phbs) {
 error_setg(errp, "invalid PHB index: %d", phb->phb_id);
 return;
@@ -1077,10 +1053,7 @@ static void pnv_phb3_realize(DeviceState *dev, Error 
**errp)
 
 pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb);
 
-if (defaults_enabled()) {
-pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb),
- TYPE_PNV_PHB3_ROOT_PORT);
-}
+pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
 }
 
 void pnv_phb3_update_regions(PnvPHB3 *phb)
@@ -1131,7 +1104,7 @@ static void pnv_phb3_class_init(ObjectClass *klass, void 
*data)
 dc->realize = pnv_phb3_realize;
 device_class_set_props(dc, pnv_phb3_properties);
 set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
-dc->user_creatable = true;
+dc->user_creatable = false;
 }
 
 static const TypeInfo pnv_phb3_type_info = {
@@ -1201,7 +1174,7 @@ static void pnv_phb3_root_port_class_init(ObjectClass 
*klass, void *data)
 
 device_class_set_parent_realize(dc, pnv_phb3_root_port_realize,
 &rpc->parent_realize);
-dc->user_creatable = true;
+dc->user_creatable = false;
 
 k->vendor_id = PCI_VENDOR_ID_IBM;
 k->device_id = 0x03dc;
diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
index d1a911f988b7..b30176209303 100644
-

[PATCH v2 04/10] block.c: bdrv_replace_child_noperm: first remove the child, and then call ->detach()

[Emanuele Giuseppe Esposito]

Doing the opposite can make ->detach() (more precisely
bdrv_unapply_subtree_drain() in bdrv_child_cb_detach) undo the subtree_drain
just performed to protect the removal of the child from the graph,
thus making the fully-enabled assert_bdrv_graph_writable fail.

Note that assert_bdrv_graph_writable is not yet fully enabled.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 block.c | 19 ++-
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/block.c b/block.c
index 372f16f4a0..d870ba5393 100644
--- a/block.c
+++ b/block.c
@@ -1448,6 +1448,11 @@ static void bdrv_child_cb_attach(BdrvChild *child)
 bdrv_apply_subtree_drain(child, bs);
 }
 
+/*
+ * Unapply the drain in the whole child subtree, as
+ * it has been already detached, and then remove from
+ * child->opaque->children.
+ */
 static void bdrv_child_cb_detach(BdrvChild *child)
 {
 BlockDriverState *bs = child->opaque;
@@ -2864,14 +2869,18 @@ static void bdrv_replace_child_noperm(BdrvChild 
**childp,
 }
 
 if (old_bs) {
-/* Detach first so that the recursive drain sections coming from @child
- * are already gone and we only end the drain sections that came from
- * elsewhere. */
+/* First remove child from child->bs->parents list */
+assert_bdrv_graph_writable(old_bs);
+QLIST_REMOVE(child, next_parent);
+/*
+ * Then call ->detach() cb.
+ * In child_of_bds case, update the child parent
+ * (child->opaque) ->children list and
+ * remove any drain left in the child subtree.
+ */
 if (child->klass->detach) {
 child->klass->detach(child);
 }
-assert_bdrv_graph_writable(old_bs);
-QLIST_REMOVE(child, next_parent);
 }
 
 child->bs = new_bs;
-- 
2.31.1

Re: [PATCH experiment 08/16] tracetool: add extern "C" around generated headers

[Philippe Mathieu-Daudé]

On 14/3/22 10:31, Paolo Bonzini wrote:

Signed-off-by: Paolo Bonzini 
---
  scripts/tracetool/format/h.py | 8 +++-
  1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/scripts/tracetool/format/h.py b/scripts/tracetool/format/h.py
index e94f0be7da..2d92fa8bd2 100644
--- a/scripts/tracetool/format/h.py
+++ b/scripts/tracetool/format/h.py
@@ -27,6 +27,9 @@ def generate(events, backend, group):
  '#define TRACE_%s_GENERATED_TRACERS_H' % group.upper(),
  '',
  '#include "%s"' % header,
+'#ifdef __cplusplus',
+'extern "C" {',
+'#endif'


Why not use G_BEGIN_DECLS?

Anyhow,
Reviewed-by: Philippe Mathieu-Daudé 

[PULL 07/12] ui/shader: free associated programs

[marcandre . lureau]

From: Marc-André Lureau 

Signed-off-by: Marc-André Lureau 
Acked-by: Gerd Hoffmann 
---
 ui/shader.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ui/shader.c b/ui/shader.c
index 4c80fc831f68..ab448c41d4c6 100644
--- a/ui/shader.c
+++ b/ui/shader.c
@@ -172,5 +172,8 @@ void qemu_gl_fini_shader(QemuGLShader *gls)
 if (!gls) {
 return;
 }
+glDeleteProgram(gls->texture_blit_prog);
+glDeleteProgram(gls->texture_blit_flip_prog);
+glDeleteProgram(gls->texture_blit_vao);
 g_free(gls);
 }
-- 
2.35.1.273.ge6ebfd0e8cbb

Re: [PATCH experiment 03/16] coroutine: small code cleanup in qemu_co_rwlock_wrlock

[Philippe Mathieu-Daudé]

On 14/3/22 10:31, Paolo Bonzini wrote:

qemu_co_rwlock_wrlock stores the current coroutine in a loc variable,
use it instead of calling qemu_coroutine_self() again.

Signed-off-by: Paolo Bonzini 
---
  util/qemu-coroutine-lock.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)


Reviewed-by: Philippe Mathieu-Daudé 

[PATCH v6 07/18] jobs: add job lock in find_* functions

[Emanuele Giuseppe Esposito]

Both blockdev.c and job-qmp.c have TOC/TOU conditions, because
they first search for the job and then perform an action on it.
Therefore, we need to do the search + action under the same
job mutex critical section.

Note: at this stage, job_{lock/unlock} and job lock guard macros
are *nop*.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 blockdev.c | 46 +++---
 job-qmp.c  | 37 +
 2 files changed, 64 insertions(+), 19 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 8722e5d4b9..39e6fe4d59 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3311,9 +3311,13 @@ out:
 aio_context_release(aio_context);
 }
 
-/* Get a block job using its ID and acquire its AioContext */
-static BlockJob *find_block_job(const char *id, AioContext **aio_context,
-Error **errp)
+/*
+ * Get a block job using its ID and acquire its AioContext.
+ * Called with job_mutex held.
+ */
+static BlockJob *find_block_job_locked(const char *id,
+   AioContext **aio_context,
+   Error **errp)
 {
 BlockJob *job;
 
@@ -3322,7 +3326,6 @@ static BlockJob *find_block_job(const char *id, 
AioContext **aio_context,
 *aio_context = NULL;
 
 job = block_job_get(id);
-
 if (!job) {
 error_set(errp, ERROR_CLASS_DEVICE_NOT_ACTIVE,
   "Block job '%s' not found", id);
@@ -3338,7 +3341,10 @@ static BlockJob *find_block_job(const char *id, 
AioContext **aio_context,
 void qmp_block_job_set_speed(const char *device, int64_t speed, Error **errp)
 {
 AioContext *aio_context;
-BlockJob *job = find_block_job(device, &aio_context, errp);
+BlockJob *job;
+
+JOB_LOCK_GUARD();
+job = find_block_job_locked(device, &aio_context, errp);
 
 if (!job) {
 return;
@@ -3352,7 +3358,10 @@ void qmp_block_job_cancel(const char *device,
   bool has_force, bool force, Error **errp)
 {
 AioContext *aio_context;
-BlockJob *job = find_block_job(device, &aio_context, errp);
+BlockJob *job;
+
+JOB_LOCK_GUARD();
+job = find_block_job_locked(device, &aio_context, errp);
 
 if (!job) {
 return;
@@ -3377,7 +3386,10 @@ out:
 void qmp_block_job_pause(const char *device, Error **errp)
 {
 AioContext *aio_context;
-BlockJob *job = find_block_job(device, &aio_context, errp);
+BlockJob *job;
+
+JOB_LOCK_GUARD();
+job = find_block_job_locked(device, &aio_context, errp);
 
 if (!job) {
 return;
@@ -3391,7 +3403,10 @@ void qmp_block_job_pause(const char *device, Error 
**errp)
 void qmp_block_job_resume(const char *device, Error **errp)
 {
 AioContext *aio_context;
-BlockJob *job = find_block_job(device, &aio_context, errp);
+BlockJob *job;
+
+JOB_LOCK_GUARD();
+job = find_block_job_locked(device, &aio_context, errp);
 
 if (!job) {
 return;
@@ -3405,7 +3420,10 @@ void qmp_block_job_resume(const char *device, Error 
**errp)
 void qmp_block_job_complete(const char *device, Error **errp)
 {
 AioContext *aio_context;
-BlockJob *job = find_block_job(device, &aio_context, errp);
+BlockJob *job;
+
+JOB_LOCK_GUARD();
+job = find_block_job_locked(device, &aio_context, errp);
 
 if (!job) {
 return;
@@ -3419,7 +3437,10 @@ void qmp_block_job_complete(const char *device, Error 
**errp)
 void qmp_block_job_finalize(const char *id, Error **errp)
 {
 AioContext *aio_context;
-BlockJob *job = find_block_job(id, &aio_context, errp);
+BlockJob *job;
+
+JOB_LOCK_GUARD();
+job = find_block_job_locked(id, &aio_context, errp);
 
 if (!job) {
 return;
@@ -3442,9 +3463,12 @@ void qmp_block_job_finalize(const char *id, Error **errp)
 void qmp_block_job_dismiss(const char *id, Error **errp)
 {
 AioContext *aio_context;
-BlockJob *bjob = find_block_job(id, &aio_context, errp);
+BlockJob *bjob;
 Job *job;
 
+JOB_LOCK_GUARD();
+bjob = find_block_job_locked(id, &aio_context, errp);
+
 if (!bjob) {
 return;
 }
diff --git a/job-qmp.c b/job-qmp.c
index 270df1eb7e..58ca9b6632 100644
--- a/job-qmp.c
+++ b/job-qmp.c
@@ -29,8 +29,11 @@
 #include "qapi/error.h"
 #include "trace/trace-root.h"
 
-/* Get a job using its ID and acquire its AioContext */
-static Job *find_job(const char *id, AioContext **aio_context, Error **errp)
+/*
+ * Get a block job using its ID and acquire its AioContext.
+ * Called with job_mutex held.
+ */
+static Job *find_job_locked(const char *id, AioContext **aio_context, Error 
**errp)
 {
 Job *job;
 
@@ -51,7 +54,10 @@ static Job *find_job(const char *id, AioContext 
**aio_context, Error **errp)
 void qmp_job_cancel(const char *id, Error **errp)
 {
 AioContext *aio_context;
-Job *job = find_job(id, &aio_context, errp);
+Job *job;
+
+JOB_LOCK_GUARD();
+job = find_job_locked(id, &aio_context, errp);
 
 if (!job) 

[PATCH v6 04/18] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED

[Emanuele Giuseppe Esposito]

Same as AIO_WAIT_WHILE macro, but if we are in the Main loop
do not release and then acquire ctx_ 's aiocontext.

Once all Aiocontext locks go away, this macro will replace
AIO_WAIT_WHILE.

Reviewed-by: Stefan Hajnoczi 
Signed-off-by: Emanuele Giuseppe Esposito 
---
 include/block/aio-wait.h | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h
index b39eefb38d..c97621ed4a 100644
--- a/include/block/aio-wait.h
+++ b/include/block/aio-wait.h
@@ -59,10 +59,13 @@ typedef struct {
 extern AioWait global_aio_wait;
 
 /**
- * AIO_WAIT_WHILE:
+ * _AIO_WAIT_WHILE:
  * @ctx: the aio context, or NULL if multiple aio contexts (for which the
  *   caller does not hold a lock) are involved in the polling condition.
  * @cond: wait while this conditional expression is true
+ * @unlock: whether to unlock and then lock again @ctx. This apples
+ * only when waiting for another AioContext from the main loop.
+ * Otherwise it's ignored.
  *
  * Wait while a condition is true.  Use this to implement synchronous
  * operations that require event loop activity.
@@ -75,7 +78,7 @@ extern AioWait global_aio_wait;
  * wait on conditions between two IOThreads since that could lead to deadlock,
  * go via the main loop instead.
  */
-#define AIO_WAIT_WHILE(ctx, cond) ({   \
+#define _AIO_WAIT_WHILE(ctx, cond, unlock) ({  \
 bool waited_ = false;  \
 AioWait *wait_ = &global_aio_wait; \
 AioContext *ctx_ = (ctx);  \
@@ -90,11 +93,11 @@ extern AioWait global_aio_wait;
 assert(qemu_get_current_aio_context() ==   \
qemu_get_aio_context());\
 while ((cond)) {   \
-if (ctx_) {\
+if (unlock && ctx_) {  \
 aio_context_release(ctx_); \
 }  \
 aio_poll(qemu_get_aio_context(), true);\
-if (ctx_) {\
+if (unlock && ctx_) {  \
 aio_context_acquire(ctx_); \
 }  \
 waited_ = true;\
@@ -103,6 +106,12 @@ extern AioWait global_aio_wait;
 qatomic_dec(&wait_->num_waiters);  \
 waited_; })
 
+#define AIO_WAIT_WHILE(ctx, cond)  \
+_AIO_WAIT_WHILE(ctx, cond, true)
+
+#define AIO_WAIT_WHILE_UNLOCKED(ctx, cond) \
+_AIO_WAIT_WHILE(ctx, cond, false)
+
 /**
  * aio_wait_kick:
  * Wake up the main thread if it is waiting on AIO_WAIT_WHILE().  During
-- 
2.31.1

Re: [PATCH v2 2/4] Introduce event-loop-base abstract class

[Stefan Hajnoczi]

On Fri, Mar 11, 2022 at 11:17:22AM +0100, Nicolas Saenz Julienne wrote:
> On Thu, 2022-03-10 at 10:25 +, Stefan Hajnoczi wrote:
> > On Thu, Mar 03, 2022 at 03:58:20PM +0100, Nicolas Saenz Julienne wrote:
> > > @@ -2935,13 +2947,6 @@ qemu_syms = custom_target('qemu.syms', output: 
> > > 'qemu.syms',
> > >   capture: true,
> > >   command: [undefsym, nm, '@INPUT@'])
> > >  
> > > -qom_ss = qom_ss.apply(config_host, strict: false)
> > > -libqom = static_library('qom', qom_ss.sources() + genh,
> > > -dependencies: [qom_ss.dependencies()],
> > > -name_suffix: 'fa')
> > > -
> > > -qom = declare_dependency(link_whole: libqom)
> > > -
> > 
> > Why was it necessary to move qom_ss and subdir('hw') up? Can
> > event_loop_base be defined down here instead?
> 
> The way I setup it up, qemuutil now depdens on event_loop_base which in turn
> depends on qom. IIUC I can't declare dependencies without declaring first the
> libraries and source sets. All has to happen sequencially. With this in mind,
> almost all libraries depend on libqemuutil so moving it down isn't possible.

I see now. The qemuutil dependency on event_loop_base is introduced in
the next patch so the reason wasn't clear at this point in the patch
series.

Stefan


signature.asc
Description: PGP signature

[PATCH v6 11/18] job.h: rename job API functions called with job_mutex held

[Emanuele Giuseppe Esposito]

With the *nop* job_lock/unlock placed, rename the job functions
of the job API that are always under job_mutex, adding "_locked"
suffix.

List of functions that get this suffix:
job_txn_unref   job_txn_add_job
job_ref job_unref
job_enter_cond  job_finish_sync
job_is_readyjob_pause
job_resume  job_user_pause
job_user_paused job_user_resume
job_nextjob_get
job_apply_verb  job_complete
job_cancel  job_user_cancel
job_cancel_sync job_complete_sync
job_finalizejob_dismiss

Note that "locked" refers to the *nop* job_lock/unlock, and not
real_job_lock/unlock.

No functional change intended.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 block.c  |   2 +-
 block/mirror.c   |   2 +-
 block/replication.c  |   4 +-
 blockdev.c   |  32 
 blockjob.c   |  14 ++--
 include/qemu/job.h   | 119 ++--
 job-qmp.c|  26 +++
 job.c| 129 ---
 qemu-img.c   |   6 +-
 tests/unit/test-bdrv-drain.c |   2 +-
 tests/unit/test-block-iothread.c |   4 +-
 tests/unit/test-blockjob-txn.c   |  14 ++--
 tests/unit/test-blockjob.c   |  30 +++
 13 files changed, 213 insertions(+), 171 deletions(-)

diff --git a/block.c b/block.c
index 5dc46fde11..9cf3416bea 100644
--- a/block.c
+++ b/block.c
@@ -4979,7 +4979,7 @@ static void bdrv_close(BlockDriverState *bs)
 void bdrv_close_all(void)
 {
 WITH_JOB_LOCK_GUARD() {
-assert(job_next(NULL) == NULL);
+assert(job_next_locked(NULL) == NULL);
 }
 GLOBAL_STATE_CODE();
 
diff --git a/block/mirror.c b/block/mirror.c
index b3b64a280a..a7c3bab382 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1157,7 +1157,7 @@ static void mirror_complete(Job *job, Error **errp)
 /* If the job is paused, it will be re-entered when it is resumed */
 WITH_JOB_LOCK_GUARD() {
 if (!job->paused) {
-job_enter_cond(job, NULL);
+job_enter_cond_locked(job, NULL);
 }
 }
 }
diff --git a/block/replication.c b/block/replication.c
index a03b28726e..50ea778937 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -150,7 +150,7 @@ static void replication_close(BlockDriverState *bs)
 commit_job = &s->commit_job->job;
 assert(commit_job->aio_context == qemu_get_current_aio_context());
 WITH_JOB_LOCK_GUARD() {
-job_cancel_sync(commit_job, false);
+job_cancel_sync_locked(commit_job, false);
 }
 }
 
@@ -729,7 +729,7 @@ static void replication_stop(ReplicationState *rs, bool 
failover, Error **errp)
  */
 if (s->backup_job) {
 WITH_JOB_LOCK_GUARD() {
-job_cancel_sync(&s->backup_job->job, true);
+job_cancel_sync_locked(&s->backup_job->job, true);
 }
 }
 
diff --git a/blockdev.c b/blockdev.c
index 39e6fe4d59..1ac9dac1b0 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -157,7 +157,7 @@ void blockdev_mark_auto_del(BlockBackend *blk)
 AioContext *aio_context = job->job.aio_context;
 aio_context_acquire(aio_context);
 
-job_cancel(&job->job, false);
+job_cancel_locked(&job->job, false);
 
 aio_context_release(aio_context);
 }
@@ -1841,7 +1841,7 @@ static void drive_backup_abort(BlkActionState *common)
 aio_context_acquire(aio_context);
 
 WITH_JOB_LOCK_GUARD() {
-job_cancel_sync(&state->job->job, true);
+job_cancel_sync_locked(&state->job->job, true);
 }
 
 aio_context_release(aio_context);
@@ -1944,7 +1944,7 @@ static void blockdev_backup_abort(BlkActionState *common)
 aio_context_acquire(aio_context);
 
 WITH_JOB_LOCK_GUARD() {
-job_cancel_sync(&state->job->job, true);
+job_cancel_sync_locked(&state->job->job, true);
 }
 
 aio_context_release(aio_context);
@@ -2396,7 +2396,7 @@ exit:
 }
 
 WITH_JOB_LOCK_GUARD() {
-job_txn_unref(block_job_txn);
+job_txn_unref_locked(block_job_txn);
 }
 }
 
@@ -3371,14 +3371,14 @@ void qmp_block_job_cancel(const char *device,
 force = false;
 }
 
-if (job_user_paused(&job->job) && !force) {
+if (job_user_paused_locked(&job->job) && !force) {
 error_setg(errp, "The block job for device '%s' is currently paused",
device);
 goto out;
 }
 
 trace_qmp_block_job_cancel(job);
-job_user_cancel(&job->job, force, errp);
+job_user_cancel_locked(&job->job, force, errp);
 out:
 aio_context_release(aio_context);
 }
@@ -3396,7 +3396,7 @@ void qmp_block_job_pause(const char *device, Error **errp)
 }
 
 trace_qmp_block_job_pause(job);
-job_user_pause(&job->j

[PATCH v6 09/18] block/mirror.c: use of job helpers in drivers to avoid TOC/TOU

[Emanuele Giuseppe Esposito]

Once job lock is used and aiocontext is removed, mirror has
to perform job operations under the same critical section,
using the helpers prepared in previous commit.

Note: at this stage, job_{lock/unlock} and job lock guard macros
are *nop*.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Stefan Hajnoczi 
---
 block/mirror.c | 19 ++-
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/block/mirror.c b/block/mirror.c
index ce6bc58d1f..b3b64a280a 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -653,9 +653,13 @@ static int mirror_exit_common(Job *job)
 BlockDriverState *target_bs;
 BlockDriverState *mirror_top_bs;
 Error *local_err = NULL;
-bool abort = job->ret < 0;
+bool abort;
 int ret = 0;
 
+WITH_JOB_LOCK_GUARD() {
+abort = job->ret < 0;
+}
+
 if (s->prepared) {
 return 0;
 }
@@ -1151,8 +1155,10 @@ static void mirror_complete(Job *job, Error **errp)
 s->should_complete = true;
 
 /* If the job is paused, it will be re-entered when it is resumed */
-if (!job->paused) {
-job_enter(job);
+WITH_JOB_LOCK_GUARD() {
+if (!job->paused) {
+job_enter_cond(job, NULL);
+}
 }
 }
 
@@ -1172,8 +1178,11 @@ static bool mirror_drained_poll(BlockJob *job)
  * from one of our own drain sections, to avoid a deadlock waiting for
  * ourselves.
  */
-if (!s->common.job.paused && !job_is_cancelled(&job->job) && !s->in_drain) 
{
-return true;
+WITH_JOB_LOCK_GUARD() {
+if (!s->common.job.paused && !job_is_cancelled_locked(&job->job)
+&& !s->in_drain) {
+return true;
+}
 }
 
 return !!s->in_flight;
-- 
2.31.1

[PATCH v2 03/10] block/io.c: fix bdrv_child_cb_drained_begin invocations from a coroutine

[Emanuele Giuseppe Esposito]

Using bdrv_do_drained_begin_quiesce() in bdrv_child_cb_drained_begin()
is not a good idea: the callback might be called when running
a drain in a coroutine, and bdrv_drained_begin_poll() does not
handle that case, resulting in assertion failure.

Instead, bdrv_do_drained_begin with no recursion and poll
will accomplish the same thing (invoking bdrv_do_drained_begin_quiesce)
but will firstly check if we are already in a coroutine, and exit
from that via bdrv_co_yield_to_drain().

Signed-off-by: Emanuele Giuseppe Esposito 
---
 block.c  |  2 +-
 block/io.c   |  8 +++-
 include/block/block-io.h | 20 +++-
 3 files changed, 19 insertions(+), 11 deletions(-)

diff --git a/block.c b/block.c
index 718e4cae8b..372f16f4a0 100644
--- a/block.c
+++ b/block.c
@@ -1211,7 +1211,7 @@ static char *bdrv_child_get_parent_desc(BdrvChild *c)
 static void bdrv_child_cb_drained_begin(BdrvChild *child)
 {
 BlockDriverState *bs = child->opaque;
-bdrv_do_drained_begin_quiesce(bs, NULL, false);
+bdrv_drained_begin_no_poll(bs);
 }
 
 static bool bdrv_child_cb_drained_poll(BdrvChild *child)
diff --git a/block/io.c b/block/io.c
index e446782ae0..e77861c464 100644
--- a/block/io.c
+++ b/block/io.c
@@ -523,7 +523,7 @@ static void coroutine_fn 
bdrv_co_yield_to_drain(BlockDriverState *bs,
 }
 }
 
-void bdrv_do_drained_begin_quiesce(BlockDriverState *bs,
+static void bdrv_do_drained_begin_quiesce(BlockDriverState *bs,
BdrvChild *parent, bool ignore_bds_parents)
 {
 IO_OR_GS_CODE();
@@ -587,6 +587,12 @@ void bdrv_subtree_drained_begin(BlockDriverState *bs)
 bdrv_do_drained_begin(bs, true, NULL, false, true);
 }
 
+void bdrv_drained_begin_no_poll(BlockDriverState *bs)
+{
+IO_CODE();
+bdrv_do_drained_begin(bs, false, NULL, false, false);
+}
+
 /**
  * This function does not poll, nor must any of its recursively called
  * functions.  The *drained_end_counter pointee will be incremented
diff --git a/include/block/block-io.h b/include/block/block-io.h
index 5e3f346806..9135b648bf 100644
--- a/include/block/block-io.h
+++ b/include/block/block-io.h
@@ -226,6 +226,17 @@ int coroutine_fn bdrv_co_copy_range(BdrvChild *src, 
int64_t src_offset,
 int64_t bytes, BdrvRequestFlags read_flags,
 BdrvRequestFlags write_flags);
 
+/**
+ * bdrv_drained_begin_no_poll:
+ *
+ * Quiesces a BDS like bdrv_drained_begin(), but does not wait for already
+ * running requests to complete.
+ * Same as bdrv_drained_begin(), but do not poll for the subgraph to
+ * actually become unquiesced. Therefore, no graph changes will occur
+ * with this function.
+ */
+void bdrv_drained_begin_no_poll(BlockDriverState *bs);
+
 /**
  * bdrv_drained_end_no_poll:
  *
@@ -332,15 +343,6 @@ bool bdrv_drain_poll(BlockDriverState *bs, bool recursive,
  */
 void bdrv_drained_begin(BlockDriverState *bs);
 
-/**
- * bdrv_do_drained_begin_quiesce:
- *
- * Quiesces a BDS like bdrv_drained_begin(), but does not wait for already
- * running requests to complete.
- */
-void bdrv_do_drained_begin_quiesce(BlockDriverState *bs,
-   BdrvChild *parent, bool ignore_bds_parents);
-
 /**
  * Like bdrv_drained_begin, but recursively begins a quiesced section for
  * exclusive access to all child nodes as well.
-- 
2.31.1

[PATCH v6 15/18] job: detect change of aiocontext within job coroutine

[Emanuele Giuseppe Esposito]

From: Paolo Bonzini 

We want to make sure access of job->aio_context is always done
under either BQL or job_mutex. The problem is that using
aio_co_enter(job->aiocontext, job->co) in job_start and job_enter_cond
makes the coroutine immediately resume, so we can't hold the job lock.
And caching it is not safe either, as it might change.

job_start is under BQL, so it can freely read job->aiocontext, but
job_enter_cond is not. In order to fix this, use aio_co_wake():
the advantage is that it won't use job->aiocontext, but the
main disadvantage is that it won't be able to detect a change of
job AioContext.

Calling bdrv_try_set_aio_context() will issue the following calls
(simplified):
* in terms of  bdrv callbacks:
  .drained_begin -> .set_aio_context -> .drained_end
* in terms of child_job functions:
  child_job_drained_begin -> child_job_set_aio_context -> child_job_drained_end
* in terms of job functions:
  job_pause_locked -> job_set_aio_context -> job_resume_locked

We can see that after setting the new aio_context, job_resume_locked
calls again job_enter_cond, which then invokes aio_co_wake(). But
while job->aiocontext has been set in job_set_aio_context,
job->co->ctx has not changed, so the coroutine would be entering in
the wrong aiocontext.

Using aio_co_schedule in job_resume_locked() might seem as a valid
alternative, but the problem is that the bh resuming the coroutine
is not scheduled immediately, and if in the meanwhile another
bdrv_try_set_aio_context() is run (see test_propagate_mirror() in
test-block-iothread.c), we would have the first schedule in the
wrong aiocontext, and the second set of drains won't even manage
to schedule the coroutine, as job->busy would still be true from
the previous job_resume_locked().

The solution is to stick with aio_co_wake(), but then detect every time
the coroutine resumes back from yielding if job->aio_context
has changed. If so, we can reschedule it to the new context.

Check for the aiocontext change in job_do_yield_locked because:
1) aio_co_reschedule_self requires to be in the running coroutine
2) since child_job_set_aio_context allows changing the aiocontext only
   while the job is paused, this is the exact place where the coroutine
   resumes, before running JobDriver's code.

Reviewed-by: Stefan Hajnoczi 
Signed-off-by: Paolo Bonzini 
---
 job.c | 24 +---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/job.c b/job.c
index 89c0e6bed9..10a5981748 100644
--- a/job.c
+++ b/job.c
@@ -543,11 +543,12 @@ void job_enter_cond_locked(Job *job, bool(*fn)(Job *job))
 return;
 }
 
-assert(!job->deferred_to_main_loop);
 timer_del(&job->sleep_timer);
 job->busy = true;
 real_job_unlock();
-aio_co_enter(job->aio_context, job->co);
+job_unlock();
+aio_co_wake(job->co);
+job_lock();
 }
 
 void job_enter(Job *job)
@@ -568,6 +569,8 @@ void job_enter(Job *job)
  */
 static void coroutine_fn job_do_yield_locked(Job *job, uint64_t ns)
 {
+AioContext *next_aio_context;
+
 real_job_lock();
 if (ns != -1) {
 timer_mod(&job->sleep_timer, ns);
@@ -579,6 +582,20 @@ static void coroutine_fn job_do_yield_locked(Job *job, 
uint64_t ns)
 qemu_coroutine_yield();
 job_lock();
 
+next_aio_context = job->aio_context;
+/*
+ * Coroutine has resumed, but in the meanwhile the job AioContext
+ * might have changed via bdrv_try_set_aio_context(), so we need to move
+ * the coroutine too in the new aiocontext.
+ */
+while (qemu_get_current_aio_context() != next_aio_context) {
+job_unlock();
+aio_co_reschedule_self(next_aio_context);
+job_lock();
+next_aio_context = job->aio_context;
+}
+
+
 /* Set by job_enter_cond_locked() before re-entering the coroutine.  */
 assert(job->busy);
 }
@@ -680,7 +697,6 @@ void job_resume_locked(Job *job)
 if (job->pause_count) {
 return;
 }
-
 /* kick only if no timer is pending */
 job_enter_cond_locked(job, job_timer_not_pending_locked);
 }
@@ -1122,6 +1138,8 @@ static void coroutine_fn job_co_entry(void *opaque)
 
 void job_start(Job *job)
 {
+assert(qemu_in_main_thread());
+
 WITH_JOB_LOCK_GUARD() {
 assert(job && !job_started(job) && job->paused &&
 job->driver && job->driver->run);
-- 
2.31.1

[PATCH v6 16/18] jobs: protect job.aio_context with BQL and job_mutex

[Emanuele Giuseppe Esposito]

In order to make it thread safe, implement a "fake rwlock",
where we allow reads under BQL *or* job_mutex held, but
writes only under BQL *and* job_mutex.

The only write we have is in child_job_set_aio_ctx, which always
happens under drain (so the job is paused).
For this reason, introduce job_set_aio_context and make sure that
the context is set under BQL, job_mutex and drain.
Also make sure all other places where the aiocontext is read
are protected.

Note: at this stage, job_{lock/unlock} and job lock guard macros
are *nop*.

Suggested-by: Paolo Bonzini 
Signed-off-by: Emanuele Giuseppe Esposito 
---
 block/replication.c |  2 +-
 blockjob.c  |  3 ++-
 include/qemu/job.h  | 19 ++-
 job.c   | 12 
 4 files changed, 33 insertions(+), 3 deletions(-)

diff --git a/block/replication.c b/block/replication.c
index 50ea778937..68018948b9 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -148,8 +148,8 @@ static void replication_close(BlockDriverState *bs)
 }
 if (s->stage == BLOCK_REPLICATION_FAILOVER) {
 commit_job = &s->commit_job->job;
-assert(commit_job->aio_context == qemu_get_current_aio_context());
 WITH_JOB_LOCK_GUARD() {
+assert(commit_job->aio_context == qemu_get_current_aio_context());
 job_cancel_sync_locked(commit_job, false);
 }
 }
diff --git a/blockjob.c b/blockjob.c
index 2c075db45b..8b9e10813d 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -154,12 +154,13 @@ static void child_job_set_aio_ctx(BdrvChild *c, 
AioContext *ctx,
 bdrv_set_aio_context_ignore(sibling->bs, ctx, ignore);
 }
 
-job->job.aio_context = ctx;
+job_set_aio_context(&job->job, ctx);
 }
 
 static AioContext *child_job_get_parent_aio_context(BdrvChild *c)
 {
 BlockJob *job = c->opaque;
+assert(qemu_in_main_thread());
 
 return job->job.aio_context;
 }
diff --git a/include/qemu/job.h b/include/qemu/job.h
index dfbf2ea501..fad4ee8cb3 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -75,7 +75,12 @@ typedef struct Job {
 ProgressMeter progress;
 
 
-/** AioContext to run the job coroutine in */
+/**
+ * AioContext to run the job coroutine in.
+ * This field can be read when holding either the BQL (so we are in
+ * the main loop) or the job_mutex.
+ * It can be only written when we hold *both* BQL and job_mutex.
+ */
 AioContext *aio_context;
 
 /** Reference count of the block job */
@@ -706,4 +711,16 @@ void job_dismiss_locked(Job **job, Error **errp);
 int job_finish_sync_locked(Job *job, void (*finish)(Job *, Error **errp),
Error **errp);
 
+/**
+ * Sets the @job->aio_context.
+ * Called with job_mutex *not* held.
+ *
+ * This function must run in the main thread to protect against
+ * concurrent read in job_finish_sync_locked(),
+ * takes the job_mutex lock to protect against the read in
+ * job_do_yield_locked(), and must be called when the coroutine
+ * is quiescent.
+ */
+void job_set_aio_context(Job *job, AioContext *ctx);
+
 #endif
diff --git a/job.c b/job.c
index 10a5981748..80a0fb34b1 100644
--- a/job.c
+++ b/job.c
@@ -368,6 +368,17 @@ Job *job_get_locked(const char *id)
 return NULL;
 }
 
+void job_set_aio_context(Job *job, AioContext *ctx)
+{
+/* protect against read in job_finish_sync_locked and job_start */
+assert(qemu_in_main_thread());
+/* protect against read in job_do_yield_locked */
+JOB_LOCK_GUARD();
+/* ensure the coroutine is quiescent while the AioContext is changed */
+assert(job->pause_count > 0);
+job->aio_context = ctx;
+}
+
 /* Called with job_mutex *not* held. */
 static void job_sleep_timer_cb(void *opaque)
 {
@@ -1261,6 +1272,7 @@ int job_finish_sync_locked(Job *job, void (*finish)(Job 
*, Error **errp),
 {
 Error *local_err = NULL;
 int ret;
+assert(qemu_in_main_thread());
 
 job_ref_locked(job);
 
-- 
2.31.1

[PATCH v2 00/10] block: bug fixes in preparation of AioContext removal

[Emanuele Giuseppe Esposito]

This serie aims to remove and clean up some bugs that came up
when trying to replace the AioContext lock and still protect
BlockDriverState fields.

They were part of the serie "Removal of Aiocontext lock
through drains: protect bdrv_replace_child_noperm", but since
that serie is still a work in progress and these fixes are
pretty much independent, I split that in two separate series.

---
v2:
* change comments in patch 2 and 3, .attach() and .detach() callbacks.
* remove job_sleep_ns patch, as it was causing random deadlocks in test 030

Emanuele Giuseppe Esposito (10):
  drains: create bh only when polling
  bdrv_parent_drained_begin_single: handle calls from coroutine context
  block/io.c: fix bdrv_child_cb_drained_begin invocations from a
coroutine
  block.c: bdrv_replace_child_noperm: first remove the child, and then
call ->detach()
  block.c: bdrv_replace_child_noperm: first call ->attach(), and then
add child
  test-bdrv-drain.c: adapt test to the coming subtree drains
  test-bdrv-drain.c: remove test_detach_by_parent_cb()
  tests/unit/test-bdrv-drain.c: graph setup functions can't run in
coroutines
  child_job_drained_poll: override polling condition only when in home
thread
  tests/qemu-iotests/030: test_stream_parallel should use
auto_finalize=False

 block.c  |  41 ++---
 block/io.c   | 137 +---
 blockjob.c   |  13 ++-
 include/block/block-io.h |  20 +++--
 tests/qemu-iotests/030   |  12 +--
 tests/unit/test-bdrv-drain.c | 169 +++
 6 files changed, 267 insertions(+), 125 deletions(-)

-- 
2.31.1

[PATCH v6 01/18] job.c: make job_mutex and job_lock/unlock() public

[Emanuele Giuseppe Esposito]

job mutex will be used to protect the job struct elements and list,
replacing AioContext locks.

Right now use a shared lock for all jobs, in order to keep things
simple. Once the AioContext lock is gone, we can introduce per-job
locks.

To simplify the switch from aiocontext to job lock, introduce
*nop* lock/unlock functions and macros.
We want to always call job_lock/unlock outside the AioContext locks,
and not vice-versa, otherwise we might get a deadlock. This is not
straightforward to do, and that's why we start with nop functions.
Once everything is protected by job_lock/unlock, we can change the nop into
an actual mutex and remove the aiocontext lock.

Since job_mutex is already being used, add static
real_job_{lock/unlock} for the existing usage.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Stefan Hajnoczi 
---
 include/qemu/job.h | 24 
 job.c  | 35 +++
 2 files changed, 47 insertions(+), 12 deletions(-)

diff --git a/include/qemu/job.h b/include/qemu/job.h
index c105b31076..d1192ffd61 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -303,6 +303,30 @@ typedef enum JobCreateFlags {
 JOB_MANUAL_DISMISS = 0x04,
 } JobCreateFlags;
 
+extern QemuMutex job_mutex;
+
+#define JOB_LOCK_GUARD() /* QEMU_LOCK_GUARD(&job_mutex) */
+
+#define WITH_JOB_LOCK_GUARD() /* WITH_QEMU_LOCK_GUARD(&job_mutex) */
+
+/**
+ * job_lock:
+ *
+ * Take the mutex protecting the list of jobs and their status.
+ * Most functions called by the monitor need to call job_lock
+ * and job_unlock manually.  On the other hand, function called
+ * by the block jobs themselves and by the block layer will take the
+ * lock for you.
+ */
+void job_lock(void);
+
+/**
+ * job_unlock:
+ *
+ * Release the mutex protecting the list of jobs and their status.
+ */
+void job_unlock(void);
+
 /**
  * Allocate and return a new job transaction. Jobs can be added to the
  * transaction using job_txn_add_job().
diff --git a/job.c b/job.c
index 075c6f3a20..2b4ffca9d4 100644
--- a/job.c
+++ b/job.c
@@ -32,6 +32,12 @@
 #include "trace/trace-root.h"
 #include "qapi/qapi-events-job.h"
 
+/*
+ * job_mutex protects the jobs list, but also makes the
+ * struct job fields thread-safe.
+ */
+QemuMutex job_mutex;
+
 static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs);
 
 /* Job State Transition Table */
@@ -74,17 +80,22 @@ struct JobTxn {
 int refcnt;
 };
 
-/* Right now, this mutex is only needed to synchronize accesses to job->busy
- * and job->sleep_timer, such as concurrent calls to job_do_yield and
- * job_enter. */
-static QemuMutex job_mutex;
+void job_lock(void)
+{
+/* nop */
+}
+
+void job_unlock(void)
+{
+/* nop */
+}
 
-static void job_lock(void)
+static void real_job_lock(void)
 {
 qemu_mutex_lock(&job_mutex);
 }
 
-static void job_unlock(void)
+static void real_job_unlock(void)
 {
 qemu_mutex_unlock(&job_mutex);
 }
@@ -450,21 +461,21 @@ void job_enter_cond(Job *job, bool(*fn)(Job *job))
 return;
 }
 
-job_lock();
+real_job_lock();
 if (job->busy) {
-job_unlock();
+real_job_unlock();
 return;
 }
 
 if (fn && !fn(job)) {
-job_unlock();
+real_job_unlock();
 return;
 }
 
 assert(!job->deferred_to_main_loop);
 timer_del(&job->sleep_timer);
 job->busy = true;
-job_unlock();
+real_job_unlock();
 aio_co_enter(job->aio_context, job->co);
 }
 
@@ -481,13 +492,13 @@ void job_enter(Job *job)
  * called explicitly. */
 static void coroutine_fn job_do_yield(Job *job, uint64_t ns)
 {
-job_lock();
+real_job_lock();
 if (ns != -1) {
 timer_mod(&job->sleep_timer, ns);
 }
 job->busy = false;
 job_event_idle(job);
-job_unlock();
+real_job_unlock();
 qemu_coroutine_yield();
 
 /* Set by job_enter_cond() before re-entering the coroutine.  */
-- 
2.31.1

[PATCH v6 13/18] job.h: define unlocked functions

[Emanuele Giuseppe Esposito]

All these functions assume that the lock is not held, and acquire
it internally.

These functions will be useful when job_lock is globally applied,
as they will allow callers to access the job struct fields
without worrying about the job lock.

Update also the comments in blockjob.c (and move them in job.c).

Note: at this stage, job_{lock/unlock} and job lock guard macros
are *nop*.

No functional change intended.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 blockjob.c | 20 
 include/qemu/job.h | 37 ++---
 job.c  | 16 
 3 files changed, 50 insertions(+), 23 deletions(-)

diff --git a/blockjob.c b/blockjob.c
index 0745f4e745..2c075db45b 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -36,21 +36,6 @@
 #include "qemu/main-loop.h"
 #include "qemu/timer.h"
 
-/*
- * The block job API is composed of two categories of functions.
- *
- * The first includes functions used by the monitor.  The monitor is
- * peculiar in that it accesses the block job list with block_job_get, and
- * therefore needs consistency across block_job_get and the actual operation
- * (e.g. block_job_set_speed).  The consistency is achieved with
- * aio_context_acquire/release.  These functions are declared in blockjob.h.
- *
- * The second includes functions used by the block job drivers and sometimes
- * by the core block layer.  These do not care about locking, because the
- * whole coroutine runs under the AioContext lock, and are declared in
- * blockjob_int.h.
- */
-
 static bool is_block_job(Job *job)
 {
 return job_type(job) == JOB_TYPE_BACKUP ||
@@ -446,11 +431,6 @@ static void block_job_event_ready_locked(Notifier *n, void 
*opaque)
 }
 
 
-/*
- * API for block job drivers and the block layer.  These functions are
- * declared in blockjob_int.h.
- */
-
 void *block_job_create(const char *job_id, const BlockJobDriver *driver,
JobTxn *txn, BlockDriverState *bs, uint64_t perm,
uint64_t shared_perm, int64_t speed, int flags,
diff --git a/include/qemu/job.h b/include/qemu/job.h
index dfd2586e63..dfbf2ea501 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -360,6 +360,7 @@ void job_txn_unref_locked(JobTxn *txn);
 
 /**
  * Create a new long-running job and return it.
+ * Called with job_mutex *not* held.
  *
  * @job_id: The id of the newly-created job, or %NULL for internal jobs
  * @driver: The class object for the newly-created job.
@@ -395,6 +396,8 @@ void job_unref_locked(Job *job);
  * @done: How much progress the job made since the last call
  *
  * Updates the progress counter of the job.
+ *
+ * Progress API is thread safe.
  */
 void job_progress_update(Job *job, uint64_t done);
 
@@ -405,6 +408,8 @@ void job_progress_update(Job *job, uint64_t done);
  *
  * Sets the expected end value of the progress counter of a job so that a
  * completion percentage can be calculated when the progress is updated.
+ *
+ * Progress API is thread safe.
  */
 void job_progress_set_remaining(Job *job, uint64_t remaining);
 
@@ -420,6 +425,8 @@ void job_progress_set_remaining(Job *job, uint64_t 
remaining);
  * length before, and job_progress_update() afterwards.
  * (So the operation acts as a parenthesis in regards to the main job
  * operation running in background.)
+ *
+ * Progress API is thread safe.
  */
 void job_progress_increase_remaining(Job *job, uint64_t delta);
 
@@ -437,13 +444,17 @@ void job_enter_cond_locked(Job *job, bool(*fn)(Job *job));
  *
  * Begins execution of a job.
  * Takes ownership of one reference to the job object.
+ *
+ * Called with job_mutex *not* held.
  */
 void job_start(Job *job);
 
 /**
  * @job: The job to enter.
+ * Called with job_mutex *not* held.
  *
  * Continue the specified job by entering the coroutine.
+ * Called with job_mutex lock *not* held.
  */
 void job_enter(Job *job);
 
@@ -452,6 +463,9 @@ void job_enter(Job *job);
  *
  * Pause now if job_pause_locked() has been called. Jobs that perform lots of
  * I/O must call this between requests so that the job can be paused.
+ *
+ * Called with job_mutex *not* held (we don't want the coroutine
+ * to yield with the lock held!).
  */
 void coroutine_fn job_pause_point(Job *job);
 
@@ -459,6 +473,8 @@ void coroutine_fn job_pause_point(Job *job);
  * @job: The job that calls the function.
  *
  * Yield the job coroutine.
+ * Called with job_mutex *not* held (we don't want the coroutine
+ * to yield with the lock held!).
  */
 void job_yield(Job *job);
 
@@ -469,6 +485,9 @@ void job_yield(Job *job);
  * Put the job to sleep (assuming that it wasn't canceled) for @ns
  * %QEMU_CLOCK_REALTIME nanoseconds.  Canceling the job will immediately
  * interrupt the wait.
+ *
+ * Called with job_mutex *not* held (we don't want the coroutine
+ * to yield with the lock held!).
  */
 void coroutine_fn job_sleep_ns(Job *job, int64_t ns);
 
@@ -580,10 +599,16 @@ Job *job_get_locked(const char *id);
  */
 int job_apply_v

[PATCH v2 01/10] drains: create bh only when polling

[Emanuele Giuseppe Esposito]

We need to prevent coroutines from calling BDRV_POLL_WHILE, because
it can create deadlocks. This is done by firstly creating a bottom half
and then yielding. The bh is then scheduled in the main loop, performs
the drain and polling, and then resumes the coroutine.

The problem is that currently we create coroutine and bh regardless
on whether we eventually poll or not. There is no need to do so,
if no poll takes place.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 block/io.c | 31 +++
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/block/io.c b/block/io.c
index efc011ce65..4a3e8d037d 100644
--- a/block/io.c
+++ b/block/io.c
@@ -447,7 +447,7 @@ static void bdrv_do_drained_begin(BlockDriverState *bs, 
bool recursive,
 {
 BdrvChild *child, *next;
 
-if (qemu_in_coroutine()) {
+if (poll && qemu_in_coroutine()) {
 bdrv_co_yield_to_drain(bs, true, recursive, parent, ignore_bds_parents,
poll, NULL);
 return;
@@ -513,12 +513,6 @@ static void bdrv_do_drained_end(BlockDriverState *bs, bool 
recursive,
 int old_quiesce_counter;
 
 assert(drained_end_counter != NULL);
-
-if (qemu_in_coroutine()) {
-bdrv_co_yield_to_drain(bs, false, recursive, parent, 
ignore_bds_parents,
-   false, drained_end_counter);
-return;
-}
 assert(bs->quiesce_counter > 0);
 
 /* Re-enable things in child-to-parent order */
@@ -541,11 +535,24 @@ static void bdrv_do_drained_end(BlockDriverState *bs, 
bool recursive,
 }
 }
 
+static void bdrv_do_drained_end_co(BlockDriverState *bs, bool recursive,
+   BdrvChild *parent, bool ignore_bds_parents,
+   int *drained_end_counter)
+{
+if (qemu_in_coroutine()) {
+bdrv_co_yield_to_drain(bs, false, recursive, parent, 
ignore_bds_parents,
+   false, drained_end_counter);
+return;
+}
+
+bdrv_do_drained_end(bs, recursive, parent, ignore_bds_parents, 
drained_end_counter);
+}
+
 void bdrv_drained_end(BlockDriverState *bs)
 {
 int drained_end_counter = 0;
 IO_OR_GS_CODE();
-bdrv_do_drained_end(bs, false, NULL, false, &drained_end_counter);
+bdrv_do_drained_end_co(bs, false, NULL, false, &drained_end_counter);
 BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
 }
 
@@ -559,7 +566,7 @@ void bdrv_subtree_drained_end(BlockDriverState *bs)
 {
 int drained_end_counter = 0;
 IO_OR_GS_CODE();
-bdrv_do_drained_end(bs, true, NULL, false, &drained_end_counter);
+bdrv_do_drained_end_co(bs, true, NULL, false, &drained_end_counter);
 BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
 }
 
@@ -580,7 +587,7 @@ void bdrv_unapply_subtree_drain(BdrvChild *child, 
BlockDriverState *old_parent)
 IO_OR_GS_CODE();
 
 for (i = 0; i < old_parent->recursive_quiesce_counter; i++) {
-bdrv_do_drained_end(child->bs, true, child, false,
+bdrv_do_drained_end_co(child->bs, true, child, false,
 &drained_end_counter);
 }
 
@@ -703,7 +710,7 @@ void bdrv_drain_all_end_quiesce(BlockDriverState *bs)
 g_assert(!bs->refcnt);
 
 while (bs->quiesce_counter) {
-bdrv_do_drained_end(bs, false, NULL, true, &drained_end_counter);
+bdrv_do_drained_end_co(bs, false, NULL, true, &drained_end_counter);
 }
 BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
 }
@@ -727,7 +734,7 @@ void bdrv_drain_all_end(void)
 AioContext *aio_context = bdrv_get_aio_context(bs);
 
 aio_context_acquire(aio_context);
-bdrv_do_drained_end(bs, false, NULL, true, &drained_end_counter);
+bdrv_do_drained_end_co(bs, false, NULL, true, &drained_end_counter);
 aio_context_release(aio_context);
 }
 
-- 
2.31.1

[PATCH v6 10/18] jobs: rename static functions called with job_mutex held

[Emanuele Giuseppe Esposito]

With the *nop* job_lock/unlock placed, rename the static
functions that are always under job_mutex, adding "_locked" suffix.

List of functions that get this suffix:
job_txn_refjob_txn_del_job
job_txn_apply  job_state_transition
job_should_pause   job_event_cancelled
job_event_completedjob_event_pending
job_event_readyjob_event_idle
job_do_yield   job_timer_not_pending
job_do_dismiss job_conclude
job_update_rc  job_commit
job_abort  job_clean
job_finalize_singlejob_cancel_async
job_completed_txn_abortjob_prepare
job_needs_finalize job_do_finalize
job_transition_to_pending  job_completed_txn_success
job_completed  job_cancel_err
job_force_cancel_err

Note that "locked" refers to the *nop* job_lock/unlock, and not
real_job_lock/unlock.

No functional change intended.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 job.c | 247 +-
 1 file changed, 141 insertions(+), 106 deletions(-)

diff --git a/job.c b/job.c
index 3e9e632a18..298432ce00 100644
--- a/job.c
+++ b/job.c
@@ -113,7 +113,8 @@ JobTxn *job_txn_new(void)
 return txn;
 }
 
-static void job_txn_ref(JobTxn *txn)
+/* Called with job_mutex held. */
+static void job_txn_ref_locked(JobTxn *txn)
 {
 txn->refcnt++;
 }
@@ -145,10 +146,11 @@ static void job_txn_add_job(JobTxn *txn, Job *job)
 job->txn = txn;
 
 QLIST_INSERT_HEAD(&txn->jobs, job, txn_list);
-job_txn_ref(txn);
+job_txn_ref_locked(txn);
 }
 
-static void job_txn_del_job(Job *job)
+/* Called with job_mutex held. */
+static void job_txn_del_job_locked(Job *job)
 {
 if (job->txn) {
 QLIST_REMOVE(job, txn_list);
@@ -157,7 +159,8 @@ static void job_txn_del_job(Job *job)
 }
 }
 
-static int job_txn_apply(Job *job, int fn(Job *))
+/* Called with job_mutex held. */
+static int job_txn_apply_locked(Job *job, int fn(Job *))
 {
 AioContext *inner_ctx;
 Job *other_job, *next;
@@ -165,10 +168,10 @@ static int job_txn_apply(Job *job, int fn(Job *))
 int rc = 0;
 
 /*
- * Similar to job_completed_txn_abort, we take each job's lock before
- * applying fn, but since we assume that outer_ctx is held by the caller,
- * we need to release it here to avoid holding the lock twice - which would
- * break AIO_WAIT_WHILE from within fn.
+ * Similar to job_completed_txn_abort_locked, we take each job's lock
+ * before applying fn, but since we assume that outer_ctx is held by
+ * the caller, we need to release it here to avoid holding the lock
+ * twice - which would break AIO_WAIT_WHILE from within fn.
  */
 job_ref(job);
 aio_context_release(job->aio_context);
@@ -197,7 +200,8 @@ bool job_is_internal(Job *job)
 return (job->id == NULL);
 }
 
-static void job_state_transition(Job *job, JobStatus s1)
+/* Called with job_mutex held. */
+static void job_state_transition_locked(Job *job, JobStatus s1)
 {
 JobStatus s0 = job->status;
 assert(s1 >= 0 && s1 < JOB_STATUS__MAX);
@@ -322,7 +326,8 @@ static bool job_started(Job *job)
 return job->co;
 }
 
-static bool job_should_pause(Job *job)
+/* Called with job_mutex held. */
+static bool job_should_pause_locked(Job *job)
 {
 return job->pause_count > 0;
 }
@@ -402,7 +407,7 @@ void *job_create(const char *job_id, const JobDriver 
*driver, JobTxn *txn,
 notifier_list_init(&job->on_ready);
 notifier_list_init(&job->on_idle);
 
-job_state_transition(job, JOB_STATUS_CREATED);
+job_state_transition_locked(job, JOB_STATUS_CREATED);
 aio_timer_init(qemu_get_aio_context(), &job->sleep_timer,
QEMU_CLOCK_REALTIME, SCALE_NS,
job_sleep_timer_cb, job);
@@ -468,31 +473,36 @@ void job_progress_increase_remaining(Job *job, uint64_t 
delta)
 
 /**
  * To be called when a cancelled job is finalised.
+ * Called with job_mutex held.
  */
-static void job_event_cancelled(Job *job)
+static void job_event_cancelled_locked(Job *job)
 {
 notifier_list_notify(&job->on_finalize_cancelled, job);
 }
 
 /**
  * To be called when a successfully completed job is finalised.
+ * Called with job_mutex held.
  */
-static void job_event_completed(Job *job)
+static void job_event_completed_locked(Job *job)
 {
 notifier_list_notify(&job->on_finalize_completed, job);
 }
 
-static void job_event_pending(Job *job)
+/* Called with job_mutex held. */
+static void job_event_pending_locked(Job *job)
 {
 notifier_list_notify(&job->on_pending, job);
 }
 
-static void job_event_ready(Job *job)
+/* Called with job_mutex held. */
+static void job_event_ready_locked(Job *job)
 {
 notifier_list_notify(&job->on_ready, job);
 }
 
-static void job_event_idle(Job *job)
+/* Called with job_mutex held. */
+static void job_event_idle_locked(Job *job)
 {
 notifier_list_notify(&job->on_idle, job);
 }
@@ -530,20 +540,24 @@ void job_enter(Job *job)

Re: [PATCH experiment 00/35] stackless coroutine backend

[Stefan Hajnoczi]

On Sun, Mar 13, 2022 at 04:18:40PM +0100, Paolo Bonzini wrote:
> On 3/11/22 13:17, Daniel P. Berrangé wrote:
> The question is what is easier to maintain, stack switching code that is
> becoming less and less portable (status quo with SafeStack, CET and the TLS
> issues that Stefan has worked on), a mixed C/C++ codebase (C++ coroutines),
> a custom source-to-source translator (this series).  The third might be more
> fun, but it would be quite a large enterprise and the C++ compiler writers
> have already done the work.

Or a C-to-C++ translator to keep the code in C but still use C++
coroutines :). (I'm joking.)

Stefan


signature.asc
Description: PGP signature

[PATCH v2 08/10] tests/unit/test-bdrv-drain.c: graph setup functions can't run in coroutines

[Emanuele Giuseppe Esposito]

Graph initialization functions like blk_new(), bdrv_new() and so on
should not run in a coroutine. In fact, they might invoke a drain
(for example blk_insert_bs eventually calls bdrv_replace_child_noperm)
that in turn can invoke callbacks like bdrv_do_drained_begin_quiesce(),
that asserts exactly that we are not in a coroutine.

Move the initialization phase of test_drv_cb and test_quiesce_common
outside the coroutine logic.

Signed-off-by: Emanuele Giuseppe Esposito 
Message-Id: <20211213104014.69858-2-eespo...@redhat.com>
---
 tests/unit/test-bdrv-drain.c | 118 ++-
 1 file changed, 73 insertions(+), 45 deletions(-)

diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c
index fa0d86209a..a3bc19965d 100644
--- a/tests/unit/test-bdrv-drain.c
+++ b/tests/unit/test-bdrv-drain.c
@@ -116,7 +116,8 @@ static void aio_ret_cb(void *opaque, int ret)
 }
 
 typedef struct CallInCoroutineData {
-void (*entry)(void);
+void (*entry)(void *);
+void *arg;
 bool done;
 } CallInCoroutineData;
 
@@ -124,15 +125,16 @@ static coroutine_fn void call_in_coroutine_entry(void 
*opaque)
 {
 CallInCoroutineData *data = opaque;
 
-data->entry();
+data->entry(data->arg);
 data->done = true;
 }
 
-static void call_in_coroutine(void (*entry)(void))
+static void call_in_coroutine(void (*entry)(void *), void *arg)
 {
 Coroutine *co;
 CallInCoroutineData data = {
 .entry  = entry,
+.arg= arg,
 .done   = false,
 };
 
@@ -192,26 +194,28 @@ static void do_drain_end_unlocked(enum drain_type 
drain_type, BlockDriverState *
 }
 }
 
-static void test_drv_cb_common(enum drain_type drain_type, bool recursive)
-{
+typedef struct TestDriverCBData {
+enum drain_type drain_type;
+bool recursive;
 BlockBackend *blk;
 BlockDriverState *bs, *backing;
-BDRVTestState *s, *backing_s;
+} TestDriverCBData;
+
+static void test_drv_cb_common(void *arg)
+{
+TestDriverCBData *data = arg;
+BlockBackend *blk = data->blk;
+BlockDriverState *bs = data->bs;
+BlockDriverState *backing = data->backing;
+enum drain_type drain_type = data->drain_type;
+bool recursive = data->recursive;
+BDRVTestState *s = bs->opaque;
+BDRVTestState *backing_s = backing->opaque;
 BlockAIOCB *acb;
 int aio_ret;
 
 QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, NULL, 0);
 
-blk = blk_new(qemu_get_aio_context(), BLK_PERM_ALL, BLK_PERM_ALL);
-bs = bdrv_new_open_driver(&bdrv_test, "test-node", BDRV_O_RDWR,
-  &error_abort);
-s = bs->opaque;
-blk_insert_bs(blk, bs, &error_abort);
-
-backing = bdrv_new_open_driver(&bdrv_test, "backing", 0, &error_abort);
-backing_s = backing->opaque;
-bdrv_set_backing_hd(bs, backing, &error_abort);
-
 /* Simple bdrv_drain_all_begin/end pair, check that CBs are called */
 g_assert_cmpint(s->drain_count, ==, 0);
 g_assert_cmpint(backing_s->drain_count, ==, 0);
@@ -245,54 +249,77 @@ static void test_drv_cb_common(enum drain_type 
drain_type, bool recursive)
 
 g_assert_cmpint(s->drain_count, ==, 0);
 g_assert_cmpint(backing_s->drain_count, ==, 0);
+}
 
-bdrv_unref(backing);
-bdrv_unref(bs);
-blk_unref(blk);
+static void test_common_cb(enum drain_type drain_type, bool in_coroutine,
+   void (*cb)(void *))
+{
+TestDriverCBData data;
+
+data.drain_type = drain_type;
+data.recursive = (drain_type != BDRV_DRAIN);
+
+data.blk = blk_new(qemu_get_aio_context(), BLK_PERM_ALL, BLK_PERM_ALL);
+data.bs = bdrv_new_open_driver(&bdrv_test, "test-node", BDRV_O_RDWR,
+  &error_abort);
+blk_insert_bs(data.blk, data.bs, &error_abort);
+
+data.backing = bdrv_new_open_driver(&bdrv_test, "backing", 0, 
&error_abort);
+bdrv_set_backing_hd(data.bs, data.backing, &error_abort);
+
+if (in_coroutine) {
+call_in_coroutine(cb, &data);
+} else {
+cb(&data);
+}
+
+bdrv_unref(data.backing);
+bdrv_unref(data.bs);
+blk_unref(data.blk);
+}
+
+static void test_drv_cb(enum drain_type drain_type, bool in_coroutine)
+{
+test_common_cb(drain_type, in_coroutine, test_drv_cb_common);
 }
 
 static void test_drv_cb_drain_all(void)
 {
-test_drv_cb_common(BDRV_DRAIN_ALL, true);
+test_drv_cb(BDRV_DRAIN_ALL, false);
 }
 
 static void test_drv_cb_drain(void)
 {
-test_drv_cb_common(BDRV_DRAIN, false);
+test_drv_cb(BDRV_DRAIN, false);
 }
 
 static void test_drv_cb_drain_subtree(void)
 {
-test_drv_cb_common(BDRV_SUBTREE_DRAIN, true);
+test_drv_cb(BDRV_SUBTREE_DRAIN, false);
 }
 
 static void test_drv_cb_co_drain_all(void)
 {
-call_in_coroutine(test_drv_cb_drain_all);
+test_drv_cb(BDRV_DRAIN_ALL, true);
 }
 
 static void test_drv_cb_co_drain(void)
 {
-call_in_coroutine(test_drv_cb_drain);
+test_drv_cb(BDRV_DRAIN, true);
 }
 
 static void test_drv_cb_co_drain_subtree(void)
 {
-ca

Re: [PATCH experiment 08/16] tracetool: add extern "C" around generated headers

[Paolo Bonzini]

On 3/14/22 14:33, Philippe Mathieu-Daudé wrote:

On 14/3/22 10:31, Paolo Bonzini wrote:

Signed-off-by: Paolo Bonzini 
---
  scripts/tracetool/format/h.py | 8 +++-
  1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/scripts/tracetool/format/h.py 
b/scripts/tracetool/format/h.py

index e94f0be7da..2d92fa8bd2 100644
--- a/scripts/tracetool/format/h.py
+++ b/scripts/tracetool/format/h.py
@@ -27,6 +27,9 @@ def generate(events, backend, group):
  '#define TRACE_%s_GENERATED_TRACERS_H' % group.upper(),
  '',
  '#include "%s"' % header,
+    '#ifdef __cplusplus',
+    'extern "C" {',
+    '#endif'


Why not use G_BEGIN_DECLS?


I wasn't sure if tracetool dependend on glib.  It's more a philosophical 
question than an actual difference, of course.


Paolo

[PATCH v6 08/18] jobs: use job locks also in the unit tests

[Emanuele Giuseppe Esposito]

Add missing job synchronization in the unit tests, with
explicit locks.

Note: at this stage, job_{lock/unlock} and job lock guard macros
are *nop*.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 tests/unit/test-bdrv-drain.c | 76 -
 tests/unit/test-block-iothread.c |  8 ++-
 tests/unit/test-blockjob-txn.c   | 32 +++
 tests/unit/test-blockjob.c   | 96 
 4 files changed, 148 insertions(+), 64 deletions(-)

diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c
index 36be84ae55..181458eecb 100644
--- a/tests/unit/test-bdrv-drain.c
+++ b/tests/unit/test-bdrv-drain.c
@@ -943,61 +943,83 @@ static void test_blockjob_common_drain_node(enum 
drain_type drain_type,
 }
 }
 
-g_assert_cmpint(job->job.pause_count, ==, 0);
-g_assert_false(job->job.paused);
-g_assert_true(tjob->running);
-g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */
+WITH_JOB_LOCK_GUARD() {
+g_assert_cmpint(job->job.pause_count, ==, 0);
+g_assert_false(job->job.paused);
+g_assert_true(tjob->running);
+g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */
+}
 
 do_drain_begin_unlocked(drain_type, drain_bs);
 
-if (drain_type == BDRV_DRAIN_ALL) {
-/* bdrv_drain_all() drains both src and target */
-g_assert_cmpint(job->job.pause_count, ==, 2);
-} else {
-g_assert_cmpint(job->job.pause_count, ==, 1);
+WITH_JOB_LOCK_GUARD() {
+if (drain_type == BDRV_DRAIN_ALL) {
+/* bdrv_drain_all() drains both src and target */
+g_assert_cmpint(job->job.pause_count, ==, 2);
+} else {
+g_assert_cmpint(job->job.pause_count, ==, 1);
+}
+g_assert_true(job->job.paused);
+g_assert_false(job->job.busy); /* The job is paused */
 }
-g_assert_true(job->job.paused);
-g_assert_false(job->job.busy); /* The job is paused */
 
 do_drain_end_unlocked(drain_type, drain_bs);
 
 if (use_iothread) {
-/* paused is reset in the I/O thread, wait for it */
+/*
+ * Here we are waiting for the paused status to change,
+ * so don't bother protecting the read every time.
+ *
+ * paused is reset in the I/O thread, wait for it
+ */
 while (job->job.paused) {
 aio_poll(qemu_get_aio_context(), false);
 }
 }
 
-g_assert_cmpint(job->job.pause_count, ==, 0);
-g_assert_false(job->job.paused);
-g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */
+WITH_JOB_LOCK_GUARD() {
+g_assert_cmpint(job->job.pause_count, ==, 0);
+g_assert_false(job->job.paused);
+g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */
+}
 
 do_drain_begin_unlocked(drain_type, target);
 
-if (drain_type == BDRV_DRAIN_ALL) {
-/* bdrv_drain_all() drains both src and target */
-g_assert_cmpint(job->job.pause_count, ==, 2);
-} else {
-g_assert_cmpint(job->job.pause_count, ==, 1);
+WITH_JOB_LOCK_GUARD() {
+if (drain_type == BDRV_DRAIN_ALL) {
+/* bdrv_drain_all() drains both src and target */
+g_assert_cmpint(job->job.pause_count, ==, 2);
+} else {
+g_assert_cmpint(job->job.pause_count, ==, 1);
+}
+g_assert_true(job->job.paused);
+g_assert_false(job->job.busy); /* The job is paused */
 }
-g_assert_true(job->job.paused);
-g_assert_false(job->job.busy); /* The job is paused */
 
 do_drain_end_unlocked(drain_type, target);
 
 if (use_iothread) {
-/* paused is reset in the I/O thread, wait for it */
+/*
+ * Here we are waiting for the paused status to change,
+ * so don't bother protecting the read every time.
+ *
+ * paused is reset in the I/O thread, wait for it
+ */
 while (job->job.paused) {
 aio_poll(qemu_get_aio_context(), false);
 }
 }
 
-g_assert_cmpint(job->job.pause_count, ==, 0);
-g_assert_false(job->job.paused);
-g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */
+WITH_JOB_LOCK_GUARD() {
+g_assert_cmpint(job->job.pause_count, ==, 0);
+g_assert_false(job->job.paused);
+g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */
+}
 
 aio_context_acquire(ctx);
-ret = job_complete_sync(&job->job, &error_abort);
+WITH_JOB_LOCK_GUARD() {
+ret = job_complete_sync(&job->job, &error_abort);
+}
 g_assert_cmpint(ret, ==, (result == TEST_JOB_SUCCESS ? 0 : -EIO));
 
 if (use_iothread) {
diff --git a/tests/unit/test-block-iothread.c b/tests/unit/test-block-iothread.c
index 94718c9319..9866262f79 100644
--- a/tests/unit/test-block-iothread.c
+++ b/tests/unit/test-block-iothread.c
@@ -456,7 +456,9 @@ static void test_attach_blockjob(void)
 }
 
 aio_context_acquire(ctx

Re: [PATCH v2 2/3] docs: rSTify MailingLists wiki; move it to QEMU Git

[Philippe Mathieu-Daudé]

Hi Kashyap,

On 14/3/22 11:49, Kashyap Chamarthy wrote:

This document is referred to from the GettingStartedDevelopers wiki
which will be rSTified in a follow-up commit.

Converted from Mediawiki to rST using:

 $> pandoc -f Mediawiki -t rst MailingLists.wiki
 -o mailing-lists.rst

It's a 1-1 conversion (I double-checked to the best I could).  I've also
checked that the hyperlinks work correctly post-conversion.

Signed-off-by: Kashyap Chamarthy 
---
  docs/devel/index.rst |  1 +
  docs/devel/mailing-lists.rst | 53 
  2 files changed, 54 insertions(+)
  create mode 100644 docs/devel/mailing-lists.rst



diff --git a/docs/devel/mailing-lists.rst b/docs/devel/mailing-lists.rst
new file mode 100644
index 00..53dcbfb007
--- /dev/null
+++ b/docs/devel/mailing-lists.rst
@@ -0,0 +1,53 @@
+.. _mailing-lists:
+
+Mailing lists
+=
+
+-  `QEMU developers mailing
+   list `__
+-  `QEMU stable mailing
+   list `__
+-  `QEMU trivial patch mailing
+   list `__
+-  `QEMU users mailing
+   list `__


This is a fair conversion from 
https://wiki.qemu.org/Contribute/MailingLists, but a good opportunity to 
improve (could be on top).


We could sort as:

 * qemu-discuss

   Meant for users. Ideally help should point at Documentation link,
   and in case of missing doc we should add it or at least a GitLab
   @Documentation ticket.

 * qemu-devel

   Meant for developers. "All patches must be sent there".

   Then developer sub-lists:

   - qemu-trivial

   - qemu-stable (this is kinda borderline, security issue fixes should
 Cc this list, however it has to be treated as a write-only list
 - a way to tag patches - no discussion happens there).

   - susbsystem specific

 > block layer

 > architecture specific

   . ARM
   . PPC
   . ...


+.. _subsystem_specific_lists:
+
+Subsystem Specific Lists
+
+
+These exist to make it a little easier to follow subsystem specific
+patches. You should however continue to CC qemu-devel so your series
+gets wide visibility.
+
+-  `QEMU ARM mailing
+   list `__
+-  `QEMU block devices mailing
+   list `__
+-  `QEMU PowerPC mailing
+   list `__
+-  `QEMU RISC-V mailing
+   list `__
+-  `QEMU s390x mailing
+   list `__

Re: [PATCH] x86: q35: require split irqchip for large CPU count

[Daniel P . Berrangé]

On Mon, Mar 14, 2022 at 12:59:38PM +, David Woodhouse wrote:
> On Mon, 2022-03-14 at 11:35 +0100, Igor Mammedov wrote:
> > On Fri, 11 Mar 2022 14:58:41 +
> > David Woodhouse <
> > dw...@infradead.org
> > > wrote:
> > 
> > > On Fri, 2022-03-11 at 09:39 -0500, Igor Mammedov wrote:
> > > > if VM is started with:
> > > > 
> > > >-enable-kvm -smp 256
> > > > 
> > > > without specifying 'split' irqchip, VM might eventually boot
> > > > but no more than 255 CPUs will be operational and following
> > > > error messages in guest could be observed:
> > > >...
> > > >smpboot: native_cpu_up: bad cpu 256
> > > >...
> > > > It's a regression introduced by [1], which removed dependency
> > > > on intremap=on that were implicitly requiring 'split' irqchip
> > > > and forgot to check for 'split' irqchip.
> > > > Instead of letting VM boot a broken VM, error out and tell
> > > > user how to fix CLI.  
> > > 
> > > Hm, wasn't that already fixed in the patches I posted in December?
> > 
> > It might be, could you point to the commit/series that fixed it.
> 
> https://lore.kernel.org/all/20211209220840.14889-1-dw...@infradead.org/
> is the patch I was thinking of, but although that moves the check to a
> more useful place and fixes the X2APIC check, it *doesn't* include the
> fix you're making; it's still using kvm_irqchip_in_kernel().
> 
> I can change that and repost the series, which is still sitting (with
> fixed Reviewed-By/Acked-By attributions that I screwed up last time) in
> https://git.infradead.org/users/dwmw2/qemu.git
> 
> > Regardless of that, fixing it in recent kernels doesn't help
> > as still supported kernels are still affected by it.
> > 
> > If there is a way to detect that fix, I can add to q35 a compat
> > property and an extra logic to enable kernel-irqchip if fix is present.
> > Otherwise the fix does not exist until minimum supported kernel
> > version reaches version where it was fixed.
> 
> Hm, I'm not sure I follow here. Do you mean recent versions of *qemu*
> when you say 'kernels'? 
> 
> I'm not even sure I agree with the observation that qemu should error
> out here. The guest boots fine and the guest can even *use* all the
> CPUs. IPIs etc. will all work fine. The only thing that doesn't work is
> delivering *external* interrupts to CPUs above 254.
> 
> Ultimately, this is the *guest's* problem. Some operating systems can
> cope; some can't.
> 
> The fact that *Linux* has a fundamental assumption that *all* CPUs can
> receive all interrupts and that affinity can't be limited in hardware,
> is a Linux problem. I tried to fix it once but it was distinctly non-
> trivial and eventually I gave up and took a different approach.
> https://lore.kernel.org/linux-iommu/87lfgj59mp@nanos.tec.linutronix.de/T/
> 
> But even if we 'fix' the check as you suggest to bail out and refuse to
> boot a certain configuration because Linux guest wouldn't be able to
> fully utilize it... Even if we boot with the split IRQ chip and the 15-
> bit MSI enlightenment, we're still in the same position. Some guests
> will be able to use it; some won't.
> 
> In fact, there are operating systems that don't even know about X2APIC.
> 
> Why should qemu refuse to even start up?

We've generally said QEMU should not reject / block startup of valid
hardware configurations, based on existance of bugs in certain guest
OS, if the config would be valid for other guest.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[PATCH v6 14/18] commit and mirror: create new nodes using bdrv_get_aio_context, and not the job aiocontext

[Emanuele Giuseppe Esposito]

We are always using the given bs AioContext, so there is no need
to take the job ones (which is identical anyways).
This also reduces the point we need to check when protecting
job.aio_context field.

Reviewed-by: Stefan Hajnoczi 
Signed-off-by: Emanuele Giuseppe Esposito 
---
 block/commit.c | 4 ++--
 block/mirror.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/block/commit.c b/block/commit.c
index c76899f640..99dfc1dacd 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -369,7 +369,7 @@ void commit_start(const char *job_id, BlockDriverState *bs,
 goto fail;
 }
 
-s->base = blk_new(s->common.job.aio_context,
+s->base = blk_new(bdrv_get_aio_context(bs),
   base_perms,
   BLK_PERM_CONSISTENT_READ
   | BLK_PERM_WRITE_UNCHANGED);
@@ -381,7 +381,7 @@ void commit_start(const char *job_id, BlockDriverState *bs,
 s->base_bs = base;
 
 /* Required permissions are already taken with block_job_add_bdrv() */
-s->top = blk_new(s->common.job.aio_context, 0, BLK_PERM_ALL);
+s->top = blk_new(bdrv_get_aio_context(bs), 0, BLK_PERM_ALL);
 ret = blk_insert_bs(s->top, top, errp);
 if (ret < 0) {
 goto fail;
diff --git a/block/mirror.c b/block/mirror.c
index a7c3bab382..8a975537c0 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1727,7 +1727,7 @@ static BlockJob *mirror_start_job(
 goto fail;
 }
 
-s->target = blk_new(s->common.job.aio_context,
+s->target = blk_new(bdrv_get_aio_context(bs),
 target_perms, target_shared_perms);
 ret = blk_insert_bs(s->target, target, errp);
 if (ret < 0) {
-- 
2.31.1

Re: [PATCH] ppc/pnv: Remove user-created PHB{3,4,5} devices

[Daniel Henrique Barboza]

On 3/14/22 10:05, Cédric Le Goater wrote:

On a real system with POWER{8,9,10} processors, PHBs are sub-units of
the processor, they can be deactivated by firmware but not plugged in
or out like a PCI adapter on a slot. Nevertheless, having user-created
PHBs in QEMU seemed to be a good idea for testing purposes :

  1. having a limited set of PHBs speedups boot time.
  2. it is useful to be able to mimic a partially broken topology you
 some time have to deal with during bring-up.

PowerNV is also used for distro install tests and having libvirt
support eases these tasks. libvirt prefers to run the machine with
-nodefaults to be sure not to drag unexpected devices which would need
to be defined in the domain file without being specified on the QEMU
command line. For this reason :

  3. -nodefaults should not include default PHBs

User-created PHB{3,4,5} devices satisfied all these needs but reality
proves to be a bit more complex, internally when modeling such
devices, and externally when dealing with the user interface.

Req 1. and 2. can be simply addressed differently with a machine option:
"phb-mask=", which QEMU would use to enable/disable PHB device
nodes when creating the device tree.

For Req 3., we need to make sure we are taking the right approach. It
seems that we should expose a new type of user-created PHB device, a
generic virtualized one, that libvirt would use and not one depending
on the processor revision. This needs more thinking.

For now, remove user-created PHB{3,4,5} devices. All the cleanups we
did are not lost and they will be useful for the next steps.

Fixes: 5bc67b052b51 ("ppc/pnv: Introduce user creatable pnv-phb4 devices")
Fixes: 1f6a88fffc75 ("ppc/pnv: Introduce support for user created PHB3 devices")
Signed-off-by: Cédric Le Goater 
---


Reviewed-by: Daniel Henrique Barboza 


  include/hw/ppc/pnv.h   |  1 -
  hw/pci-host/pnv_phb3.c | 33 ++--
  hw/pci-host/pnv_phb4.c | 62 ++
  hw/pci-host/pnv_phb4_pec.c |  7 ++---
  hw/ppc/pnv.c   | 25 +--
  5 files changed, 10 insertions(+), 118 deletions(-)

diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index 1e34ddd502d8..86cb7d7f971b 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -190,7 +190,6 @@ DECLARE_INSTANCE_CHECKER(PnvChip, PNV_CHIP_POWER10,
  
  PowerPCCPU *pnv_chip_find_cpu(PnvChip *chip, uint32_t pir);

  void pnv_phb_attach_root_port(PCIHostState *pci, const char *name);
-void pnv_chip_parent_fixup(PnvChip *chip, Object *obj, int index);
  
  #define TYPE_PNV_MACHINE   MACHINE_TYPE_NAME("powernv")

  typedef struct PnvMachineClass PnvMachineClass;
diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
index aafd46b635e8..ac801ac83582 100644
--- a/hw/pci-host/pnv_phb3.c
+++ b/hw/pci-host/pnv_phb3.c
@@ -994,30 +994,6 @@ static void pnv_phb3_realize(DeviceState *dev, Error 
**errp)
  PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
  int i;
  
-/* User created devices */

-if (!phb->chip) {
-Error *local_err = NULL;
-BusState *s;
-
-phb->chip = pnv_get_chip(pnv, phb->chip_id);
-if (!phb->chip) {
-error_setg(errp, "invalid chip id: %d", phb->chip_id);
-return;
-}
-
-/*
- * Reparent user created devices to the chip to build
- * correctly the device tree.
- */
-pnv_chip_parent_fixup(phb->chip, OBJECT(phb), phb->phb_id);
-
-s = qdev_get_parent_bus(DEVICE(phb->chip));
-if (!qdev_set_parent_bus(DEVICE(phb), s, &local_err)) {
-error_propagate(errp, local_err);
-return;
-}
-}
-
  if (phb->phb_id >= PNV_CHIP_GET_CLASS(phb->chip)->num_phbs) {
  error_setg(errp, "invalid PHB index: %d", phb->phb_id);
  return;
@@ -1077,10 +1053,7 @@ static void pnv_phb3_realize(DeviceState *dev, Error 
**errp)
  
  pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb);
  
-if (defaults_enabled()) {

-pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb),
- TYPE_PNV_PHB3_ROOT_PORT);
-}
+pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
  }
  
  void pnv_phb3_update_regions(PnvPHB3 *phb)

@@ -1131,7 +1104,7 @@ static void pnv_phb3_class_init(ObjectClass *klass, void 
*data)
  dc->realize = pnv_phb3_realize;
  device_class_set_props(dc, pnv_phb3_properties);
  set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
-dc->user_creatable = true;
+dc->user_creatable = false;
  }
  
  static const TypeInfo pnv_phb3_type_info = {

@@ -1201,7 +1174,7 @@ static void pnv_phb3_root_port_class_init(ObjectClass 
*klass, void *data)
  
  device_class_set_parent_realize(dc, pnv_phb3_root_port_realize,

  &rpc->parent_realize);
-dc->user_creatable = true;
+dc->user_creatable = false;
  
  k->vendor_id = PCI_VENDOR_ID_

[PULL 0/1] I2C patches for 2022-03-14

[Philippe Mathieu-Daudé]

From: Philippe Mathieu-Daudé 

The following changes since commit 15df33ceb73cb6bb3c6736cf4d2cff51129ed4b4:

  Merge remote-tracking branch 'remotes/quic/tags/pull-hex-20220312-1' into 
staging (2022-03-13 17:29:18 +)

are available in the Git repository at:

  https://github.com/philmd/qemu.git tags/i2c-20220314

for you to fetch changes up to 1cbab82e9d1bdb2c7b9ef46a396fdc03ea3fa04c:

  hw/nvram: at24 return 0xff if 1 byte address (2022-03-14 14:48:35 +0100)


I2C patch queue

- Fix AT24 EEPROM partial write (Patrick Venture)



Patrick Venture (1):
  hw/nvram: at24 return 0xff if 1 byte address

 hw/nvram/eeprom_at24c.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

-- 
2.34.1

Re: [PULL v2 00/12] Hexagon (target/hexagon) queue

[Peter Maydell]

On Sat, 12 Mar 2022 at 20:13, Taylor Simpson  wrote:
>
> The following changes since commit 1416688c53be6535be755b44c15fb2eb9defd20f:
>
>   Merge remote-tracking branch 
> 'remotes/mcayland/tags/q800-updates-for-7.0-20220309' into staging 
> (2022-03-10 13:16:37 +)
>
> are available in the Git repository at:
>
>   https://github.com/quic/qemu tags/pull-hex-20220312-1
>
> for you to fetch changes up to 4a818b3767220dcd21cf5cc7f12e33e28c2073ed:
>
>   target/hexagon: remove unused variable (2022-03-12 09:14:22 -0800)
>
> 
> Hexagon bug fixes and additional tests
>
> Also includes a patch from Zongyuan Li  to remove an 
> unused variable
>
>  Changes in v2 
> Fix problems with build-user-hexagon CI job


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/7.0
for any user-visible changes.

-- PMM

Re: [PATCH experiment 12/16] remove "new" keyword from trace-events

[Philippe Mathieu-Daudé]

On 14/3/22 10:31, Paolo Bonzini wrote:

This is invalid in C++.

Signed-off-by: Paolo Bonzini 
---
  util/trace-events | 16 
  1 file changed, 8 insertions(+), 8 deletions(-)


Reviewed-by: Philippe Mathieu-Daudé 

[PATCH v1 1/1] hw: aspeed_scu: Add AST2600 hpll calculation function

[Steven Lee]

AST2600's HPLL register offset and bit definition are different from
AST2500. Add a hpll calculation function for ast2600 and modify apb frequency
calculation function based on SCU200 register description in ast2600v11.pdf.

Signed-off-by: Steven Lee 
---
 hw/misc/aspeed_scu.c | 43 
 include/hw/misc/aspeed_scu.h | 17 ++
 2 files changed, 56 insertions(+), 4 deletions(-)

diff --git a/hw/misc/aspeed_scu.c b/hw/misc/aspeed_scu.c
index d06e179a6e..3b11e98d66 100644
--- a/hw/misc/aspeed_scu.c
+++ b/hw/misc/aspeed_scu.c
@@ -205,6 +205,8 @@ static const uint32_t ast2500_a1_resets[ASPEED_SCU_NR_REGS] 
= {
  [BMC_DEV_ID]  = 0x2402U
 };
 
+static uint32_t aspeed_2600_scu_calc_hpll(AspeedSCUState *s, uint32_t 
hpll_reg);
+
 static uint32_t aspeed_scu_get_random(void)
 {
 uint32_t num;
@@ -215,9 +217,19 @@ static uint32_t aspeed_scu_get_random(void)
 uint32_t aspeed_scu_get_apb_freq(AspeedSCUState *s)
 {
 AspeedSCUClass *asc = ASPEED_SCU_GET_CLASS(s);
-uint32_t hpll = asc->calc_hpll(s, s->regs[HPLL_PARAM]);
+uint32_t hpll, hpll_reg, clk_sel_reg;
+
+if (asc->calc_hpll == aspeed_2600_scu_calc_hpll) {
+hpll_reg = s->regs[AST2600_HPLL_PARAM];
+clk_sel_reg = s->regs[AST2600_CLK_SEL];
+} else {
+hpll_reg = s->regs[HPLL_PARAM];
+clk_sel_reg = s->regs[CLK_SEL];
+}
+
+hpll = asc->calc_hpll(s, hpll_reg);
 
-return hpll / (SCU_CLK_GET_PCLK_DIV(s->regs[CLK_SEL]) + 1)
+return hpll / (SCU_CLK_GET_PCLK_DIV(clk_sel_reg) + 1)
 / asc->apb_divider;
 }
 
@@ -357,7 +369,10 @@ static const MemoryRegionOps aspeed_ast2500_scu_ops = {
 
 static uint32_t aspeed_scu_get_clkin(AspeedSCUState *s)
 {
-if (s->hw_strap1 & SCU_HW_STRAP_CLK_25M_IN) {
+AspeedSCUClass *asc = ASPEED_SCU_GET_CLASS(s);
+
+if (s->hw_strap1 & SCU_HW_STRAP_CLK_25M_IN ||
+asc->calc_hpll == aspeed_2600_scu_calc_hpll) {
 return 2500;
 } else if (s->hw_strap1 & SCU_HW_STRAP_CLK_48M_IN) {
 return 4800;
@@ -426,6 +441,26 @@ static uint32_t aspeed_2500_scu_calc_hpll(AspeedSCUState 
*s, uint32_t hpll_reg)
 return clkin * multiplier;
 }
 
+static uint32_t aspeed_2600_scu_calc_hpll(AspeedSCUState *s, uint32_t hpll_reg)
+{
+uint32_t multiplier = 1;
+uint32_t clkin = aspeed_scu_get_clkin(s);
+
+if (hpll_reg & SCU_AST2600_H_PLL_OFF) {
+return 0;
+}
+
+if (!(hpll_reg & SCU_H_PLL_BYPASS_EN)) {
+uint32_t p = (hpll_reg >> 19) & 0xf;
+uint32_t n = (hpll_reg >> 13) & 0x3f;
+uint32_t m = hpll_reg & 0x1fff;
+
+multiplier = ((m + 1) / (n + 1)) / (p + 1);
+}
+
+return clkin * multiplier;
+}
+
 static void aspeed_scu_reset(DeviceState *dev)
 {
 AspeedSCUState *s = ASPEED_SCU(dev);
@@ -716,7 +751,7 @@ static void aspeed_2600_scu_class_init(ObjectClass *klass, 
void *data)
 dc->desc = "ASPEED 2600 System Control Unit";
 dc->reset = aspeed_ast2600_scu_reset;
 asc->resets = ast2600_a3_resets;
-asc->calc_hpll = aspeed_2500_scu_calc_hpll; /* No change since AST2500 */
+asc->calc_hpll = aspeed_2600_scu_calc_hpll;
 asc->apb_divider = 4;
 asc->nr_regs = ASPEED_AST2600_SCU_NR_REGS;
 asc->ops = &aspeed_ast2600_scu_ops;
diff --git a/include/hw/misc/aspeed_scu.h b/include/hw/misc/aspeed_scu.h
index c14aff2bcb..91c500c5bc 100644
--- a/include/hw/misc/aspeed_scu.h
+++ b/include/hw/misc/aspeed_scu.h
@@ -316,4 +316,21 @@ uint32_t aspeed_scu_get_apb_freq(AspeedSCUState *s);
 SCU_HW_STRAP_VGA_SIZE_SET(VGA_16M_DRAM) |   \
 SCU_AST2500_HW_STRAP_RESERVED1)
 
+/* SCU200   H-PLL Parameter Register (for Aspeed AST2600 SOC)
+ *
+ *  28:26  H-PLL Parameters
+ *  25 Enable H-PLL reset
+ *  24 Enable H-PLL bypass mode
+ *  23 Turn off H-PLL
+ *  22:19  H-PLL Post Divider (P)
+ *  18:13   H-PLL Numerator (M)
+ *  12:0H-PLL Denumerator (N)
+ *
+ *  (Output frequency) = CLKIN(25MHz) * [(M+1) / (N+1)] / (P+1)
+ *
+ * The default frequency is 1200Mhz when CLKIN = 25MHz
+ */
+#define SCU_AST2600_H_PLL_BYPASS_EN(0x1 << 24)
+#define SCU_AST2600_H_PLL_OFF  (0x1 << 23)
+
 #endif /* ASPEED_SCU_H */
-- 
2.17.1

[PATCH-for-7.0 v3] softmmu: List CPU types again

[Philippe Mathieu-Daudé]

From: Philippe Mathieu-Daudé 

Commit e0220bb5b2 made cpus.c target-agnostic but didn't notice
the cpu_list() function is only defined in target-specific code
in "cpu.h". Move list_cpus() declaration to "exec/cpu-common.h"
because this function is not softmmu-specific and can also be
used by user-mode, along with moving its implementation to cpu.c,
which is compiled per target.

Fixes: e0220bb5b2 ("softmmu: Build target-agnostic objects once")
Reported-by: Max Filippov 
Suggested-by: Paolo Bonzini 
Reviewed-by: Thomas Huth 
Tested-by: Max Filippov 
Signed-off-by: Philippe Mathieu-Daudé 
---
Since v2:
- include exec/cpu-all.h -> exec/cpu-common.h (thuth)
---
 cpu.c | 9 +
 include/exec/cpu-common.h | 2 ++
 include/sysemu/cpus.h | 2 --
 softmmu/cpus.c| 8 
 4 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/cpu.c b/cpu.c
index d564886149..be1f8b074c 100644
--- a/cpu.c
+++ b/cpu.c
@@ -35,6 +35,7 @@
 #include "sysemu/tcg.h"
 #include "sysemu/kvm.h"
 #include "sysemu/replay.h"
+#include "exec/cpu-common.h"
 #include "exec/exec-all.h"
 #include "exec/translate-all.h"
 #include "exec/log.h"
@@ -266,6 +267,14 @@ const char *parse_cpu_option(const char *cpu_option)
 return cpu_type;
 }
 
+void list_cpus(const char *optarg)
+{
+/* XXX: implement xxx_cpu_list for targets that still miss it */
+#if defined(cpu_list)
+cpu_list();
+#endif
+}
+
 #if defined(CONFIG_USER_ONLY)
 void tb_invalidate_phys_addr(target_ulong addr)
 {
diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index 7f7b5943c7..50a7d2912e 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -158,4 +158,6 @@ int cpu_memory_rw_debug(CPUState *cpu, vaddr addr,
 /* vl.c */
 extern int singlestep;
 
+void list_cpus(const char *optarg);
+
 #endif /* CPU_COMMON_H */
diff --git a/include/sysemu/cpus.h b/include/sysemu/cpus.h
index 868f1192de..b5c87d48b3 100644
--- a/include/sysemu/cpus.h
+++ b/include/sysemu/cpus.h
@@ -55,6 +55,4 @@ extern int smp_cores;
 extern int smp_threads;
 #endif
 
-void list_cpus(const char *optarg);
-
 #endif
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index e1d84c8ccb..7b75bb66d5 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -728,14 +728,6 @@ int vm_stop_force_state(RunState state)
 }
 }
 
-void list_cpus(const char *optarg)
-{
-/* XXX: implement xxx_cpu_list for targets that still miss it */
-#if defined(cpu_list)
-cpu_list();
-#endif
-}
-
 void qmp_memsave(int64_t addr, int64_t size, const char *filename,
  bool has_cpu, int64_t cpu_index, Error **errp)
 {
-- 
2.34.1

Re: [PATCH v2] block-qdict: Fix -Werror=maybe-uninitialized build failure

[Philippe Mathieu-Daudé]

On 11/3/22 23:16, Murilo Opsfelder Araujo wrote:

Building QEMU on Fedora 37 (Rawhide Prerelease) ppc64le failed with the
following error:

 $ ../configure --prefix=/usr/local/qemu-disabletcg 
--target-list=ppc-softmmu,ppc64-softmmu --disable-tcg --disable-linux-user
 ...
 $ make -j$(nproc)
 ...
 FAILED: libqemuutil.a.p/qobject_block-qdict.c.o


This part >>>


 cc -m64 -mlittle-endian -Ilibqemuutil.a.p -I. -I.. 
-Isubprojects/libvhost-user -I../subprojects/libvhost-user -Iqapi -Itrace -Iui 
-Iui/shader -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include 
-I/usr/include/sysprof-4 -I/usr/include/lib
 mount -I/usr/include/blkid -I/usr/include/gio-unix-2.0 
-I/usr/include/p11-kit-1 -I/usr/include/pixman-1 -fdiagnostics-color=auto -Wall 
-Winvalid-pch -Werror -std=gnu11 -O2 -g -isystem /root/qemu/linux-headers 
-isystem linux-headers -iquote
  . -iquote /root/qemu -iquote /root/qemu/include -iquote 
/root/qemu/disas/libvixl -pthread -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
-Wredundant-decls -Wundef -Wwrite
 -strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv 
-Wold-style-declaration -Wold-style-definition -Wtype-limits -Wformat-security 
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body -Wnested-externs 
-Wendif-label
 s -Wexpansion-to-defined -Wimplicit-fallthrough=2 
-Wno-missing-include-dirs -Wno-shift-negative-value -Wno-psabi 
-fstack-protector-strong -fPIE -MD -MQ libqemuutil.a.p/qobject_block-qdict.c.o 
-MF libqemuutil.a.p/qobject_block-qdict.c.o.d -
 o libqemuutil.a.p/qobject_block-qdict.c.o -c ../qobject/block-qdict.c


<<< is noise (doesn't provide any value) and could be stripped.


 In file included from /root/qemu/include/qapi/qmp/qdict.h:16,
  from /root/qemu/include/block/qdict.h:13,
  from ../qobject/block-qdict.c:11:
 /root/qemu/include/qapi/qmp/qobject.h: In function ‘qdict_array_split’:
 /root/qemu/include/qapi/qmp/qobject.h:49:17: error: ‘subqdict’ may be used 
uninitialized [-Werror=maybe-uninitialized]
49 | typeof(obj) _obj = (obj);   \
   | ^~~~
 ../qobject/block-qdict.c:227:16: note: ‘subqdict’ declared here
   227 | QDict *subqdict;
   |^~~~
 cc1: all warnings being treated as errors

Fix build failure by expanding the ternary operation.
Tested with `make check-unit` (the check-block-qdict test passed).

Signed-off-by: Murilo Opsfelder Araujo 
Cc: Kevin Wolf 
Cc: Hanna Reitz 
Cc: Markus Armbruster 
---
v1: https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg03224.html

  qobject/block-qdict.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c
index 1487cc5dd8..4a83bda2c3 100644
--- a/qobject/block-qdict.c
+++ b/qobject/block-qdict.c
@@ -251,12 +251,12 @@ void qdict_array_split(QDict *src, QList **dst)
  if (is_subqdict) {
  qdict_extract_subqdict(src, &subqdict, prefix);
  assert(qdict_size(subqdict) > 0);
+qlist_append_obj(*dst, QOBJECT(subqdict));
  } else {
  qobject_ref(subqobj);
  qdict_del(src, indexstr);
+qlist_append_obj(*dst, subqobj);
  }
-
-qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict));
  }
  }
  

[PATCH v1 0/1] hw: aspeed_scu: Add AST2600 hpll calculation function

[Steven Lee]

AST2600's HPLL register offset and bit definition are different from AST2500.
The patch series adds a hpll calculation function for ast2600 and modify apb 
frequency
calculation function based on SCU200 register description and note in 
ast2600v11.pdf.

Please help to review.

Thanks,
Steven

Steven Lee (1):
  hw: aspeed_scu: Add AST2600 hpll calculation function

 hw/misc/aspeed_scu.c | 43 
 include/hw/misc/aspeed_scu.h | 17 ++
 2 files changed, 56 insertions(+), 4 deletions(-)

-- 
2.17.1

Re: [PATCH v2 2/4] Introduce event-loop-base abstract class

[Nicolas Saenz Julienne]

On Mon, 2022-03-14 at 13:33 +, Stefan Hajnoczi wrote:
> On Fri, Mar 11, 2022 at 11:17:22AM +0100, Nicolas Saenz Julienne wrote:
> > On Thu, 2022-03-10 at 10:25 +, Stefan Hajnoczi wrote:
> > > On Thu, Mar 03, 2022 at 03:58:20PM +0100, Nicolas Saenz Julienne wrote:
> > > > @@ -2935,13 +2947,6 @@ qemu_syms = custom_target('qemu.syms', output: 
> > > > 'qemu.syms',
> > > >   capture: true,
> > > >   command: [undefsym, nm, '@INPUT@'])
> > > >  
> > > > -qom_ss = qom_ss.apply(config_host, strict: false)
> > > > -libqom = static_library('qom', qom_ss.sources() + genh,
> > > > -dependencies: [qom_ss.dependencies()],
> > > > -name_suffix: 'fa')
> > > > -
> > > > -qom = declare_dependency(link_whole: libqom)
> > > > -
> > > 
> > > Why was it necessary to move qom_ss and subdir('hw') up? Can
> > > event_loop_base be defined down here instead?
> > 
> > The way I setup it up, qemuutil now depdens on event_loop_base which in turn
> > depends on qom. IIUC I can't declare dependencies without declaring first 
> > the
> > libraries and source sets. All has to happen sequencially. With this in 
> > mind,
> > almost all libraries depend on libqemuutil so moving it down isn't possible.
> 
> I see now. The qemuutil dependency on event_loop_base is introduced in
> the next patch so the reason wasn't clear at this point in the patch
> series.

I'll mention it in the commit message.

-- 
Nicolás Sáenz

[PATCH] MAINTAINERS: change Vladimir's email address

[Vladimir Sementsov-Ogievskiy]

Old vsement...@virtuozzo.com is not accessible anymore.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---

Hi all!

That's my new address: v.sementsov...@ya.ru , the old one is not
available anymore.

I've also subscribed this new address for qemu-devel mailing list, but
not yet get any message from it :(

The patch is also available as pgp-signed tag at:

  https://src.openvz.org/scm/~vsementsov/qemu.git tags/change-address


 MAINTAINERS | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index f2e9ce1da2..8488de5c0b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2500,7 +2500,7 @@ F: scsi/*
 
 Block Jobs
 M: John Snow 
-M: Vladimir Sementsov-Ogievskiy 
+M: Vladimir Sementsov-Ogievskiy 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: blockjob.c
@@ -2539,7 +2539,7 @@ T: git https://repo.or.cz/qemu/armbru.git block-next
 
 Dirty Bitmaps
 M: Eric Blake 
-M: Vladimir Sementsov-Ogievskiy 
+M: Vladimir Sementsov-Ogievskiy 
 R: John Snow 
 L: qemu-bl...@nongnu.org
 S: Supported
@@ -2762,13 +2762,13 @@ F: scripts/*.py
 F: tests/*.py
 
 Benchmark util
-M: Vladimir Sementsov-Ogievskiy 
+M: Vladimir Sementsov-Ogievskiy 
 S: Maintained
 F: scripts/simplebench/
 T: git https://src.openvz.org/scm/~vsementsov/qemu.git simplebench
 
 Transactions helper
-M: Vladimir Sementsov-Ogievskiy 
+M: Vladimir Sementsov-Ogievskiy 
 S: Maintained
 F: include/qemu/transactions.h
 F: util/transactions.c
@@ -3352,7 +3352,7 @@ F: block/iscsi-opts.c
 
 Network Block Device (NBD)
 M: Eric Blake 
-M: Vladimir Sementsov-Ogievskiy 
+M: Vladimir Sementsov-Ogievskiy 
 L: qemu-bl...@nongnu.org
 S: Maintained
 F: block/nbd*
@@ -3448,7 +3448,7 @@ F: block/dmg.c
 parallels
 M: Stefan Hajnoczi 
 M: Denis V. Lunev 
-M: Vladimir Sementsov-Ogievskiy 
+M: Vladimir Sementsov-Ogievskiy 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: block/parallels.c
-- 
2.35.1

Re: [PATCH v2 4/4] util/event-loop-base: Introduce options to set the thread pool size

[Stefan Hajnoczi]

On Fri, Mar 11, 2022 at 11:40:30AM +0100, Nicolas Saenz Julienne wrote:
> On Thu, 2022-03-10 at 10:45 +, Stefan Hajnoczi wrote:
> > On Thu, Mar 03, 2022 at 04:13:07PM +0100, Nicolas Saenz Julienne wrote:
> > > @@ -537,10 +546,19 @@
> > >  # 0 means that the engine will use its default
> > >  # (default:0, since 6.1)
> > >  #
> > > +# @thread-pool-min: minimum number of threads readily available in the 
> > > thread
> > > +#   pool
> > > +#   (default:0, since 6.2)
> > > +#
> > > +# @thread-pool-max: maximum number of threads the thread pool can contain
> > > +#   (default:64, since 6.2)
> > 
> > Here and elsewhere:
> > s/6.2/7.1/
> 
> Yes, forgot to mention it was a placeholder, as I wasn't sure what version to
> target.
> 
> > > @@ -294,6 +314,36 @@ void thread_pool_submit(ThreadPool *pool, 
> > > ThreadPoolFunc *func, void *arg)
> > >  thread_pool_submit_aio(pool, func, arg, NULL, NULL);
> > >  }
> > >  
> > > +void thread_pool_update_params(ThreadPool *pool, AioContext *ctx)
> > > +{
> > > +qemu_mutex_lock(&pool->lock);
> > > +
> > > +pool->min_threads = ctx->thread_pool_min;
> > > +pool->max_threads = ctx->thread_pool_max;
> > > +
> > > +/*
> > > + * We either have to:
> > > + *  - Increase the number available of threads until over the 
> > > min_threads
> > > + *threshold.
> > > + *  - Decrease the number of available threads until under the 
> > > max_threads
> > > + *threshold.
> > > + *  - Do nothing. the current number of threads fall in between the 
> > > min and
> > > + *max thresholds. We'll let the pool manage itself.
> > > + */
> > > +for (int i = pool->cur_threads; i < pool->min_threads; i++) {
> > > +spawn_thread(pool);
> > > +}
> > > +
> > > +while (pool->cur_threads > pool->max_threads) {
> > > +qemu_sem_post(&pool->sem);
> > > +qemu_mutex_unlock(&pool->lock);
> > > +qemu_cond_wait(&pool->worker_stopped, &pool->lock);
> > > +qemu_mutex_lock(&pool->lock);
> > 
> > Same question as Patch 1. This looks incorrect because qemu_cond_wait()
> > already drops pool->lock if it needs to block.
> 
> Yes, I'll fix that.
> 
> > Also, why wait? If worker threads are blocked for some reason then our
> > thread will block too.
> 
> Exiting thread_pool_update_params() before honoring the new constraints is a
> source of potential race conditions (having to worry for situations where
> cur_threads > max_threads), and on systems where determinism is important it's
> crucial to have a clear boundary between 'unsafe' and 'safe' states.

On the other hand it creates a reliability problem where a random worker
thread can block all of QEMU. Maybe it's better to let a blocked worker
thread terminate eventually than to hang QEMU?

Stefan


signature.asc
Description: PGP signature

Re: [PATCH] ppc/pnv: Remove user-created PHB{3,4,5} devices

[Frederic Barrat]

On 14/03/2022 14:05, Cédric Le Goater wrote:

On a real system with POWER{8,9,10} processors, PHBs are sub-units of
the processor, they can be deactivated by firmware but not plugged in
or out like a PCI adapter on a slot. Nevertheless, having user-created
PHBs in QEMU seemed to be a good idea for testing purposes :

  1. having a limited set of PHBs speedups boot time.
  2. it is useful to be able to mimic a partially broken topology you
 some time have to deal with during bring-up.

PowerNV is also used for distro install tests and having libvirt
support eases these tasks. libvirt prefers to run the machine with
-nodefaults to be sure not to drag unexpected devices which would need
to be defined in the domain file without being specified on the QEMU
command line. For this reason :

  3. -nodefaults should not include default PHBs

User-created PHB{3,4,5} devices satisfied all these needs but reality
proves to be a bit more complex, internally when modeling such
devices, and externally when dealing with the user interface.

Req 1. and 2. can be simply addressed differently with a machine option:
"phb-mask=", which QEMU would use to enable/disable PHB device
nodes when creating the device tree.

For Req 3., we need to make sure we are taking the right approach. It
seems that we should expose a new type of user-created PHB device, a
generic virtualized one, that libvirt would use and not one depending
on the processor revision. This needs more thinking.

For now, remove user-created PHB{3,4,5} devices. All the cleanups we
did are not lost and they will be useful for the next steps.

Fixes: 5bc67b052b51 ("ppc/pnv: Introduce user creatable pnv-phb4 devices")
Fixes: 1f6a88fffc75 ("ppc/pnv: Introduce support for user created PHB3 devices")
Signed-off-by: Cédric Le Goater 
---


So with this patch, -nodefaults has basically no effect on the PHBs and 
root ports creation. They are always there. It makes sense, since we are 
removing the capability to add a subset of them.


Reviewed-by: Frederic Barrat 



  include/hw/ppc/pnv.h   |  1 -
  hw/pci-host/pnv_phb3.c | 33 ++--
  hw/pci-host/pnv_phb4.c | 62 ++
  hw/pci-host/pnv_phb4_pec.c |  7 ++---
  hw/ppc/pnv.c   | 25 +--
  5 files changed, 10 insertions(+), 118 deletions(-)

diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index 1e34ddd502d8..86cb7d7f971b 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -190,7 +190,6 @@ DECLARE_INSTANCE_CHECKER(PnvChip, PNV_CHIP_POWER10,
  
  PowerPCCPU *pnv_chip_find_cpu(PnvChip *chip, uint32_t pir);

  void pnv_phb_attach_root_port(PCIHostState *pci, const char *name);
-void pnv_chip_parent_fixup(PnvChip *chip, Object *obj, int index);
  
  #define TYPE_PNV_MACHINE   MACHINE_TYPE_NAME("powernv")

  typedef struct PnvMachineClass PnvMachineClass;
diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c
index aafd46b635e8..ac801ac83582 100644
--- a/hw/pci-host/pnv_phb3.c
+++ b/hw/pci-host/pnv_phb3.c
@@ -994,30 +994,6 @@ static void pnv_phb3_realize(DeviceState *dev, Error 
**errp)
  PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
  int i;
  
-/* User created devices */

-if (!phb->chip) {
-Error *local_err = NULL;
-BusState *s;
-
-phb->chip = pnv_get_chip(pnv, phb->chip_id);
-if (!phb->chip) {
-error_setg(errp, "invalid chip id: %d", phb->chip_id);
-return;
-}
-
-/*
- * Reparent user created devices to the chip to build
- * correctly the device tree.
- */
-pnv_chip_parent_fixup(phb->chip, OBJECT(phb), phb->phb_id);
-
-s = qdev_get_parent_bus(DEVICE(phb->chip));
-if (!qdev_set_parent_bus(DEVICE(phb), s, &local_err)) {
-error_propagate(errp, local_err);
-return;
-}
-}
-
  if (phb->phb_id >= PNV_CHIP_GET_CLASS(phb->chip)->num_phbs) {
  error_setg(errp, "invalid PHB index: %d", phb->phb_id);
  return;
@@ -1077,10 +1053,7 @@ static void pnv_phb3_realize(DeviceState *dev, Error 
**errp)
  
  pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb);
  
-if (defaults_enabled()) {

-pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb),
- TYPE_PNV_PHB3_ROOT_PORT);
-}
+pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb), TYPE_PNV_PHB3_ROOT_PORT);
  }
  
  void pnv_phb3_update_regions(PnvPHB3 *phb)

@@ -1131,7 +1104,7 @@ static void pnv_phb3_class_init(ObjectClass *klass, void 
*data)
  dc->realize = pnv_phb3_realize;
  device_class_set_props(dc, pnv_phb3_properties);
  set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
-dc->user_creatable = true;
+dc->user_creatable = false;
  }
  
  static const TypeInfo pnv_phb3_type_info = {

@@ -1201,7 +1174,7 @@ static void pnv_phb3_root_port_class_init(ObjectClass 
*klass, void *data)
  
  device_class_set_parent_realize(dc,

[PATCH v6 03/18] job.c: API functions not used outside should be static

[Emanuele Giuseppe Esposito]

job_event_* functions can all be static, as they are not used
outside job.c.

Same applies for job_txn_add_job().

Signed-off-by: Emanuele Giuseppe Esposito 
---
 include/qemu/job.h | 18 --
 job.c  | 22 +++---
 2 files changed, 19 insertions(+), 21 deletions(-)

diff --git a/include/qemu/job.h b/include/qemu/job.h
index 86ec46c09e..6000463126 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -356,18 +356,6 @@ JobTxn *job_txn_new(void);
  */
 void job_txn_unref(JobTxn *txn);
 
-/**
- * @txn: The transaction (may be NULL)
- * @job: Job to add to the transaction
- *
- * Add @job to the transaction.  The @job must not already be in a transaction.
- * The caller must call either job_txn_unref() or job_completed() to release
- * the reference that is automatically grabbed here.
- *
- * If @txn is NULL, the function does nothing.
- */
-void job_txn_add_job(JobTxn *txn, Job *job);
-
 /**
  * Create a new long-running job and return it.
  *
@@ -429,12 +417,6 @@ void job_progress_set_remaining(Job *job, uint64_t 
remaining);
  */
 void job_progress_increase_remaining(Job *job, uint64_t delta);
 
-/** To be called when a cancelled job is finalised. */
-void job_event_cancelled(Job *job);
-
-/** To be called when a successfully completed job is finalised. */
-void job_event_completed(Job *job);
-
 /**
  * Conditionally enter the job coroutine if the job is ready to run, not
  * already busy and fn() returns true. fn() is called while under the job_lock
diff --git a/job.c b/job.c
index 2b4ffca9d4..cafd597ba4 100644
--- a/job.c
+++ b/job.c
@@ -125,7 +125,17 @@ void job_txn_unref(JobTxn *txn)
 }
 }
 
-void job_txn_add_job(JobTxn *txn, Job *job)
+/**
+ * @txn: The transaction (may be NULL)
+ * @job: Job to add to the transaction
+ *
+ * Add @job to the transaction.  The @job must not already be in a transaction.
+ * The caller must call either job_txn_unref() or job_completed() to release
+ * the reference that is automatically grabbed here.
+ *
+ * If @txn is NULL, the function does nothing.
+ */
+static void job_txn_add_job(JobTxn *txn, Job *job)
 {
 if (!txn) {
 return;
@@ -427,12 +437,18 @@ void job_progress_increase_remaining(Job *job, uint64_t 
delta)
 progress_increase_remaining(&job->progress, delta);
 }
 
-void job_event_cancelled(Job *job)
+/**
+ * To be called when a cancelled job is finalised.
+ */
+static void job_event_cancelled(Job *job)
 {
 notifier_list_notify(&job->on_finalize_cancelled, job);
 }
 
-void job_event_completed(Job *job)
+/**
+ * To be called when a successfully completed job is finalised.
+ */
+static void job_event_completed(Job *job)
 {
 notifier_list_notify(&job->on_finalize_completed, job);
 }
-- 
2.31.1

[PATCH v6 00/18] job: replace AioContext lock with job_mutex

[Emanuele Giuseppe Esposito]

In this series, we want to remove the AioContext lock and instead
use the already existent job_mutex to protect the job structures
and list. This is part of the work to get rid of AioContext lock
usage in favour of smaller granularity locks.

In order to simplify reviewer's job, job lock/unlock functions and
macros are added as empty prototypes (nop) in patch 1.
They are converted to use the actual job mutex only in the last
patch. In this way we can freely create locking sections
without worrying about deadlocks with the aiocontext lock.

Patch 2 defines what fields in the job structure need protection.
Patches 3-6 are in preparation to the job locks, moving functions
from global to static and introducing helpers.

Patch 7-9 introduce the (nop) job lock into the job API and
its users, and patches 10-13 categorize respectively locked and unlocked 
functions in the job API.

Patches 14-17 take care of protecting job->aio_context, and
finally patch 18 makes the prototypes in patch 1 use the
job_mutex and removes all aiocontext lock at the same time.

Tested this series by running unit tests, qemu-iotests and qtests
(x86_64).

---
v6:
* patch 4 and 6 squashed with patch 19 (enable job lock and
  reduce/remove AioContext lock)
* patch 19: job_unref_locked read the aiocontext inside the
  job lock.

v5:
* just restructured patches a little bit better, as there were
  functions used before they were defined.
* rebased on kwolf/block branch and API split serie

v4:
* move "protected by job_mutex" from patch 2 to 15, where the job_mutex is
  actually added.
* s/aio_co_enter/aio_co_schedule in job.c, and adjust tests accordingly.
* remove job_get_aio_context, add job_set_aio_context. Use "fake rwlock"
  to protect job->aiocontext.
* get rid of useless getters method, namely:
  job_get_status
  job_get_pause_count
  job_get_paused
  job_get_busy
  They are all used only by tests, and such getter is pretty useless.
  Replace with job_lock(); assert(); job_unlock();
* use job lock macros instead of job lock/unlock in unit tests.
* convert also blockjob functions to have _locked
* put the job_lock/unlock patches before the _locked ones
* replace aio_co_enter in job.c and detect change of context

v3:
* add "_locked" suffix to the functions called under job_mutex lock
* rename _job_lock in real_job_lock
* job_mutex is now public, and drivers like monitor use it directly
* introduce and protect job_get_aio_context
* remove mirror-specific APIs and just use WITH_JOB_GUARD
* more extensive use of WITH_JOB_GUARD and JOB_LOCK_GUARD

RFC v2:
* use JOB_LOCK_GUARD and WITH_JOB_LOCK_GUARD
* mu(u)ltiple typos in commit messages
* job API split patches are sent separately in another series
* use of empty job_{lock/unlock} and JOB_LOCK_GUARD/WITH_JOB_LOCK_GUARD
  to avoid deadlocks and simplify the reviewer job
* move patch 11 (block_job_query: remove atomic read) as last

Emanuele Giuseppe Esposito (17):
  job.c: make job_mutex and job_lock/unlock() public
  job.h: categorize fields in struct Job
  job.c: API functions not used outside should be static
  aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED
  job.h: add _locked duplicates for job API functions called with and
without job_mutex
  jobs: protect jobs with job_lock/unlock
  jobs: add job lock in find_* functions
  jobs: use job locks also in the unit tests
  block/mirror.c: use of job helpers in drivers to avoid TOC/TOU
  jobs: rename static functions called with job_mutex held
  job.h: rename job API functions called with job_mutex held
  block_job: rename block_job functions called with job_mutex held
  job.h: define unlocked functions
  commit and mirror: create new nodes using bdrv_get_aio_context, and
not the job aiocontext
  jobs: protect job.aio_context with BQL and job_mutex
  job.c: enable job lock/unlock and remove Aiocontext locks
  block_job_query: remove atomic read

Paolo Bonzini (1):
  job: detect change of aiocontext within job coroutine

 block.c  |  19 +-
 block/backup.c   |   4 +-
 block/commit.c   |   4 +-
 block/mirror.c   |  21 +-
 block/replication.c  |  10 +-
 blockdev.c   | 143 +++
 blockjob.c   | 126 +++---
 include/block/aio-wait.h |  17 +-
 include/block/blockjob.h |  29 +-
 include/qemu/job.h   | 304 +-
 job-qmp.c|  87 ++--
 job.c| 660 +++
 monitor/qmp-cmds.c   |   7 +-
 qemu-img.c   |  41 +-
 tests/unit/test-bdrv-drain.c |  80 ++--
 tests/unit/test-block-iothread.c |   8 +-
 tests/unit/test-blockjob-txn.c   |  32 +-
 tests/unit/test-blockjob.c   | 113 --
 18 files changed, 1036 insertions(+), 669 deletions(-)

-- 
2.31.1