On 230313 0945, Philippe Mathieu-Daudé wrote:
> Hi Alex,
>
> Sorry for the late review, *sigh*.
>
> On 13/3/23 09:24, Alexander Bulekov wrote:
> > Add a flag to the DeviceState, when a device is engaged in PIO/MMIO/DMA.
> > This flag is set/checked prior to calling a device's MemoryRegion
> > handlers, and set when device code initiates DMA. The purpose of this
> > flag is to prevent two types of DMA-based reentrancy issues:
> >
> > 1.) mmio -> dma -> mmio case
> > 2.) bh -> dma write -> mmio case
> >
> > These issues have led to problems such as stack-exhaustion and
> > use-after-frees.
> >
> > Summary of the problem from Peter Maydell:
> > https://lore.kernel.org/qemu-devel/cafeaca_23vc7he3iam-jva6w38lk4hjowae5kcknhprd5fp...@mail.gmail.com
> >
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/62
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/540
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/541
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/556
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/557
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/827
> > Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1282
> >
> > Reviewed-by: Darren Kenny <darren.ke...@oracle.com>
> > Reviewed-by: Stefan Hajnoczi <stefa...@redhat.com>
> > Signed-off-by: Alexander Bulekov <alx...@bu.edu>
> > Acked-by: Peter Xu <pet...@redhat.com>
> > ---
> > include/hw/qdev-core.h | 7 +++++++
> > softmmu/memory.c | 17 +++++++++++++++++
> > softmmu/trace-events | 1 +
> > 3 files changed, 25 insertions(+)
> >
> > diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> > index bd50ad5ee1..7623703943 100644
> > --- a/include/hw/qdev-core.h
> > +++ b/include/hw/qdev-core.h
> > @@ -162,6 +162,10 @@ struct NamedClockList {
> > QLIST_ENTRY(NamedClockList) node;
> > };
> > +typedef struct {
> > + bool engaged_in_io;
>
> Do you plan to add more fields?
Not right now, but maybe some need will come up.
> > +} MemReentrancyGuard;
> > +
> > /**
> > * DeviceState:
> > * @realized: Indicates whether the device has been fully constructed.
> > @@ -194,6 +198,9 @@ struct DeviceState {
> > int alias_required_for_version;
> > ResettableState reset;
> > GSList *unplug_blockers;
> > +
> > + /* Is the device currently in mmio/pio/dma? Used to prevent
> > re-entrancy */
> > + MemReentrancyGuard mem_reentrancy_guard;
>
> At this point I'm not sure anymore this is a device or MR property.
It's designed to be an MR property. If it were MR specific, it wouldn't
handle the BH -> DMA case, or this one, where there are two MRs (doorbell
and oper) involed.
https://gitlab.com/qemu-project/qemu/-/issues/540
>
> > };
> > struct DeviceListener {
> > diff --git a/softmmu/memory.c b/softmmu/memory.c
> > index 4699ba55ec..57bf18a257 100644
> > --- a/softmmu/memory.c
> > +++ b/softmmu/memory.c
> > @@ -533,6 +533,7 @@ static MemTxResult access_with_adjusted_size(hwaddr
> > addr,
> > uint64_t access_mask;
> > unsigned access_size;
> > unsigned i;
> > + DeviceState *dev = NULL;
> > MemTxResult r = MEMTX_OK;
> > if (!access_size_min) {
> > @@ -542,6 +543,19 @@ static MemTxResult access_with_adjusted_size(hwaddr
> > addr,
> > access_size_max = 4;
> > }
> > + /* Do not allow more than one simultanous access to a device's IO
> > Regions */
>
> Typo "simultaneous".
>
> 1/ access_with_adjusted_size() is complex enough and we are having hard
> time getting it right. I'd prefer we don't intermix size adjustment
> and re-entrancy check in the same function. This check could belong
> to the callers.
>
Would moving the code within this function to keep it separate from the
size adjustment be good enough? Otherwise we would end up with duplicate
code in the read/write callers.
The size-adjustment seems to be orthogonal (the MR won't change)?
> 2/ I'm not keen on calling QOM object_dynamic_cast() in this hot path;
> and mixing QDev API within MR one. At least, can we cache this value
> once in memory_region_do_init() since we have access to @owner?
>
Sounds like a good idea. Is it ever possible for the owner/owner's
address to change?
Thanks
-Alex
> > + if (mr->owner &&
> > + !mr->ram_device && !mr->ram && !mr->rom_device && !mr->readonly) {
> > + dev = (DeviceState *) object_dynamic_cast(mr->owner, TYPE_DEVICE);
> > + if (dev) {
> > + if (dev->mem_reentrancy_guard.engaged_in_io) {
> > + trace_memory_region_reentrant_io(get_cpu_index(), mr,
> > addr, size);
> > + return MEMTX_ERROR;
>
> MEMTX_ERROR is device-specific, I'm not sure it is right to return it
> from this generic path. Maybe you meant MEMTX_ACCESS_ERROR?
>
> > + }
> > + dev->mem_reentrancy_guard.engaged_in_io = true;
> > + }
> > + }
> > +
> > /* FIXME: support unaligned access? */
> > access_size = MAX(MIN(size, access_size_max), access_size_min);
> > access_mask = MAKE_64BIT_MASK(0, access_size * 8);
> > @@ -556,6 +570,9 @@ static MemTxResult access_with_adjusted_size(hwaddr
> > addr,
> > access_mask, attrs);
> > }
> > }
> > + if (dev) {
> > + dev->mem_reentrancy_guard.engaged_in_io = false;
> > + }
> > return r;
> > }
>
