Hi.
Ok. Back to the bug with this patch. The initial problem with this patch is
that "make check" fails.
Please help with subpages.
It turned out that tests use MALLOC_PERTURB_ which is normally off. Who
does not know - this is a way to tell glibc to fill released memory with
some value and then debug accesses to released memory. Some bright mind
made it random what confuses a lot (and btw valgrind found nothing :-/ ).
So I spend some time before figured out how to reproduce it outside of the
qtest thingy.
The tree is qemu.org/master "bd5c51e Michael Roth qemu-char: don't issue
CHR_EVENT_OPEN in a BH" + replayed patches till the one from $subj on top
of it. QEMU is configured as "configure --target-list=x86_64-softmmu".
The magic is:
export MALLOC_PERTURB_=123
nc -l -U ~/qtest-16318.sock &
nc -l -U ~/qtest-16318.qmp &
x86_64-softmmu/qemu-system-x86_64 \
-qtest unix:/home/alexey/qtest-16318.sock,nowait \
-qtest-log /dev/null \
-qmp unix:/home/alexey/qtest-16318.qmp,nowait \
-pidfile ~/qtest-16318.pid -machine accel=qtest -vnc none
Immediate crash at (the very last backtrace in this mail is that crash).
x86_cpu_apic_realize() creates a subpage for IO:
#0 aik_dbg_start (f=f@entry=0x5555558c4b41 "subpage_init",
l=l@entry=0x6a0, mr=mr@entry=0x555556556d30)
at /home/alexey/pcipassthru/qemu-impreza/hw/i386/pc.c:1297
#1 0x0000555555774299 in subpage_init (base=0x0, as=0x5555564a9260) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:1696
#2 register_subpage (d=d@entry=0x555556523d00,
section=section@entry=0x7fffffffd620)
at /home/alexey/pcipassthru/qemu-impreza/exec.c:845
#3 0x000055555577447d in mem_add (listener=0x555556523d08,
section=<optimized out>)
at /home/alexey/pcipassthru/qemu-impreza/exec.c:881
#4 0x00005555557c2d69 in address_space_update_topology_pass
(as=as@entry=0x5555564a9260, adding=adding@entry=0x1, old_view=...,
new_view=...) at /home/alexey/pcipassthru/qemu-impreza/memory.c:751
#5 0x00005555557c64b8 in address_space_update_topology (as=0x5555564a9260)
at /home/alexey/pcipassthru/qemu-impreza/memory.c:766
#6 memory_region_transaction_commit () at
/home/alexey/pcipassthru/qemu-impreza/memory.c:790
#7 0x00005555557c79cd in memory_region_add_subregion_common
(mr=0x555556523c30, offset=offset@entry=0x7e,
subregion=subregion@entry=0x555556550a28) at
/home/alexey/pcipassthru/qemu-impreza/memory.c:1518
#8 0x00005555557c7ae8 in memory_region_add_subregion (mr=<optimized out>,
offset=offset@entry=0x7e,
subregion=subregion@entry=0x555556550a28) at
/home/alexey/pcipassthru/qemu-impreza/memory.c:1527
#9 0x0000555555663995 in sysbus_add_io (dev=dev@entry=0x55555654e700,
addr=addr@entry=0x7e, mem=mem@entry=0x555556550a28)
at /home/alexey/pcipassthru/qemu-impreza/hw/core/sysbus.c:242
#10 0x000055555579cfce in vapic_init (dev=0x55555654e700) at
/home/alexey/pcipassthru/qemu-impreza/hw/i386/kvmvapic.c:707
#11 0x0000555555661651 in device_realize (dev=0x55555654e700,
err=0x7fffffffda40)
at /home/alexey/pcipassthru/qemu-impreza/hw/core/qdev.c:178
#12 0x0000555555662cf3 in device_set_realized (obj=0x55555654e700,
value=0x1, err=0x7fffffffdb50)
at /home/alexey/pcipassthru/qemu-impreza/hw/core/qdev.c:699
#13 0x000055555573358e in property_set_bool (obj=0x55555654e700,
v=<optimized out>, opaque=0x55555653c1f0, name=<optimized out>,
errp=0x7fffffffdb50) at
/home/alexey/pcipassthru/qemu-impreza/qom/object.c:1301
#14 0x0000555555736445 in object_property_set_qobject (obj=0x55555654e700,
value=<optimized out>, name=0x555555896553 "realized",
errp=0x7fffffffdb50) at
/home/alexey/pcipassthru/qemu-impreza/qom/qom-qobject.c:24
#15 0x000055555573525e in object_property_set_bool
(obj=obj@entry=0x55555654e700, value=value@entry=0x1,
name=name@entry=0x555555896553 "realized", errp=errp@entry=0x7fffffffdb50)
at /home/alexey/pcipassthru/qemu-impreza/qom/object.c:852
#16 0x0000555555661c3a in qdev_init (dev=dev@entry=0x55555654e700) at
/home/alexey/pcipassthru/qemu-impreza/hw/core/qdev.c:163
#17 0x0000555555661e91 in qdev_init_nofail (dev=dev@entry=0x55555654e700)
at /home/alexey/pcipassthru/qemu-impreza/hw/core/qdev.c:277
#18 0x0000555555663789 in sysbus_create_varargs
(name=name@entry=0x5555558c73a1 "kvmvapic", addr=addr@entry=0xffffffffffffffff)
at /home/alexey/pcipassthru/qemu-impreza/hw/core/sysbus.c:157
#19 0x00005555557a4ead in sysbus_create_simple (irq=0x0,
addr=0xffffffffffffffff, name=0x5555558c73a1 "kvmvapic")
at /home/alexey/pcipassthru/qemu-impreza/include/hw/sysbus.h:75
#20 apic_init_common (dev=0x555556535350) at
/home/alexey/pcipassthru/qemu-impreza/hw/intc/apic_common.c:311
#21 0x0000555555790fb6 in icc_device_realize (dev=0x555556535350,
errp=0x7fffffffdc80)
at /home/alexey/pcipassthru/qemu-impreza/hw/cpu/icc_bus.c:50
#22 0x0000555555662cf3 in device_set_realized (obj=0x555556535350,
value=0x1, err=0x7fffffffdd90)
at /home/alexey/pcipassthru/qemu-impreza/hw/core/qdev.c:699
#23 0x000055555573358e in property_set_bool (obj=0x555556535350,
v=<optimized out>, opaque=0x555556535610, name=<optimized out>,
errp=0x7fffffffdd90) at
/home/alexey/pcipassthru/qemu-impreza/qom/object.c:1301
#24 0x0000555555736445 in object_property_set_qobject (obj=0x555556535350,
value=<optimized out>, name=0x555555896553 "realized",
errp=0x7fffffffdd90) at
/home/alexey/pcipassthru/qemu-impreza/qom/qom-qobject.c:24
#25 0x000055555573525e in object_property_set_bool
(obj=obj@entry=0x555556535350, value=value@entry=0x1,
name=name@entry=0x555555896553 "realized", errp=errp@entry=0x7fffffffdd90)
at /home/alexey/pcipassthru/qemu-impreza/qom/object.c:852
#26 0x0000555555661c3a in qdev_init (dev=0x555556535350) at
/home/alexey/pcipassthru/qemu-impreza/hw/core/qdev.c:163
#27 0x00005555557d9a7c in x86_cpu_apic_realize (errp=0x7fffffffddd0,
cpu=0x55555653df50)
at /home/alexey/pcipassthru/qemu-impreza/target-i386/cpu.c:2327
#28 x86_cpu_realizefn (dev=0x55555653df50, errp=0x7fffffffde20) at
/home/alexey/pcipassthru/qemu-impreza/target-i386/cpu.c:2397
#29 0x0000555555662cf3 in device_set_realized (obj=0x55555653df50,
value=0x1, err=0x7fffffffdf30)
at /home/alexey/pcipassthru/qemu-impreza/hw/core/qdev.c:699
#30 0x000055555573358e in property_set_bool (obj=0x55555653df50,
v=<optimized out>, opaque=0x55555652e390, name=<optimized out>,
errp=0x7fffffffdf30) at
/home/alexey/pcipassthru/qemu-impreza/qom/object.c:1301
---Type <return> to continue, or q <return> to quit---
#31 0x0000555555736445 in object_property_set_qobject (obj=0x55555653df50,
value=<optimized out>, name=0x555555896553 "realized",
errp=0x7fffffffdf30) at
/home/alexey/pcipassthru/qemu-impreza/qom/qom-qobject.c:24
#32 0x000055555573525e in object_property_set_bool (obj=0x55555653df50,
value=<optimized out>, name=0x555555896553 "realized",
errp=0x7fffffffdf30) at
/home/alexey/pcipassthru/qemu-impreza/qom/object.c:852
#33 0x000055555579f3b0 in pc_new_cpu (cpu_model=<optimized out>,
apic_id=0x0, icc_bridge=<optimized out>, errp=0x7fffffffdf70)
at /home/alexey/pcipassthru/qemu-impreza/hw/i386/pc.c:911
#34 0x00005555557a0fc1 in pc_cpus_init (cpu_model=0x5555558c7a2d "qemu64",
cpu_model@entry=0x0,
icc_bridge=icc_bridge@entry=0x55555652b420) at
/home/alexey/pcipassthru/qemu-impreza/hw/i386/pc.c:964
#35 0x00005555557a129f in pc_init1 (system_memory=0x555556522e60,
system_io=0x555556523c30, ram_size=ram_size@entry=0x8000000,
boot_device=boot_device@entry=0x555555891aaa "cad",
kernel_filename=kernel_filename@entry=0x0,
kernel_cmdline=kernel_cmdline@entry=0x5555558d8fb6 "",
initrd_filename=initrd_filename@entry=0x0,
cpu_model=cpu_model@entry=0x0, pci_enabled=pci_enabled@entry=0x1,
kvmclock_enabled=kvmclock_enabled@entry=0x1)
at /home/alexey/pcipassthru/qemu-impreza/hw/i386/pc_piix.c:98
#36 0x00005555557a1aea in pc_init_pci (args=<optimized out>) at
/home/alexey/pcipassthru/qemu-impreza/hw/i386/pc_piix.c:242
#37 0x00005555555dcea0 in main (argc=<optimized out>, argv=<optimized out>,
envp=<optimized out>)
at /home/alexey/pcipassthru/qemu-impreza/vl.c:4307
This subpage is released later due to some magic which I do not understand:
(gdb) bt
#0 aik_dbg (f=f@entry=0x5555558c4c20 "destroy_page_desc", l=l@entry=0x305)
at /home/alexey/pcipassthru/qemu-impreza/hw/i386/pc.c:1284
#1 0x0000555555773d48 in destroy_page_desc (section_index=<optimized out>)
at /home/alexey/pcipassthru/qemu-impreza/exec.c:773
#2 destroy_l2_mapping (level=0x0, lp=0x555556559e10) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:791
#3 destroy_l2_mapping (lp=0x555556559e10, level=0x0) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:777
#4 0x0000555555773c88 in destroy_l2_mapping (level=0x1, lp=0x555556559610)
at /home/alexey/pcipassthru/qemu-impreza/exec.c:789
#5 destroy_l2_mapping (lp=0x555556559610, level=0x1) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:777
#6 0x0000555555773c88 in destroy_l2_mapping (level=0x2, lp=0x555556558e10)
at /home/alexey/pcipassthru/qemu-impreza/exec.c:789
#7 destroy_l2_mapping (lp=0x555556558e10, level=0x2) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:777
#8 0x0000555555773c88 in destroy_l2_mapping (level=0x3, lp=0x555556523d00)
at /home/alexey/pcipassthru/qemu-impreza/exec.c:789
#9 destroy_l2_mapping (lp=0x555556523d00, level=0x3) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:777
#10 0x0000555555773df8 in destroy_all_mappings (d=0x555556523d00) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:800
#11 mem_begin (listener=0x555556523d08) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:1732
#12 0x00005555557c6168 in memory_region_transaction_commit () at
/home/alexey/pcipassthru/qemu-impreza/memory.c:787
#13 0x00005555557c79cd in memory_region_add_subregion_common
(mr=mr@entry=0x555556522e60, offset=offset@entry=0xfee00000,
subregion=subregion@entry=0x55555652d7b8) at
/home/alexey/pcipassthru/qemu-impreza/memory.c:1518
#14 0x00005555557c7a72 in memory_region_add_subregion_overlap
(mr=0x555556522e60, offset=0xfee00000, subregion=0x55555652d7b8,
priority=<optimized out>) at
/home/alexey/pcipassthru/qemu-impreza/memory.c:1537
#15 0x00005555557a1038 in pc_cpus_init (cpu_model=0x5555558c7a2d "qemu64",
cpu_model@entry=0x0,
icc_bridge=icc_bridge@entry=0x55555652b420) at
/home/alexey/pcipassthru/qemu-impreza/hw/i386/pc.c:976
#16 0x00005555557a129f in pc_init1 (system_memory=0x555556522e60,
system_io=0x555556523c30, ram_size=ram_size@entry=0x8000000,
boot_device=boot_device@entry=0x555555891aaa "cad",
kernel_filename=kernel_filename@entry=0x0,
kernel_cmdline=kernel_cmdline@entry=0x5555558d8fb6 "",
initrd_filename=initrd_filename@entry=0x0,
cpu_model=cpu_model@entry=0x0, pci_enabled=pci_enabled@entry=0x1,
kvmclock_enabled=kvmclock_enabled@entry=0x1)
at /home/alexey/pcipassthru/qemu-impreza/hw/i386/pc_piix.c:98
#17 0x00005555557a1aea in pc_init_pci (args=<optimized out>) at
/home/alexey/pcipassthru/qemu-impreza/hw/i386/pc_piix.c:242
#18 0x00005555555dcea0 in main (argc=<optimized out>, argv=<optimized out>,
envp=<optimized out>)
at /home/alexey/pcipassthru/qemu-impreza/vl.c:4307
(gdb)
And - crash:
#0 object_unref (obj=0xa7a7a7a7a7a7a7a7) at
/home/alexey/pcipassthru/qemu-impreza/qom/object.c:691
#1 0x00005555557c505c in memory_region_unref (mr=<optimized out>) at
/home/alexey/pcipassthru/qemu-impreza/memory.c:1172
#2 0x0000555555775953 in phys_sections_clear () at
/home/alexey/pcipassthru/qemu-impreza/exec.c:826
#3 0x0000555555775999 in core_begin (listener=<optimized out>) at
/home/alexey/pcipassthru/qemu-impreza/exec.c:1738
#4 0x00005555557c6168 in memory_region_transaction_commit () at
/home/alexey/pcipassthru/qemu-impreza/memory.c:787
#5 0x00005555557c79cd in memory_region_add_subregion_common
(mr=mr@entry=0x555556522e60, offset=offset@entry=0xfee00000,
subregion=subregion@entry=0x55555652d7b8) at
/home/alexey/pcipassthru/qemu-impreza/memory.c:1518
#6 0x00005555557c7a72 in memory_region_add_subregion_overlap
(mr=0x555556522e60, offset=0xfee00000, subregion=0x55555652d7b8,
priority=<optimized out>) at
/home/alexey/pcipassthru/qemu-impreza/memory.c:1537
#7 0x00005555557a1038 in pc_cpus_init (cpu_model=0x5555558c7a2d "qemu64",
cpu_model@entry=0x0,
icc_bridge=icc_bridge@entry=0x55555652b420) at
/home/alexey/pcipassthru/qemu-impreza/hw/i386/pc.c:976
#8 0x00005555557a129f in pc_init1 (system_memory=0x555556522e60,
system_io=0x555556523c30, ram_size=ram_size@entry=0x8000000,
boot_device=boot_device@entry=0x555555891aaa "cad",
kernel_filename=kernel_filename@entry=0x0,
kernel_cmdline=kernel_cmdline@entry=0x5555558d8fb6 "",
initrd_filename=initrd_filename@entry=0x0,
cpu_model=cpu_model@entry=0x0, pci_enabled=pci_enabled@entry=0x1,
kvmclock_enabled=kvmclock_enabled@entry=0x1)
at /home/alexey/pcipassthru/qemu-impreza/hw/i386/pc_piix.c:98
#9 0x00005555557a1aea in pc_init_pci (args=<optimized out>) at
/home/alexey/pcipassthru/qemu-impreza/hw/i386/pc_piix.c:242
#10 0x00005555555dcea0 in main (argc=<optimized out>, argv=<optimized out>,
envp=<optimized out>)
at /home/alexey/pcipassthru/qemu-impreza/vl.c:4307
(gdb)
On 06/13/2013 07:02 PM, Alexey Kardashevskiy wrote:
> Fails on qtest_init() in tests/libqtest.c, "Broken pipe". I cannot easily
> see what is wrong here with this patch but it is 100% reproducible on x86_64
> :(
>
>
> On 06/13/2013 04:28 PM, Alexey Kardashevskiy wrote:
>> Hi!
>>
>> I do not know how (yet) but this patch breaks qtest on x86 (I bisected it):
>>
>>
>> make check-qtest V=1
>> QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64
>> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$((RANDOM % 255 + 1))} gtester -k
>> --verbose -m=quick tests/fdc-test tests/ide-test tests/hd-geo-test
>> tests/rtc-test tests/i440fx-test tests/fw_cfg-test
>> TEST: tests/fdc-test... (pid=13049)
>> Broken pipe
>> FAIL: tests/fdc-test
>> TEST: tests/ide-test... (pid=13053)
>> /x86_64/ide/identify:
>> Broken pipe
>> FAIL
>> GTester: last random seed: R02S2f8a8fd53ff256765db44cefb0a920ce
>> (pid=13057)
>> /x86_64/ide/bmdma/setup:
>> Broken pipe
>> FAIL
>> GTester: last random seed: R02S0cec5d222cfd196e6e839e06d7ddde89
>> (pid=13061)
>> /x86_64/ide/bmdma/simple_rw: FAIL
>> GTester: last random seed: R02S46a30a1ccd33dc104919118330810a85
>> (pid=13062)
>> /x86_64/ide/bmdma/short_prdt: FAIL
>> GTester: last random seed: R02S19fdcc95895b870371ed5ddcc8b77eda
>> (pid=13063)
>>
>> [...]
>>
>>
>> On 06/04/2013 10:13 PM, Paolo Bonzini wrote:
>>> Add ref/unref calls at the following places:
>>>
>>> - places where memory regions are stashed by a listener and
>>> used outside the BQL (including in Xen or KVM).
>>>
>>> - memory_region_find callsites
>>>
>>> - creation of aliases and containers (only the aliased/contained
>>> region gets a reference to avoid loops)
>>>
>>> - around calls to del_subregion/add_subregion, where the region
>>> could disappear after the first call
>>>
>>> Signed-off-by: Paolo Bonzini <pbonz...@redhat.com>
>>> ---
>>> exec.c | 6 +++++-
>>> hw/core/loader.c | 1 +
>>> hw/display/exynos4210_fimd.c | 6 ++++++
>>> hw/display/framebuffer.c | 12 +++++++-----
>>> hw/i386/kvmvapic.c | 1 +
>>> hw/misc/vfio.c | 2 ++
>>> hw/virtio/dataplane/hostmem.c | 7 +++++++
>>> hw/virtio/vhost.c | 2 ++
>>> hw/virtio/virtio-balloon.c | 1 +
>>> hw/xen/xen_pt.c | 4 ++++
>>> include/hw/virtio/dataplane/hostmem.h | 1 +
>>> kvm-all.c | 2 ++
>>> memory.c | 20 ++++++++++++++++++++
>>> target-arm/kvm.c | 2 ++
>>> target-sparc/mmu_helper.c | 1 +
>>> xen-all.c | 2 ++
>>> 16 files changed, 64 insertions(+), 6 deletions(-)
>
>
--
Alexey
