Public bug reported:
Fair warning: this might be specific to QTest (specifically its
clock_step) command. This reproducer only works with -accel qtest. Build
with --enable-sanitizers to exit once we hit 1G RSS.
export ASAN_OPTIONS=hard_rss_limit_mb=1000
cat << EOF | ./i386-softmmu/qemu-system-i386 -nographic \
-nodefaults -qtest stdio -accel qtest
writeq 0xfed0000e 0x15151515151515f1
clock_step
clock_step
clock_step
clock_step
writeq 0xfed00100 0x5e90c5be00ff5e9e
writeq 0xfed00109 0xffffe0ff5cfec0ff
clock_step
EOF
On my machine it takes around 10 seconds to reach the RSS limit.
Unfortunately, I can't find a way to tell ASAN to log each malloc to
figure out whats going on, but running the original fuzzing test case
with the libfuzzer -trace_malloc=2 flag, I found that the allocations
happen here:
MALLOC[130968] 0x60300069ac90 32
#0 0x55fa3f615851 in __sanitizer_print_stack_trace
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2683851)
#1 0x55fa3f55fe88 in fuzzer::PrintStackTrace()
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x25cde88)
#2 0x55fa3f5447d6 in fuzzer::MallocHook(void const volatile*, unsigned
long)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x25b27d6)
#3 0x55fa3f61bbb7 in __sanitizer::RunMallocHooks(void const*, unsigned
long)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2689bb7)
#4 0x55fa3f596d75 in __asan::Allocator::Allocate(unsigned long, unsigned
long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2604d75)
#5 0x55fa3f596f7a in __asan::asan_calloc(unsigned long, unsigned long,
__sanitizer::BufferedStackTrace*)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2604f7a)
#6 0x55fa3f60d173 in calloc
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x267b173)
#7 0x7fb300737548 in g_malloc0
(/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x54548)
#8 0x55fa40157689 in async_run_on_cpu
/home/alxndr/Development/qemu/cpus-common.c:163:10
#9 0x55fa409fab83 in hpet_timer
/home/alxndr/Development/qemu/hw/timer/hpet.c:376:9
#10 0x55fa416a5751 in timerlist_run_timers
/home/alxndr/Development/qemu/util/qemu-timer.c:572:9
#11 0x55fa3fcfdac4 in qtest_clock_warp
/home/alxndr/Development/qemu/softmmu/cpus.c:507:9
#12 0x55fa3fd65c35 in qtest_process_command
/home/alxndr/Development/qemu/softmmu/qtest.c:665:9
#13 0x55fa3fd5e128 in qtest_process_inbuf
/home/alxndr/Development/qemu/softmmu/qtest.c:710:9
#14 0x55fa3fd5de67 in qtest_server_inproc_recv
/home/alxndr/Development/qemu/softmmu/qtest.c:817:9
#15 0x55fa4142b64b in qtest_sendf
/home/alxndr/Development/qemu/tests/qtest/libqtest.c:424:5
#16 0x55fa4142c482 in qtest_clock_step_next
/home/alxndr/Development/qemu/tests/qtest/libqtest.c:864:5
#17 0x55fa414b12d1 in general_fuzz
/home/alxndr/Development/qemu/tests/qtest/fuzz/general_fuzz.c:581:17
It doesn't look like we ever exit out of the loop in
timerlist_run_timers, ie timer_list->active_timers is always True.
Info From GDB:
#0 0x0000555558070d31 in address_space_stl_internal (as=0x55555f0e8f20
<address_space_memory>, addr=0x0, val=0x0, attrs=..., result=0x0,
endian=DEVICE_LITTLE_ENDIAN) at
/home/alxndr/Development/qemu/memory_ldst.inc.c:323
#1 0x0000555558071339 in address_space_stl_le (as=0x55555f0e8f20
<address_space_memory>, addr=0x0, val=0x0, attrs=..., result=0x0) at
/home/alxndr/Development/qemu/memory_ldst.inc.c:357
#2 0x000055555a6a6f95 in update_irq (timer=0x61f0000005b8, set=0x1) at
/home/alxndr/Development/qemu/hw/timer/hpet.c:210
#3 0x000055555a6ae55f in hpet_timer (opaque=0x61f0000005b8) at
/home/alxndr/Development/qemu/hw/timer/hpet.c:386
#4 0x000055555c03d178 in timerlist_run_timers (timer_list=0x60b0000528f0) at
/home/alxndr/Development/qemu/util/qemu-timer.c:572
#5 0x000055555c03d6b5 in qemu_clock_run_timers (type=QEMU_CLOCK_VIRTUAL) at
/home/alxndr/Development/qemu/util/qemu-timer.c:586
#6 0x0000555558c3d0c4 in qtest_clock_warp (dest=0x3461864) at
/home/alxndr/Development/qemu/softmmu/cpus.c:507
-Alex
** Affects: qemu
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1888714
Title:
Memory Leak in hpet_timer results in unusable machine
Status in QEMU:
New
Bug description:
Fair warning: this might be specific to QTest (specifically its
clock_step) command. This reproducer only works with -accel qtest.
Build with --enable-sanitizers to exit once we hit 1G RSS.
export ASAN_OPTIONS=hard_rss_limit_mb=1000
cat << EOF | ./i386-softmmu/qemu-system-i386 -nographic \
-nodefaults -qtest stdio -accel qtest
writeq 0xfed0000e 0x15151515151515f1
clock_step
clock_step
clock_step
clock_step
writeq 0xfed00100 0x5e90c5be00ff5e9e
writeq 0xfed00109 0xffffe0ff5cfec0ff
clock_step
EOF
On my machine it takes around 10 seconds to reach the RSS limit.
Unfortunately, I can't find a way to tell ASAN to log each malloc to
figure out whats going on, but running the original fuzzing test case
with the libfuzzer -trace_malloc=2 flag, I found that the allocations
happen here:
MALLOC[130968] 0x60300069ac90 32
#0 0x55fa3f615851 in __sanitizer_print_stack_trace
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2683851)
#1 0x55fa3f55fe88 in fuzzer::PrintStackTrace()
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x25cde88)
#2 0x55fa3f5447d6 in fuzzer::MallocHook(void const volatile*, unsigned
long)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x25b27d6)
#3 0x55fa3f61bbb7 in __sanitizer::RunMallocHooks(void const*, unsigned
long)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2689bb7)
#4 0x55fa3f596d75 in __asan::Allocator::Allocate(unsigned long, unsigned
long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2604d75)
#5 0x55fa3f596f7a in __asan::asan_calloc(unsigned long, unsigned long,
__sanitizer::BufferedStackTrace*)
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x2604f7a)
#6 0x55fa3f60d173 in calloc
(/home/alxndr/Development/qemu/build/i386-softmmu/qemu-fuzz-i386+0x267b173)
#7 0x7fb300737548 in g_malloc0
(/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x54548)
#8 0x55fa40157689 in async_run_on_cpu
/home/alxndr/Development/qemu/cpus-common.c:163:10
#9 0x55fa409fab83 in hpet_timer
/home/alxndr/Development/qemu/hw/timer/hpet.c:376:9
#10 0x55fa416a5751 in timerlist_run_timers
/home/alxndr/Development/qemu/util/qemu-timer.c:572:9
#11 0x55fa3fcfdac4 in qtest_clock_warp
/home/alxndr/Development/qemu/softmmu/cpus.c:507:9
#12 0x55fa3fd65c35 in qtest_process_command
/home/alxndr/Development/qemu/softmmu/qtest.c:665:9
#13 0x55fa3fd5e128 in qtest_process_inbuf
/home/alxndr/Development/qemu/softmmu/qtest.c:710:9
#14 0x55fa3fd5de67 in qtest_server_inproc_recv
/home/alxndr/Development/qemu/softmmu/qtest.c:817:9
#15 0x55fa4142b64b in qtest_sendf
/home/alxndr/Development/qemu/tests/qtest/libqtest.c:424:5
#16 0x55fa4142c482 in qtest_clock_step_next
/home/alxndr/Development/qemu/tests/qtest/libqtest.c:864:5
#17 0x55fa414b12d1 in general_fuzz
/home/alxndr/Development/qemu/tests/qtest/fuzz/general_fuzz.c:581:17
It doesn't look like we ever exit out of the loop in
timerlist_run_timers, ie timer_list->active_timers is always True.
Info From GDB:
#0 0x0000555558070d31 in address_space_stl_internal (as=0x55555f0e8f20
<address_space_memory>, addr=0x0, val=0x0, attrs=..., result=0x0,
endian=DEVICE_LITTLE_ENDIAN) at
/home/alxndr/Development/qemu/memory_ldst.inc.c:323
#1 0x0000555558071339 in address_space_stl_le (as=0x55555f0e8f20
<address_space_memory>, addr=0x0, val=0x0, attrs=..., result=0x0) at
/home/alxndr/Development/qemu/memory_ldst.inc.c:357
#2 0x000055555a6a6f95 in update_irq (timer=0x61f0000005b8, set=0x1) at
/home/alxndr/Development/qemu/hw/timer/hpet.c:210
#3 0x000055555a6ae55f in hpet_timer (opaque=0x61f0000005b8) at
/home/alxndr/Development/qemu/hw/timer/hpet.c:386
#4 0x000055555c03d178 in timerlist_run_timers (timer_list=0x60b0000528f0) at
/home/alxndr/Development/qemu/util/qemu-timer.c:572
#5 0x000055555c03d6b5 in qemu_clock_run_timers (type=QEMU_CLOCK_VIRTUAL) at
/home/alxndr/Development/qemu/util/qemu-timer.c:586
#6 0x0000555558c3d0c4 in qtest_clock_warp (dest=0x3461864) at
/home/alxndr/Development/qemu/softmmu/cpus.c:507
-Alex
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1888714/+subscriptions
