On 15/03/2022 13:25, Anup Patel wrote:
On Tue, Mar 15, 2022 at 5:47 PM Ralf Ramsauer
<ralf.ramsa...@oth-regensburg.de> wrote:
On 15/03/2022 09:33, Anup Patel wrote:
On Tue, Mar 15, 2022 at 12:18 PM Alistair Francis <alistai...@gmail.com> wrote:
On Sun, Mar 13, 2022 at 12:12 PM Ralf Ramsauer
<ralf.ramsa...@oth-regensburg.de> wrote:
Hi,
I'm trying to run Linux/QEMU+KVM inside an emulated
qemu-system-riscv64 VM (x86 host). On latest&greatest QEMU (1416688c53),
I run Linux inside QEMU. On host side:
qemu-system-riscv64 -nographic \
-machine virt \
-cpu 'rv64,h=true' \
-smp 8 -m 8G \
-bios $latest_opensbi \
-kernel $latest_upstream_linux \
[…snip…]
Inside that machine, I boot Linux with KVM enabled:
[…]
[ 0.228939] kvm [1]: hypervisor extension available
[ 0.229000] kvm [1]: using Sv48x4 G-stage page table format
[ 0.229033] kvm [1]: VMID 14 bits available
[…]
KVM seems to load correctly. Inside that machine, I compiled QEMU from
the same sources with the KVM accelerator enabled. When I try to start
QEMU with KVM enabled, I get
./qemu-system-riscv64 -nographic \
-monitor none \
-machine virt \
-smp 1 -m 1G \
-bios ./pc-bios/opensbi-riscv64-generic-fw_dynamic.bin \
-serial stdio \
-enable-kvm
The QEMU RISC-V KVM support is quite new so I haven't got a chance
to play with it.
Btw, the kernel's KVM selftests seem to run.
Nevertheless, we should not pass any firmware when using KVM
so can you try "-bios none" ?
I have no kernel specified in the guest, so I'd then expect to boot into
an completely empty machine (besides the tiny startup stub in the MROM).
However, I still get the exact same error as mentioned in my initial mail.
When KVM is enabled and the kernel is not provided, there is simply
nothing in Guest RAM (0x80000000) or ROM (0x00000000) which means
you are booting garbage.
In all machine models ROM is at 0x1000, afaict. This is also the
DEFAULT_RESETVEC of Qemu/RiscV. At least those few instructions should
execute.
Cf.
https://github.com/qemu/qemu/blob/master/hw/riscv/boot.c#L285
and
https://github.com/qemu/qemu/blob/master/hw/riscv/virt.c#L1371
Again, without "-enable-kvm", everything works as expected (i.e., I
don't see any output, because there's basically no guest code to execute).
Strange, but even in this case you might be running garbage.
Yeah, and in case of !kvm, booting garbage results in no output. With
kvm enabled, it faults.
Just out of curiosity: if you don't pass a bios to the machine, what
would then the boot sequence be?
For QEMU KVM:
1) There is no M-mode
2) Guest directly boots in S-mode
3) The SBI services are provided by in-kernel KVM module
Aaah! I understand. Thanks for the clarification.
You need to specify the Guest kernel using "-kernel" parameter when
using QEMU KVM.
Yikes, that one works!
qemu-system-riscv64 -monitor none \
-machine virt -smp 1 -m 1G \
-bios none -nographic -serial stdio \
-enable-kvm -kernel ./Image
[ 0.000000] Linux version 5.17.0-rc7-00235-gaad611a868d1
(ralf@redstone02) (riscv64-linux-gnu-gcc (Ubuntu 10.3.0-8ubuntu1)
10.3.0, GNU ld (GNU Binutils for Ubuntu) 2.37) #10 SMP Sun Mar 13
01:11:32 CET 2022
[ 0.000000] Machine model: riscv-virtio,qemu
[ 0.000000] efi: UEFI not found.
[ 0.000000] Zone ranges:
[ 0.000000] DMA32 [mem 0x0000000080000000-0x00000000bfffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000080000000-0x00000000bfffffff]
[ 0.000000] Initmem setup node 0 [mem
0x0000000080000000-0x00000000bfffffff]
[ 0.000000] SBI specification v0.2 detected
[ 0.000000] SBI implementation ID=0x3 Version=0x51100
[ 0.000000] SBI TIME extension detected
[ 0.000000] SBI IPI extension detected
[ 0.000000] SBI RFENCE extension detected
[ 0.000000] SBI HSM extension detected
[…]
I thought it'd be better to get OpenSBI running before even trying to
pass a kernel. I didn't know that SBI services are emulated/moderated(?)
by KVM, and that there must be no bios specified when using KVM. Will
have a closer look at the code.
Anup, thank you!
Ralf
Regards,
Anup
Thanks
Ralf
Regards,
Anup
[ 4860.559194] kvm [9942]: VCPU exit error -95
[ 4860.584262] kvm [9942]: SEPC=0x0 SSTATUS=0x200004120 HSTATUS=0x2002001c0
[ 4860.586839] kvm [9942]: SCAUSE=0x14 STVAL=0x0 HTVAL=0x0 HTINST=0x0
on the ringbuffer, together with a register dump of qemu [1] on the
console. Needless to say, but without -enable-kvm, it works fine.
As far as I see that, SCAUSE=0x14 reports a 'Instruction Guest PF',
which would be kind of correct, if the guest's PC really tries to
execute at phys 0x0. DRAM of the 'virt' machine definition should start
at 0x80000000, where OpenSBI resides. So I wonder if an erroneous reset
PC might be the culprit…
Before digging deeper into that issue, I wanted to ask if Qemu/KVM
inside an emulated riscv64+H-extension is actually supported, or if this
is a known bug and has some ongoing work.
RISC-V KVM should work. I haven't had a chance to try it myself though.
I have CCed two people who hopefully can help.
Alistair
Thanks
Ralf
[1]
pc 0000000000000000
mhartid 0000000000000000
mstatus 0000000200000000
mip 0000000000000000
mie 0000000000000000
mideleg 0000000000000000
medeleg 0000000000000000
mtvec 0000000000000000
stvec 0000000000000000
mepc 0000000000000000
sepc 0000000000000000
mcause 0000000000000000
scause 0000000000000000
mtval 0000000000000000
stval 0000000000000000
mscratch 0000000000000000
sscratch 0000000000000000
satp 0000000000000000
x0/zero 0000000000000000 x1/ra 0000000000000000 x2/sp
0000000000000000 x3/gp 0000000000000000
x4/tp 0000000000000000 x5/t0 0000000000000000 x6/t1
0000000000000000 x7/t2 0000000000000000
x8/s0 0000000000000000 x9/s1 0000000000000000 x10/a0
0000000000000000 x11/a1 00000000bf000000
x12/a2 0000000000000000 x13/a3 0000000000000000 x14/a4
0000000000000000 x15/a5 0000000000000000
x16/a6 0000000000000000 x17/a7 0000000000000000 x18/s2
0000000000000000 x19/s3 0000000000000000
x20/s4 0000000000000000 x21/s5 0000000000000000 x22/s6
0000000000000000 x23/s7 0000000000000000
x24/s8 0000000000000000 x25/s9 0000000000000000 x26/s10
0000000000000000 x27/s11 0000000000000000
x28/t3 0000000000000000 x29/t4 0000000000000000 x30/t5
0000000000000000 x31/t6 0000000000000000
