On Tue, Jan 26, 2021 at 12:59 PM Jukka Kaartinen <jukka.kaarti...@unikie.com> wrote:
> > > On Tue, Jan 26, 2021 at 2:54 AM Stefano Stabellini <sstabell...@kernel.org> > wrote: > >> On Sat, 23 Jan 2021, Jukka Kaartinen wrote: >> > Thanks for the response! >> > >> > On Sat, Jan 23, 2021 at 2:27 AM Stefano Stabellini < >> sstabell...@kernel.org> wrote: >> > + xen-devel, Roman, >> > >> > >> > On Fri, 22 Jan 2021, Jukka Kaartinen wrote: >> > > Hi Stefano, >> > > I'm Jukka Kaartinen a SW developer working on enabling >> hypervisors on mobile platforms. One of our HW that we use on >> > development is >> > > Raspberry Pi 4B. I wonder if you could help me a bit :). >> > > >> > > I'm trying to enable the GPU with Xen + Raspberry Pi for >> > dom0. >> https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=232323#p1797605 >> > > >> > > I got so far that GPU drivers are loaded (v3d & vc4) without >> errors. But now Xen returns error when X is starting: >> > > (XEN) traps.c:1986:d0v1 HSR=0x93880045 pc=0x00007f97b14e70 >> gva=0x7f7f817000 gpa=0x0000401315d000 >> > > I tried to debug what causes this and looks >> like find_mmio_handler cannot find handler. >> > > (See more here: >> https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=232323&start=25#p1801691 >> ) >> > > >> > > Any ideas why the handler is not found? >> > >> > >> > Hi Jukka, >> > >> > I am glad to hear that you are interested in Xen on RaspberryPi >> :-) I >> > haven't tried the GPU yet, I have been using the serial only. >> > Roman, did you ever get the GPU working? >> > >> > >> > The error is a data abort error: Linux is trying to access an >> address >> > which is not mapped to dom0. The address seems to be >> 0x401315d000. It is >> > a pretty high address; I looked in device tree but couldn't spot >> it. >> > >> > >From the HSR (the syndrom register) it looks like it is a >> translation >> > fault at EL1 on stage1. As if the Linux address mapping was wrong. >> > Anyone has any ideas how this could happen? Maybe a >> reserved-memory >> > misconfiguration? >> > >> > I had issues with loading the driver in the first place. Apparently >> swiotlb is used, maybe it can cause this. I also tried to enable CMA. >> > config.txt: >> > dtoverlay=vc4-fkms-v3d,cma=320M@0x0-0x40000000 >> > gpu_mem=128 >> >> Also looking at your other reply and the implementation of >> vc4_bo_create, it looks like this is a CMA problem. >> >> It would be good to run a test with the swiotlb-xen disabled: >> >> diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c >> index 467fa225c3d0..2bdd12785d14 100644 >> --- a/arch/arm/xen/mm.c >> +++ b/arch/arm/xen/mm.c >> @@ -138,8 +138,7 @@ void xen_destroy_contiguous_region(phys_addr_t >> pstart, unsigned int order) >> static int __init xen_mm_init(void) >> { >> struct gnttab_cache_flush cflush; >> - if (!xen_initial_domain()) >> - return 0; >> + return 0; >> xen_swiotlb_init(1, false); >> >> cflush.op = 0; >> > With this change the kernel is not booting up. (btw. I'm using USB SSD for > my OS.) > [ 0.071081] bcm2835-dma fe007000.dma: Unable to set DMA mask > [ 0.076277] bcm2835-dma fe007b00.dma: Unable to set DMA mask > (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=25: not implemented > (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented > [ 0.592695] pci 0000:00:00.0: Failed to add - passthrough or MSI/MSI-X > might fail! > (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented > [ 0.606819] pci 0000:01:00.0: Failed to add - passthrough or MSI/MSI-X > might fail! > [ 1.212820] usb 1-1: device descriptor read/64, error 18 > [ 1.452815] usb 1-1: device descriptor read/64, error 18 > [ 1.820813] usb 1-1: device descriptor read/64, error 18 > [ 2.060815] usb 1-1: device descriptor read/64, error 18 > [ 2.845548] usb 1-1: device descriptor read/8, error -61 > [ 2.977603] usb 1-1: device descriptor read/8, error -61 > [ 3.237530] usb 1-1: device descriptor read/8, error -61 > [ 3.369585] usb 1-1: device descriptor read/8, error -61 > [ 3.480765] usb usb1-port1: unable to enumerate USB device > > Traces stop here. I could try with a memory card. Maybe it makes a > difference. > > >> >> It is going to be fine just to boot Dom0 and DomUs without PV drivers. >> Also, can you post the device tree that you are using here? Just in case >> there is an issue with Xen parsing any possible /reserved-memory nodes >> with CMA info that need to be passed on to Dom0. > > Here is the device dumped from command line: > dtc -I fs /proc/device-tree > > https://drive.google.com/file/d/17u18dJHxRfbGZMtRXIwtLVZZfMj9KwN-/view?usp=sharing > > Here is log from XEN when it goes through the device tree: https://drive.google.com/file/d/1jvT3ZNJeXHfxPOffiaRSRg8FPwQHS1mb/view?usp=sharing > >> >> > > p.s. >> > > While testing I found issue with Xen master branch and your >> patch: xen/rpi4: implement watchdog-based reset >> > > >> > > Looks like black listing the bcm2835-pm >> > > @@ -37,12 +41,69 @@ static const struct dt_device_match >> rpi4_blacklist_dev[] __initconst = >> > > * The aux peripheral also shares a page with the aux UART. >> > > */ >> > > DT_MATCH_COMPATIBLE("brcm,bcm2835-aux"), >> > > + /* Special device used for rebooting */ >> > > + DT_MATCH_COMPATIBLE("brcm,bcm2835-pm"), >> > > >> > > will prevent v3d driver to locate phandle. I think it will use >> the same resource: >> > > pm: watchdog@7e100000 { >> > > compatible = "brcm,bcm2835-pm", "brcm,bcm2835-pm-wdt"; >> > > #power-domain-cells = <1>; >> > > #reset-cells = <1>; >> > > reg = <0x7e100000 0x114>, >> > > <0x7e00a000 0x24>, >> > > <0x7ec11000 0x20>; >> > > clocks = <&clocks BCM2835_CLOCK_V3D>, >> > > <&clocks BCM2835_CLOCK_PERI_IMAGE>, >> > > <&clocks BCM2835_CLOCK_H264>, >> > > <&clocks BCM2835_CLOCK_ISP>; >> > > clock-names = "v3d", "peri_image", "h264", "isp"; >> > > system-power-controller; >> > > >> > > }; >> > >> > Yeah, I imagine it could be possible. Can you post the error >> message you >> > are seeing from the v3d driver? >> > >> > This is the error: >> > [ 0.069682] OF: /v3dbus/v3d@7ec04000: could not find phandle >> > [ 0.074828] OF: /v3dbus/v3d@7ec04000: could not find phandle >> > v3d driver is not loaded. >> > >> > -- >> > Br, >> > Jukka Kaartinen >> > >> > > > > > -- > Br, > Jukka Kaartinen > -- Br, Jukka Kaartinen
