On Sat, Oct 3, 2015 at 5:21 PM, Guenter Roeck <li...@roeck-us.net> wrote: > On 10/03/2015 02:31 PM, Peter Crosthwaite wrote: >> >> Hi, >> >> I have done an audit of the ARMv7 boards to see what can boot a >> vanilla linux kernel. The basic approach is to build ARM >> multi_v7_defconfig kernel and boot QEMU using the DTBs built out by >> the kernel. The intersection of what mainline Linux has a DTB for and >> what QEMU models is tested. The boards that do/should work include: >> >> vexpress-a9 >> vexpress-a15 >> smdkc210 (exynos) >> xilinx-zynq-a9 >> cubieboard (allwinner) >> highbank >> midway >> virt >> >> Have I missed anything? >> >> The basic rule is, no source code, config, or DTB edits allowed (i.e. >> what is a day0 user of QEMU+Linux going to experience?). >> >> So going board-by-board, these are the issues. >> >> highbank/midway: >> >> The kernel expects a secure monitor to be present which does at-least >> two things: >> >> * PSCI >> * Some cache-maintainance ops. >> >> QEMU has PSCI support, but the 0.1 command codings specified in the >> DTB are different to QEMUs 0.1 encodings. So I switched monitor mode >> for the CPUs on (it was conservatively set to off by the original >> authors of the QEMU ARM monitor support) and turned PSCI on. I had to >> write some patches to remap the PSCI encodings to highbank's custom >> ones and then the kernel at least entry-points all 4 cpus. It then >> deadlocks on something that I am yet to figure out. So the PSCI SMP >> stuff is shelved. >> >> The cache-maintenance is a bigger issue, that stops even a single core >> boot. The kernel cache controller driver is doing an smc that QEMU >> goes haywire on (as there simply is no firmware to catch the smc). But >> caches in QEMU are a nop, so I wrote some monitor firmware that just >> erets the smc without action and it works. >> >> I am using Sata (sysbus-ahci) for the boot media which works straight >> out of the box. >> >> So single core highbank works with the dummy monitor firmware and the >> monitor mode restrictor removed. >> > I have highbank working with multi_v7_defconfig. It generates a warning, > though. > > WARNING: CPU: 0 PID: 0 at arch/arm/mach-highbank/highbank.c:60 > highbank_l2c310_write_sec+0x34/0x58() > [ 0.000000] Highbank L2C310: ignoring write to reg 0x104 >
Yes I am ignoring this. > I think this is a kernel problem. > > It also generates an SMC request. > > SMC cmd 0x102 val 0x1 > > My "fix" for that is to simply log and ignore all SMC requests in qemu. > Not really feasible for production qemu, but after all I am testing the > kernel, > not qemu. That also helped for various other HW. Maybe your monitor firmware > can help me get rid of that. > Yes. My change doesn't touch target-arm and keeps it all in highbank.c. > On top of that, I had to enable EL3. I don't remember why, just that it was > needed. > To bring your SMC printf into play most probably. It would just undef instruction otherwise. I have the same change in my tree. > >> cubieboard: >> >> The kernel expects a system level clock controller device that isn't >> modelled. A dummy model which hardcodes all registers to a know reset >> value and just lets the kernel read and write whatever it wants >> resolves. I'm not 100% sure whether this is needed but the kernel >> prints multiple errors with many pages of traces without. So it is at >> least needed to reduce the dmesg noise. >> >> QEMU cubieboard has no usable storage media, but the real hardware >> does have AHCI sata. I added sysbus-ahci at the right place but turns >> out the SATA controller has some custom power/clock (not really >> sure??) registers specific to this SoC. It sets/clears bits then polls >> them back expecting them to change to the other value asynchronously. >> The kernel device probe then times-out. So I subclassed sysbus-ahci >> and added the missing registers and forced the polled registers to the >> "I'm done" state. It works. >> >> I am using meta-sunxi Yocto-layer to build out the allwinner custom >> kernel/rootfs etc, and with the clock and Sata changes I get a boot. >> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC >> messages are around the point of failure which is not modelled in >> QEMU, so that is suspect. >> >> So with a dummy clock-controller and a slightly modified Sata >> controller (as a new device) cubieboard progresses, but it is still >> stuck on something that looks RTC related. >> >> xilinx-zynq-a9: >> >> The kernel wants to use cpufreq to set the CPU frequency. The kernel >> assumes ownership of the CPU clock divider but not the PLLs. QEMU uses >> the HW default for the silicon (x26) for the PLL which is incompatible >> with the dtb-set CPU frequency. The kernel boot asserts this and >> crashes. The ideal solution is to provide some sort of pre-boot >> firmware to set it to a compatible value. In real HW, this is the FSBL >> bootloader. >> >> TBH I think this is a kernel bug, as since cpufreq can be deconfigured >> it is ultimately optional. This means it shouldn't be a showstopper on >> a boot. The kernel should just switch off cpufreq and proceed and >> behave as if the feature was never configured in the kernel. >> >> There is also a critical bug in the SLCR model (my bad) where write to >> registers are ignored. I have a patch. >> >> More reading about the SLCR issues here (thanks to Guenter for getting >> the ball rolling here): >> >> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03374.html >> > Seemed to be going nowhere, though :-(. > I'll get there :) >> I have a patch with sets up the needed SLCR values from software but >> it needs more work. >> >> The kernel also expect the ADC device which is absent. Thanks to >> Guenter who patched this in which is under review: >> >> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03375.html >> > Still working on the review comments. > > >> I am using SD card for the rootfs media. SD has a bug where it cannot >> handle a backing file of a non-round size. I am just passing the ext3 >> image in unchanged (so unpadded) to QEMU and QEMU rounds down the >> image to the SD block size (512k!!), so I am losing us to 512k off the >> end of my ext3 filesystem. I have a hacky patch that changes this to >> round-up. >> >> With the minimal firmware, slcr bugfix, Guenters ADC, the SD hack a >> vanilla Zynq will be in good shape. >> >> virt: >> >> Virt largely works, but there are no immediately obvious >> storage/network options that are supported by the kernel. So I have to >> make some exceptions to the kernel config rule, that is, I add: >> >> CONFIG_VIRTIO_PCI=y" >> CONFIG_VIRTIO_BLK=y" >> CONFIG_VIRTIO_NET=y" >> >> To get disks and network for the virt board. >> >> Could we add these configs to mainline Linux's defconfig? Note, we >> don't need to add a DTS for this board, as QEMU's bootloader will >> generate and provide it for us. >> >> There is the known highmem PCI issue that is currently under discussion: >> >> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html >> >> So aside from the missing VIRTIO kernel configs and highmem bug, virt >> is in good shape. >> >> vexpress: >> >> vexpress boots up to rootfs probing, however the only storage media >> that seems to be supported is flash, which doesn't have the needed >> kernel configs to boot from (not 100% sure on that but I gave up >> quickly). What are people who use this board in real hardware doing? >> It may just work best with initrd style boots. >> > I use it with > -drive file=rootfs.ext3,format=raw,if=sd \ > -append "root=/dev/mmcblk0 ..." > I now have the same. Did you run into the 512k bug? or are your padding your rootfs images? > with multi_v7_defconfig and vexpress-v2p-ca9.dtb. This works for both > vexpress-a9 and vexpress-a15 (with vexpress-v2p-ca15-tc1.dtb). > >> exynos: >> >> QEMU models an Exynos skdkc210 but there is only a 310 available in >> QEMU dtbs. QEMU exynos however does not model a lot on the board level >> so we can largely get away with booting 310 dtb on 210. >> > I needed a couple of fixes to get the smdkc210 machine to work with > exynos4210-smdkv310.dtb. > > - CMU support was submitted a long time ago by Maksim Kozlov, but never > accepted. I took the code and merged the absolutely necessary parts of > (I dropped clock support). > - exynos4210_mct needs a one-line bug fix to initialize a timer before > it is used. > - dma support for exynos4 must be enabled (pl330 must be instantiated). > - If I recall correctly, smdkc210 also needed SMC support. > > I have to change the configuration file, though. QEMU currently does not > implement exynos cpuidle, so I have to disable CONFIG_CPU_IDLE and > CONFIG_ARM_EXYNOS_CPUIDLE. > > I test both multi_v7_defconfig and exynos_defconfig with -M smdkc210. > Are your changes only needed for the exynos_defconfig? >> Similar to vexpress, the exynos board has a lack of usable boot media. >> The board does model USB (EHCI), however the DTB disables it as the >> smdk310 board does not support it. So the alternative would be to use >> one of the more full-featured exynos DTBs (with way more peripherals) >> from mainline Linux, but the kernel is liable to then probe all sorts >> of missing hardware. >> >> Exynos SDHCI support was on list for a few revs and quite some time, >> merging it and using SD may solve the problem. >> >> I have not attempted the SMP support. >> > I run smdkc210 with "-smp 2". If I recall correctly, this is mandatory, > but I may be wrong. > Makes sense. I saw a warning to that effect. >> Unlike all other boards, Exynos is quite consuming of RAM. All other >> boards work with Yocto's default of 128MB but Exynos crashes unless >> you jack it up. So users need to be careful of the -m option. >> >> realview: >> >> Realview is not tested, but I discuss it here as it is a bit of a red >> herring. The kernel has dtb support for the realview-1176 board, >> however this is not modelled by QEMU. I creating a Realview variant >> with 1176 in QEMU, however the memory maps of all the peripherals were >> mismatched. >> >> Turns, out, QEMU is is modelling the realview FPGA emulation platform >> which has a different peripheral map to the 1176 realview board >> proper. So QEMU doesn't support the realview boards, just the FPGA >> emulation platform, for which I can't find a kernel dtb. Should we >> upstream one with some dtsi's for the CPU variation supported in QEMU? >> >> So Realview does not work within my testing parameters with no >> immediate prospects. >> > I have realview-pb-a8, realview-eb-mpcore, and realview-eb working, > but not with a default configuration. > So if it can be reduced to a "ARCH=arm make some_defconfig; make -j;" I think we should go for it. Your repo has what looks like QEMU-specific realview defconfigs. Upstream has a realview_defconfig, is this likely to work or are there issues? >> how I am testing: >> >> So this is all powered by the Yocto project (Poky). I got some good >> help from Nathan Rossi. I have a poky fork which changes the qemuarm >> target to build my mainline (4.2.1) kernel (multi_v7_defconfig + >> VIRTIO), all the DTBs and a usable rootfs. You can then specify the >> type of machine (from the list above) to yocto's runqemu command. The >> command sets up boot media and network automatically. >> >> To use it: >> >> git clone https://github.com/pcrost/poky.git >> cd poky >> source ./oe-init-build-env >> MACHINE=qemuarm bitbake core-image-minimal #this takes a while >> QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp >> > Just for reference, my code is at github. > > https://github.com/groeck/linux-build-test > https://github.com/groeck/qemu > There's some cool stuff in your QEMU branch. I made some high level comments on the github. The IMX timer will need a bit of a rebase following Jean-Christophe's refactoring. Regards, Peter > Guenter >
