On 10/03/2015 05:31 PM, Peter Crosthwaite wrote: > Hi, > > I have done an audit of the ARMv7 boards to see what can boot a > vanilla linux kernel. The basic approach is to build ARM > multi_v7_defconfig kernel and boot QEMU using the DTBs built out by > the kernel. The intersection of what mainline Linux has a DTB for and > what QEMU models is tested. The boards that do/should work include: > > vexpress-a9 > vexpress-a15 > smdkc210 (exynos) > xilinx-zynq-a9 > cubieboard (allwinner) > highbank > midway > virt > > Have I missed anything? > > The basic rule is, no source code, config, or DTB edits allowed (i.e. > what is a day0 user of QEMU+Linux going to experience?). > > So going board-by-board, these are the issues. > > highbank/midway: > > The kernel expects a secure monitor to be present which does at-least > two things: > > * PSCI > * Some cache-maintainance ops. > > QEMU has PSCI support, but the 0.1 command codings specified in the > DTB are different to QEMUs 0.1 encodings. So I switched monitor mode > for the CPUs on (it was conservatively set to off by the original > authors of the QEMU ARM monitor support) and turned PSCI on. I had to > write some patches to remap the PSCI encodings to highbank's custom > ones and then the kernel at least entry-points all 4 cpus. It then > deadlocks on something that I am yet to figure out. So the PSCI SMP > stuff is shelved. > > The cache-maintenance is a bigger issue, that stops even a single core > boot. The kernel cache controller driver is doing an smc that QEMU > goes haywire on (as there simply is no firmware to catch the smc). But > caches in QEMU are a nop, so I wrote some monitor firmware that just > erets the smc without action and it works. > > I am using Sata (sysbus-ahci) for the boot media which works straight > out of the box. > > So single core highbank works with the dummy monitor firmware and the > monitor mode restrictor removed. > > cubieboard: > > The kernel expects a system level clock controller device that isn't > modelled. A dummy model which hardcodes all registers to a know reset > value and just lets the kernel read and write whatever it wants > resolves. I'm not 100% sure whether this is needed but the kernel > prints multiple errors with many pages of traces without. So it is at > least needed to reduce the dmesg noise. > > QEMU cubieboard has no usable storage media, but the real hardware > does have AHCI sata. I added sysbus-ahci at the right place but turns > out the SATA controller has some custom power/clock (not really > sure??) registers specific to this SoC. It sets/clears bits then polls > them back expecting them to change to the other value asynchronously. > The kernel device probe then times-out. So I subclassed sysbus-ahci > and added the missing registers and forced the polled registers to the > "I'm done" state. It works. >
I'm looking into the cubieboard now. Is our emulation based on any particular model? (1-4?) I'm trying to see if I can find anything that resembles a spec to see what kind of registers this SoC has for its SATA controller. Feel free to point me towards your patches, too. > I am using meta-sunxi Yocto-layer to build out the allwinner custom > kernel/rootfs etc, and with the clock and Sata changes I get a boot. > But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC > messages are around the point of failure which is not modelled in > QEMU, so that is suspect. > > So with a dummy clock-controller and a slightly modified Sata > controller (as a new device) cubieboard progresses, but it is still > stuck on something that looks RTC related. > > xilinx-zynq-a9: > > The kernel wants to use cpufreq to set the CPU frequency. The kernel > assumes ownership of the CPU clock divider but not the PLLs. QEMU uses > the HW default for the silicon (x26) for the PLL which is incompatible > with the dtb-set CPU frequency. The kernel boot asserts this and > crashes. The ideal solution is to provide some sort of pre-boot > firmware to set it to a compatible value. In real HW, this is the FSBL > bootloader. > > TBH I think this is a kernel bug, as since cpufreq can be deconfigured > it is ultimately optional. This means it shouldn't be a showstopper on > a boot. The kernel should just switch off cpufreq and proceed and > behave as if the feature was never configured in the kernel. > > There is also a critical bug in the SLCR model (my bad) where write to > registers are ignored. I have a patch. > > More reading about the SLCR issues here (thanks to Guenter for getting > the ball rolling here): > > https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03374.html > > I have a patch with sets up the needed SLCR values from software but > it needs more work. > > The kernel also expect the ADC device which is absent. Thanks to > Guenter who patched this in which is under review: > > https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03375.html > > I am using SD card for the rootfs media. SD has a bug where it cannot > handle a backing file of a non-round size. I am just passing the ext3 > image in unchanged (so unpadded) to QEMU and QEMU rounds down the > image to the SD block size (512k!!), so I am losing us to 512k off the > end of my ext3 filesystem. I have a hacky patch that changes this to > round-up. > > With the minimal firmware, slcr bugfix, Guenters ADC, the SD hack a > vanilla Zynq will be in good shape. > > virt: > > Virt largely works, but there are no immediately obvious > storage/network options that are supported by the kernel. So I have to > make some exceptions to the kernel config rule, that is, I add: > > CONFIG_VIRTIO_PCI=y" > CONFIG_VIRTIO_BLK=y" > CONFIG_VIRTIO_NET=y" > > To get disks and network for the virt board. > > Could we add these configs to mainline Linux's defconfig? Note, we > don't need to add a DTS for this board, as QEMU's bootloader will > generate and provide it for us. > > There is the known highmem PCI issue that is currently under discussion: > > https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html > > So aside from the missing VIRTIO kernel configs and highmem bug, virt > is in good shape. > > vexpress: > > vexpress boots up to rootfs probing, however the only storage media > that seems to be supported is flash, which doesn't have the needed > kernel configs to boot from (not 100% sure on that but I gave up > quickly). What are people who use this board in real hardware doing? > It may just work best with initrd style boots. > > exynos: > > QEMU models an Exynos skdkc210 but there is only a 310 available in > QEMU dtbs. QEMU exynos however does not model a lot on the board level > so we can largely get away with booting 310 dtb on 210. > > Similar to vexpress, the exynos board has a lack of usable boot media. > The board does model USB (EHCI), however the DTB disables it as the > smdk310 board does not support it. So the alternative would be to use > one of the more full-featured exynos DTBs (with way more peripherals) > from mainline Linux, but the kernel is liable to then probe all sorts > of missing hardware. > > Exynos SDHCI support was on list for a few revs and quite some time, > merging it and using SD may solve the problem. > > I have not attempted the SMP support. > > Unlike all other boards, Exynos is quite consuming of RAM. All other > boards work with Yocto's default of 128MB but Exynos crashes unless > you jack it up. So users need to be careful of the -m option. > > realview: > > Realview is not tested, but I discuss it here as it is a bit of a red > herring. The kernel has dtb support for the realview-1176 board, > however this is not modelled by QEMU. I creating a Realview variant > with 1176 in QEMU, however the memory maps of all the peripherals were > mismatched. > > Turns, out, QEMU is is modelling the realview FPGA emulation platform > which has a different peripheral map to the 1176 realview board > proper. So QEMU doesn't support the realview boards, just the FPGA > emulation platform, for which I can't find a kernel dtb. Should we > upstream one with some dtsi's for the CPU variation supported in QEMU? > > So Realview does not work within my testing parameters with no > immediate prospects. > > how I am testing: > > So this is all powered by the Yocto project (Poky). I got some good > help from Nathan Rossi. I have a poky fork which changes the qemuarm > target to build my mainline (4.2.1) kernel (multi_v7_defconfig + > VIRTIO), all the DTBs and a usable rootfs. You can then specify the > type of machine (from the list above) to yocto's runqemu command. The > command sets up boot media and network automatically. > > To use it: > > git clone https://github.com/pcrost/poky.git > cd poky > source ./oe-init-build-env > MACHINE=qemuarm bitbake core-image-minimal #this takes a while > QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp > > The QEMUBIN is optional as Yocto does build QEMU for you, but this > lets you BYO QEMU if you are doing qemu development outside of the > Yocto flow. Change MACHINE_SUBTYPE to one of the supported boards to > see results. I suggest starting with virt. Skip cubieboard, that > assumes my SATA patches. Everything else you will see varying degrees > of success. Pass qemuparams="-m 256" to runqemu for the smdkc200 > (Exynos) board. Highbank and Midway will blank screen as the > outstanding issues happen pre-UART. A meaningful logbuf can be pulled > from RAM over the monitor. Useful instructions here: > > http://www.wiki.xilinx.com/QEMU-Linux+Kernel+logbuf+Extraction > > Some QEMU patches to follow to fix a few of the things mentioned. > > We should get this into regression testing. Yocto has some automated > testing features in it's own right that we should be able to leverage. > Yet to investigate (Richard pasted some stuff on an earlier thread). > > The exercise should be redone for ARM64, ARMv5 and ARMv6 and ARMv7M, > then other arches (many of which Poky supports). > > Regards, > Peter >
