On Fri, Apr 27, 2018 at 2:04 AM, Peter Maydell <peter.mayd...@linaro.org> wrote:
> On 26 April 2018 at 14:57, Eduardo Habkost <ehabk...@redhat.com> wrote: > > Peter, do you have additional tests you run before merging a pull > > request? Additional test sets run before tagging a release? > > I run make && make check, on a variety of hosts, before merging > any pull request. That's the only testing I do, and I don't > do anything more before I do a tag. > This is what I do on riscv, however it is not yet automated: - QEMU upstream `make check` tests are run in Travis and they should all pass. - The RISC-V ISA compatibility tests from the `riscv-tests` repository are run and currently all of the tests pass. - Fedora Linux and Debian Linux are used for running self-hosted GCC bootstraps using the latest Linux kernel and GLIBC user-space in the `virt` machine. A parallel GCC bootstrap is used for stress testing MTTCG, SMP and the interrupt controllers in a multi-core RISC-V virtual machine. - Linux 4.6.2 with bbl and busybox are used for backward compatibility testing in the `spike_v1.9.1` machine. - Linux 4.15.0 with bbl and busybox are used for forward compatibility testing in the `spike_v1.10` machine. - HiFive1 binaries from the `freedom-u-sdk` are used for testing the `sifive_e` machine. I only run the 24-hour type tests if I am changing anything related to interrupts. We are working on interrupt controller emulation and the SMP PLIC used by the 'virt' machine was authored by me, however Stefan O'Rear contributed to the interrupt handling code and I've been very careful to test that I haven't broken and to fully understand the logic and atomicity of interrupts. We are working on a new interrupt controller and are specifically looking at the possibility of adding a GPIO mock that could be combined with some sort of replay mechanism to test the interrupt controllers. We could bring a snapshot of riscv-tests into the QEMU tree however it would be best if they retained the BSD license so we can share locally added tests with upstream as upstream is adding tests periodically. We have recently made changes to the riscv page walker where we have documented it in such a way that each clause represents a descriprition for a test case. Our goal is that we would like the code to read similarly to the RISC-V Privileged ISA specification where possible. - https://github.com/riscv/riscv-qemu/blob/qemu-2.13-for-upstream/target/riscv/cpu_helper.c#L210-L243 BTW Andrew Waterman reviewed my update to the page walker and caught a bug. Linux testing doesn't exercise every dimension of the page permission checks. We effectively need malware type tests. Multi-core PTE update atomicity tests will also be interesting. The SiFive verification team have been working on PMP tests however I am not sure whether these will be made open source. The RISC-V QEMU PMP code needs testing. It also needs to implement recent changes to the specification for coarse grain PMPs (4K granularity) given we use the TLB page size so that PMP can be implemented with reasonable performance. We would like to automate all of this and we are working towards that goal. One of the dependencies for docker tests are the riscv toolchain packages and we are working on this... Adding port specific tests, like riscv tests to QEMU not only benefits the particular port but also exercises a log of generic code such as the TCG, the IO system, IO mutex, Virt IO and many other shared code paths.
