On Thu, 2021-08-26 at 14:27 +0200, Lukasz Majewski wrote: > Hi Richard, Khem, > > > Hi Richard, > > > > > On Tue, 2021-08-24 at 09:36 +0200, ?ukasz Majewski wrote: > > > > Hi Khem, > > > > > > > > > On Mon, Aug 23, 2021 at 1:09 PM Lukasz Majewski <lu...@denx.de> > > > > > wrote: > > > > > > > > > > > > On Mon, 23 Aug 2021 12:52:44 -0700 > > > > > > Khem Raj <raj.k...@gmail.com> wrote: > > > > > > > > > > > > > On Mon, Aug 23, 2021 at 11:24 AM Lukasz Majewski > > > > > > > <lu...@denx.de> wrote: > > > > > > > > > > > > > > > Hi Khem, > > > > > > > > > > > > > > > > > On 8/23/21 8:08 AM, ?ukasz Majewski wrote: > > > > > > > > > > This patch introduces new recipe - namely > > > > > > > > > > 'glibc-tests', which builds and installs glibc test > > > > > > > > > > suite to OE/Yocto built image. > > > > > > > > > > > > > > > > > > > > It reuses code from already available > > > > > > > > > > 'glibc-testsuite' recipe, which is run with 'bitbake > > > > > > > > > > glibc-testsuite -c check' and uses qemu to execute > > > > > > > > > > remotely (via SSH) tests on some emulated machine. > > > > > > > > > > > > > > > > > > > > This recipe installs eligible tests on some rootfs > > > > > > > > > > image. Afterwards, either all of them or only time > > > > > > > > > > related subset, those tests can be executed on the > > > > > > > > > > real hardware, to facilitate validation of this > > > > > > > > > > platform with for example Y2038 problem compliance. > > > > > > > > > > > > > > > > > > > > By default all tests are executed, with 'ptest-runner > > > > > > > > > > glibc-tests'. To test only time related subset - one > > > > > > > > > > needs to call: cd /usr/lib/glibc-tests/ptest/; rm > > > > > > > > > > run-ptest; \ ln -s run-ptest-time run-ptest; cd -; > > > > > > > > > > ptest-runner glibc-tests > > > > > > > > > > > > > > > > > > > > To facilitate debugging, source files are provided by > > > > > > > > > > default with the unstripped debugging symbols. Such > > > > > > > > > > approach would reduce the already complex recipe (as > > > > > > > > > > it inherits base glibc one), so there is no need to > > > > > > > > > > also install *-dbg and *-src packages. > > > > > > > > > > > > > > > > > > > > > > > > > > > does it have to be a separate recipe I wonder if we can > > > > > > > > > have it built as part of glibc itself controlled via > > > > > > > > > ptest knob > > > > > > > > > > > > > > > > I've followed the glibc-testsuite recipe to provide tests > > > > > > > > for ptest. Test creation is similar to it, but doesn't > > > > > > > > require QEMU run (tests are executed on the HW). > > > > > > > > > > > > > > > > Another rationale was to have some kind of "features" > > > > > > > > separation in different file (liked glibc-mtrace), which > > > > > > > > is easier to maintain and review. > > > > > > > > > > > > > > > > Last but not least - separate recipe (and built binaries) > > > > > > > > allow some kind of magic with selection of used test > > > > > > > > programs (this may be useful if one would like to have > > > > > > > > different sets of tests in different packages) > > > > > > > > > > > > > > That’s seems ok I think the names are quite confusing now > > > > > > > not that it was not so much better before but now we have > > > > > > > glibc-tests and glibc-testsuites which do same things but in > > > > > > > very different way maybe glibc-testsuite should be renamed > > > > > > > to something like glibc-tests-crosd or some such > > > > > > > > > > > > I think that glibc-testsuite_2.34.bb shall be renamed to > > > > > > glibc-tests-qemu_2.34.bb as it is more descriptive. > > > > > > > > > > is it only runnable in qemu ? I thought we could use a read > > > > > board as well with IP address > > > > > > > > It looks like by default the QEMU is used in this recipe. > > > > > > > > This recipe provides custom check-test-wrapper, which can log > > > > into remote board via ssh (when TOOLCHAIN_TEST_TARGET = "ssh"). > > > > > > > > This looks a bit awkward for me, as the goal would be to run built > > > > tests on target (exact) HW without the need of SSH. > > > > > > > > It is also easier to debug the code with the real HW. > > > > > > I'm a bit worried we're going around in circles with this. We did > > > have a different form of tests a while back but I was told those > > > didn't work well and they were replaced with the current setup which > > > worked with the driver in the glibc source code. > > > > I've looked into the glibc-testing.inc from SHA1: > > 0a42ae7f7ca8fcebe4630b701e50e2352f9b7c3b > > > > There glibc tests were built and copied on the target via NFS. > > > > > This was done around > > > the time we added support for the other toolchain test suites > > > (binutils+gcc). > > > > I'm not familiar with the "binutils+gcc" issue, which was solved in > > the past. Could you share any reference? > > > > > We ended up with options to use full system qemu or > > > user mode qemu emulation depending on the speed of the target (e.g. > > > with kvm or not). > > > > From what I see in the recipe - you either run user mode QEMU by > > default (at least when I do run it): > > > > NOTE: make PARALLELMFLAGS=-j 8 SHELL=/bin/bash -i > > QEMU_SYSROOT=/home/lukma/work/yocto/y2038/build/tmp/work/armv7at2hf-neon-poky-linux-gnueabi/glibc-testsuite/2.34-r0/recipe-sysroot > > QEMU_OPTIONS=qemu-arm -r 5.1.0 SSH_HOST=localhost SSH_HOS T_USER=root > > SSH_HOST_PORT=2222 > > test-wrapper=/home/lukma/work/yocto/y2038/build/tmp/work/armv7at2hf-neon-poky-linux-gnueabi/glibc-testsuite/2.34-r0/check-test-wrapper > > user check > > > > Or you log to the board via SSH (the 'ssh' mode). > > > > > > > > Has this patchset been run with all the tests in glibc or just a > > > subset? > > > > This patch set allows running all tests [*], or (preferably in my use > > case) only a time related ones. The latter is to validate Y2038 code, > > which testing has two issues: > > > > 1. It cannot be tested with QEMU user mode [1]. Instead "linux > > timespaces" were suggested, but it lacks support for CLOCK_REALTIME > > manipulation for those tests [2]. > > > > 2. Testing with SSH is possible, but is not as robust as expected > > (i.e. imposes SSHD on the target board, and ETH connection). > > > > > > > > I'm always very nervous about having two ways to do similar things. > > > > > > > Technically, I just reuse the glibc-testsuite_2.34.bb to the point > > where tests are _only_ built. Then I pack those binaries and install > > on the target's rootfs to be run with ptest framework. > > > > Such approach has several advantages: > > > > - No need for SSH (which can hang, as you need sshfs in user space for > > the setup) > > > > - Run on the real HW (and environment) with the same version of glibc > > on the real target. > > > > - Sources and debugging symbols available on the target, so easier GDB > > debugging. > > > > - Using ptest framework to execute tests > > > > - Time related tests are "self contained", so could be executed > > without built data. > > > > At least for Y2038 validation such approach seems more appealing. > > > > Do you have any more feedback regarding this patch?
We're at feature freeze and this wasn't a planned change so it is too late for this release cycle, I'm struggling with the things that were planned. I need to spend some time looking at what the other code is doing and how we're using it on the autobuilder and whether we want to change to use the ptest approach or not. I added Nathan Rossi to cc since he worked on the other code for this and may have more insight. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#155361): https://lists.openembedded.org/g/openembedded-core/message/155361 Mute This Topic: https://lists.openembedded.org/mt/85087396/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
