Hi Richard, Khem, > Hi Richard, > > > On Tue, 2021-08-24 at 09:36 +0200, ?ukasz Majewski wrote: > > > Hi Khem, > > > > > > > On Mon, Aug 23, 2021 at 1:09 PM Lukasz Majewski <lu...@denx.de> > > > > wrote: > > > > > > > > > > On Mon, 23 Aug 2021 12:52:44 -0700 > > > > > Khem Raj <raj.k...@gmail.com> wrote: > > > > > > > > > > > On Mon, Aug 23, 2021 at 11:24 AM Lukasz Majewski > > > > > > <lu...@denx.de> wrote: > > > > > > > > > > > > > Hi Khem, > > > > > > > > > > > > > > > On 8/23/21 8:08 AM, ?ukasz Majewski wrote: > > > > > > > > > This patch introduces new recipe - namely > > > > > > > > > 'glibc-tests', which builds and installs glibc test > > > > > > > > > suite to OE/Yocto built image. > > > > > > > > > > > > > > > > > > It reuses code from already available > > > > > > > > > 'glibc-testsuite' recipe, which is run with 'bitbake > > > > > > > > > glibc-testsuite -c check' and uses qemu to execute > > > > > > > > > remotely (via SSH) tests on some emulated machine. > > > > > > > > > > > > > > > > > > This recipe installs eligible tests on some rootfs > > > > > > > > > image. Afterwards, either all of them or only time > > > > > > > > > related subset, those tests can be executed on the > > > > > > > > > real hardware, to facilitate validation of this > > > > > > > > > platform with for example Y2038 problem compliance. > > > > > > > > > > > > > > > > > > By default all tests are executed, with 'ptest-runner > > > > > > > > > glibc-tests'. To test only time related subset - one > > > > > > > > > needs to call: cd /usr/lib/glibc-tests/ptest/; rm > > > > > > > > > run-ptest; \ ln -s run-ptest-time run-ptest; cd -; > > > > > > > > > ptest-runner glibc-tests > > > > > > > > > > > > > > > > > > To facilitate debugging, source files are provided by > > > > > > > > > default with the unstripped debugging symbols. Such > > > > > > > > > approach would reduce the already complex recipe (as > > > > > > > > > it inherits base glibc one), so there is no need to > > > > > > > > > also install *-dbg and *-src packages. > > > > > > > > > > > > > > > > > > > > > > > > does it have to be a separate recipe I wonder if we can > > > > > > > > have it built as part of glibc itself controlled via > > > > > > > > ptest knob > > > > > > > > > > > > > > I've followed the glibc-testsuite recipe to provide tests > > > > > > > for ptest. Test creation is similar to it, but doesn't > > > > > > > require QEMU run (tests are executed on the HW). > > > > > > > > > > > > > > Another rationale was to have some kind of "features" > > > > > > > separation in different file (liked glibc-mtrace), which > > > > > > > is easier to maintain and review. > > > > > > > > > > > > > > Last but not least - separate recipe (and built binaries) > > > > > > > allow some kind of magic with selection of used test > > > > > > > programs (this may be useful if one would like to have > > > > > > > different sets of tests in different packages) > > > > > > > > > > > > That’s seems ok I think the names are quite confusing now > > > > > > not that it was not so much better before but now we have > > > > > > glibc-tests and glibc-testsuites which do same things but in > > > > > > very different way maybe glibc-testsuite should be renamed > > > > > > to something like glibc-tests-crosd or some such > > > > > > > > > > I think that glibc-testsuite_2.34.bb shall be renamed to > > > > > glibc-tests-qemu_2.34.bb as it is more descriptive. > > > > > > > > is it only runnable in qemu ? I thought we could use a read > > > > board as well with IP address > > > > > > It looks like by default the QEMU is used in this recipe. > > > > > > This recipe provides custom check-test-wrapper, which can log > > > into remote board via ssh (when TOOLCHAIN_TEST_TARGET = "ssh"). > > > > > > This looks a bit awkward for me, as the goal would be to run built > > > tests on target (exact) HW without the need of SSH. > > > > > > It is also easier to debug the code with the real HW. > > > > I'm a bit worried we're going around in circles with this. We did > > have a different form of tests a while back but I was told those > > didn't work well and they were replaced with the current setup which > > worked with the driver in the glibc source code. > > I've looked into the glibc-testing.inc from SHA1: > 0a42ae7f7ca8fcebe4630b701e50e2352f9b7c3b > > There glibc tests were built and copied on the target via NFS. > > > This was done around > > the time we added support for the other toolchain test suites > > (binutils+gcc). > > I'm not familiar with the "binutils+gcc" issue, which was solved in > the past. Could you share any reference? > > > We ended up with options to use full system qemu or > > user mode qemu emulation depending on the speed of the target (e.g. > > with kvm or not). > > From what I see in the recipe - you either run user mode QEMU by > default (at least when I do run it): > > NOTE: make PARALLELMFLAGS=-j 8 SHELL=/bin/bash -i > QEMU_SYSROOT=/home/lukma/work/yocto/y2038/build/tmp/work/armv7at2hf-neon-poky-linux-gnueabi/glibc-testsuite/2.34-r0/recipe-sysroot > QEMU_OPTIONS=qemu-arm -r 5.1.0 SSH_HOST=localhost SSH_HOS T_USER=root > SSH_HOST_PORT=2222 > test-wrapper=/home/lukma/work/yocto/y2038/build/tmp/work/armv7at2hf-neon-poky-linux-gnueabi/glibc-testsuite/2.34-r0/check-test-wrapper > user check > > Or you log to the board via SSH (the 'ssh' mode). > > > > > Has this patchset been run with all the tests in glibc or just a > > subset? > > This patch set allows running all tests [*], or (preferably in my use > case) only a time related ones. The latter is to validate Y2038 code, > which testing has two issues: > > 1. It cannot be tested with QEMU user mode [1]. Instead "linux > timespaces" were suggested, but it lacks support for CLOCK_REALTIME > manipulation for those tests [2]. > > 2. Testing with SSH is possible, but is not as robust as expected > (i.e. imposes SSHD on the target board, and ETH connection). > > > > > I'm always very nervous about having two ways to do similar things. > > > > Technically, I just reuse the glibc-testsuite_2.34.bb to the point > where tests are _only_ built. Then I pack those binaries and install > on the target's rootfs to be run with ptest framework. > > Such approach has several advantages: > > - No need for SSH (which can hang, as you need sshfs in user space for > the setup) > > - Run on the real HW (and environment) with the same version of glibc > on the real target. > > - Sources and debugging symbols available on the target, so easier GDB > debugging. > > - Using ptest framework to execute tests > > - Time related tests are "self contained", so could be executed > without built data. > > At least for Y2038 validation such approach seems more appealing. >
Do you have any more feedback regarding this patch? > > > > Cheers, > > > > Richard > > > > Links: > [1] - > https://www.mail-archive.com/qemu-discuss@nongnu.org/msg05898.html > [2] - > https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg2361284.html > > Note: > [*] - some glibc tests rely on the built directory to provide correct > data/environment for execution. Time related tests are (and should be) > self contained, so those could be just executed as binaries. > > Best regards, > > Lukasz Majewski > > -- > > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: > lu...@denx.de Best regards, Lukasz Majewski -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lu...@denx.de
pgpBd8k0q29X9.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#155354): https://lists.openembedded.org/g/openembedded-core/message/155354 Mute This Topic: https://lists.openembedded.org/mt/85087396/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
