Hi Khem, > On Mon, Aug 23, 2021 at 1:09 PM Lukasz Majewski <lu...@denx.de> wrote: > > > > On Mon, 23 Aug 2021 12:52:44 -0700 > > Khem Raj <raj.k...@gmail.com> wrote: > > > > > On Mon, Aug 23, 2021 at 11:24 AM Lukasz Majewski <lu...@denx.de> > > > wrote: > > > > > > > Hi Khem, > > > > > > > > > On 8/23/21 8:08 AM, ?ukasz Majewski wrote: > > > > > > This patch introduces new recipe - namely 'glibc-tests', > > > > > > which builds and installs glibc test suite to OE/Yocto > > > > > > built image. > > > > > > > > > > > > It reuses code from already available 'glibc-testsuite' > > > > > > recipe, which is run with 'bitbake glibc-testsuite -c > > > > > > check' and uses qemu to execute remotely (via SSH) tests on > > > > > > some emulated machine. > > > > > > > > > > > > This recipe installs eligible tests on some rootfs image. > > > > > > Afterwards, either all of them or only time related subset, > > > > > > those tests can be executed on the real hardware, to > > > > > > facilitate validation of this platform with for example > > > > > > Y2038 problem compliance. > > > > > > > > > > > > By default all tests are executed, with 'ptest-runner > > > > > > glibc-tests'. To test only time related subset - one needs > > > > > > to call: cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \ > > > > > > ln -s run-ptest-time run-ptest; cd -; ptest-runner > > > > > > glibc-tests > > > > > > > > > > > > To facilitate debugging, source files are provided by > > > > > > default with the unstripped debugging symbols. Such > > > > > > approach would reduce the already complex recipe (as it > > > > > > inherits base glibc one), so there is no need to also > > > > > > install *-dbg and *-src packages. > > > > > > > > > > > > > > > does it have to be a separate recipe I wonder if we can have > > > > > it built as part of glibc itself controlled via ptest knob > > > > > > > > I've followed the glibc-testsuite recipe to provide tests for > > > > ptest. Test creation is similar to it, but doesn't require QEMU > > > > run (tests are executed on the HW). > > > > > > > > Another rationale was to have some kind of "features" > > > > separation in different file (liked glibc-mtrace), which is > > > > easier to maintain and review. > > > > > > > > Last but not least - separate recipe (and built binaries) allow > > > > some kind of magic with selection of used test programs (this > > > > may be useful if one would like to have different sets of tests > > > > in different packages) > > > > > > That’s seems ok I think the names are quite confusing now not > > > that it was not so much better before but now we have glibc-tests > > > and glibc-testsuites which do same things but in very different > > > way maybe glibc-testsuite should be renamed to something like > > > glibc-tests-crosd or some such > > > > I think that glibc-testsuite_2.34.bb shall be renamed to > > glibc-tests-qemu_2.34.bb as it is more descriptive. > > is it only runnable in qemu ? I thought we could use a read board as > well with IP address
It looks like by default the QEMU is used in this recipe.
This recipe provides custom check-test-wrapper, which can log
into remote board via ssh (when TOOLCHAIN_TEST_TARGET = "ssh").
This looks a bit awkward for me, as the goal would be to run built
tests on target (exact) HW without the need of SSH.
It is also easier to debug the code with the real HW.
>
> >
> > Then, glibc-tests_2.34.bb could be left as it is now, as inheriting
> > ptest requires it to be added as 'glibc-tests-ptest' in
> > meta/conf/distro/include/ptest-packagelists.inc
> >
> > (I've tried to rename it to glibc-tests-ptest_2.34.bb, but then it
> > was required to add glibc-tests-ptest-ptest to the
> > ptest-packagelist.inc file, which is a bit clumsy IMHO).
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > Signed-off-by: Lukasz Majewski <lu...@denx.de>
> > > > > > ---
> > > > > > .../distro/include/ptest-packagelists.inc | 1 +
> > > > > > meta/recipes-core/glibc/glibc-tests_2.34.bb | 101
> > > > > > ++++++++++++++++++
> > > > > > meta/recipes-core/glibc/glibc/run-ptest-all | 25 +++++
> > > > > > meta/recipes-core/glibc/glibc/run-ptest-time | 30 ++++++
> > > > > > 4 files changed, 157 insertions(+) create mode 100644
> > > > > > meta/recipes-core/glibc/glibc-tests_2.34.bb create mode
> > > > > > 100755 meta/recipes-core/glibc/glibc/run-ptest-all create
> > > > > > mode 100755 meta/recipes-core/glibc/glibc/run-ptest-time
> > > > > >
> > > > > > diff --git a/meta/conf/distro/include/ptest-packagelists.inc
> > > > > > b/meta/conf/distro/include/ptest-packagelists.inc index
> > > > > > 3872bdc942..e460ad1c8b 100644 ---
> > > > > > a/meta/conf/distro/include/ptest-packagelists.inc +++
> > > > > > b/meta/conf/distro/include/ptest-packagelists.inc @@ -59,6
> > > > > > +59,7 @@ PTESTS_FAST = "\ slang-ptest \
> > > > > > wayland-ptest \
> > > > > > zlib-ptest \
> > > > > > + glibc-tests-ptest \
> > > > > > "
> > > > > > PTESTS_FAST:remove:mips64 = "qemu-ptest"
> > > > > > PTESTS_PROBLEMS:append:mips64 = "qemu-ptest"
> > > > > > diff --git a/meta/recipes-core/glibc/glibc-tests_2.34.bb
> > > > > > b/meta/recipes-core/glibc/glibc-tests_2.34.bb new file mode
> > > > > > 100644 index 0000000000..896809f700
> > > > > > --- /dev/null
> > > > > > +++ b/meta/recipes-core/glibc/glibc-tests_2.34.bb
> > > > > > @@ -0,0 +1,101 @@
> > > > > > +require glibc_${PV}.bb
> > > > > > +require glibc-tests.inc
> > > > > > +
> > > > > > +inherit ptest
> > > > > > +
> > > > > > +SRC_URI:append = " \
> > > > > > + file://run-ptest-all file://run-ptest-time \
> > > > > > +"
> > > > > > +
> > > > > > +SUMMARY = "glibc tests to be run with ptest"
> > > > > > +
> > > > > > +# Erase some variables already set by glibc_${PV}
> > > > > > +python __anonymous() {
> > > > > > + # Remove packages provided by glibc build, we only
> > > > > > need a subset of them
> > > > > > + d.setVar("PACKAGES", "${PN}")
> > > > > > +
> > > > > > + d.setVar("PROVIDES", "${PN}")
> > > > > > + d.setVar("RPROVIDES", "${PN} glibc-ptest")
> > > > > > +
> > > > > > + d.setVar("RRECOMMENDS", "")
> > > > > > +}
> > > > > > +
> > > > > > +# Remove any lefovers from original glibc recipe
> > > > > > +RPROVIDES:${PN} = "${PN}"
> > > > > > +RRECOMMENDS:${PN} = ""
> > > > > > +RDEPENDS:${PN} = "sed"
> > > > > > +
> > > > > > +# Just build tests for target - do not run them
> > > > > > +do_check:append () {
> > > > > > + oe_runmake -i check run-built-tests=no
> > > > > > +}
> > > > > > +addtask do_check after do_compile before
> > > > > > do_install_ptest_base +
> > > > > > +glibc_strip_build_directory () {
> > > > > > + # Delete all non executable files from build directory
> > > > > > + find ${B} ! -executable -type f -delete
> > > > > > +
> > > > > > + # Remove build dynamic libraries and links to them as
> > > > > > + # those are already installed in the target device
> > > > > > + find ${B} -type f -name "*.so" -delete
> > > > > > + find ${B} -type l -name "*.so*" -delete
> > > > > > +
> > > > > > + # Remove headers (installed with glibc)
> > > > > > + find ${B} -type f -name "*.h" -delete
> > > > > > +
> > > > > > + find ${B} -type f -name "isomac" -delete
> > > > > > + find ${B} -type f -name "annexc" -delete
> > > > > > +}
> > > > > > +
> > > > > > +do_install_ptest_base () {
> > > > > > + glibc_strip_build_directory
> > > > > > +
> > > > > > + # Install build test programs to the image
> > > > > > + install -d ${D}${PTEST_PATH}/tests/glibc-ptest/
> > > > > > + cp -r ${B}/* ${D}${PTEST_PATH}/tests/glibc-ptest/
> > > > > > +
> > > > > > + install -d ${D}${PTEST_PATH}
> > > > > > + cp ${WORKDIR}/run-ptest-* ${D}${PTEST_PATH}/
> > > > > > +
> > > > > > + # By default run all glibc tests
> > > > > > + cd ${D}${PTEST_PATH}/
> > > > > > + ln -s run-ptest-all run-ptest
> > > > > > +}
> > > > > > +
> > > > > > +# The datadir directory is required to allow core (and
> > > > > > reused) +# glibc cleanup function to finish correctly, as
> > > > > > this directory +# is not created for ptests
> > > > > > +stash_locale_package_cleanup:prepend () {
> > > > > > + mkdir -p ${PKGD}${datadir}
> > > > > > +}
> > > > > > +
> > > > > > +stash_locale_sysroot_cleanup:prepend () {
> > > > > > + mkdir -p ${SYSROOT_DESTDIR}${datadir}
> > > > > > +}
> > > > > > +
> > > > > > +# Prevent the do_package() task to set 'libc6' prefix
> > > > > > +# for glibc tests related packages
> > > > > > +python populate_packages:prepend () {
> > > > > > + if d.getVar('DEBIAN_NAMES'):
> > > > > > + d.setVar('DEBIAN_NAMES', '')
> > > > > > +}
> > > > > > +
> > > > > > +FILES:${PN} = "${PTEST_PATH}/*
> > > > > > /usr/src/debug/glibc-tests/*" +
> > > > > > +EXCLUDE_FROM_SHLIBS = "1"
> > > > > > +
> > > > > > +# Install debug data in .debug and sources in
> > > > > > /usr/src/debug +# It is more handy to have _all_ the
> > > > > > sources and symbols in one +# place (package) as this
> > > > > > recipe will be used for validation and +# debugging.
> > > > > > +PACKAGE_DEBUG_SPLIT_STYLE = "debug"
> > > > > > +
> > > > > > +# glibc test cases violate by default some Yocto/OE checks
> > > > > > (staticdev, +# textrel)
> > > > > > +# 'debug-files' - add everything (including debug) into one
> > > > > > package +# (no need to install/build *-src
> > > > > > package) +INSANE_SKIP:${PN} += "staticdev textrel
> > > > > > debug-files rpaths" +
> > > > > > +deltask do_stash_locale
> > > > > > +do_install[noexec] = "1"
> > > > > > +do_populate_sysroot[noexec] = "1"
> > > > > > diff --git a/meta/recipes-core/glibc/glibc/run-ptest-all
> > > > > > b/meta/recipes-core/glibc/glibc/run-ptest-all new file mode
> > > > > > 100755 index 0000000000..47bfe5238c
> > > > > > --- /dev/null
> > > > > > +++ b/meta/recipes-core/glibc/glibc/run-ptest-all
> > > > > > @@ -0,0 +1,25 @@
> > > > > > +#!/bin/sh
> > > > > > +# ptest script for glibc - to run time related tests to
> > > > > > +# facilitate Y2038 validation
> > > > > > +# Run with 'ptest-runner glibc-tests'
> > > > > > +
> > > > > > +output() {
> > > > > > + retcode=$?
> > > > > > + if [ $retcode -eq 0 ]
> > > > > > + then echo "PASS: $i"
> > > > > > + elif [ $retcode -eq 77 ]
> > > > > > + then echo "SKIP: $i"
> > > > > > + else echo "FAIL: $i"
> > > > > > + fi
> > > > > > +}
> > > > > > +
> > > > > > +# Allow altering time on the target
> > > > > > +export GLIBC_TEST_ALLOW_TIME_SETTING="1"
> > > > > > +
> > > > > > +tst=$(ls -r ${PWD}/tests/glibc-ptest/*/*)
> > > > > > +
> > > > > > +for i in ${tst}
> > > > > > +do
> > > > > > + $i >/dev/null 2>&1
> > > > > > + output
> > > > > > +done
> > > > > > diff --git a/meta/recipes-core/glibc/glibc/run-ptest-time
> > > > > > b/meta/recipes-core/glibc/glibc/run-ptest-time new file mode
> > > > > > 100755 index 0000000000..24dedc6e32
> > > > > > --- /dev/null
> > > > > > +++ b/meta/recipes-core/glibc/glibc/run-ptest-time
> > > > > > @@ -0,0 +1,30 @@
> > > > > > +#!/bin/sh
> > > > > > +# ptest script for glibc - to run time related tests to
> > > > > > +# facilitate Y2038 validation
> > > > > > +# Run with 'ptest-runner glibc-tests'
> > > > > > +
> > > > > > +output() {
> > > > > > + retcode=$?
> > > > > > + if [ $retcode -eq 0 ]
> > > > > > + then echo "PASS: $i"
> > > > > > + elif [ $retcode -eq 77 ]
> > > > > > + then echo "SKIP: $i"
> > > > > > + else echo "FAIL: $i"
> > > > > > + fi
> > > > > > +}
> > > > > > +
> > > > > > +# Allow altering time on the target
> > > > > > +export GLIBC_TEST_ALLOW_TIME_SETTING="1"
> > > > > > +
> > > > > > +tst_time64=$(ls -r ${PWD}/tests/glibc-ptest/*/*-time64)
> > > > > > +
> > > > > > +# Remove '-time64' suffix - those tests are also time
> > > > > > +# related
> > > > > > +tst_time_tmp=$(sed -e "s/-time64$//" <<< ${tst_time64})
> > > > > > +tst_time="${tst_time_tmp} ${tst_time64}"
> > > > > > +
> > > > > > +for i in ${tst_time}
> > > > > > +do
> > > > > > + $i >/dev/null 2>&1
> > > > > > + output
> > > > > > +done
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> > > > Best regards,
> > > >
> > > > Lukasz Majewski
> > > >
> > > > --
> > > >
> > > > DENX Software Engineering GmbH, Managing Director: Wolfgang
> > > > Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194
> > > > Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax:
> > > > (+49)-8142-66989-80 Email: lu...@denx.de
> >
> >
> >
> >
> > Best regards,
> >
> > Lukasz Majewski
> >
> > --
> >
> > DENX Software Engineering GmbH, Managing Director: Wolfgang
> > Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell,
> > Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email:
> > lu...@denx.de
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lu...@denx.de
pgpAfDRybYZZV.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#155206): https://lists.openembedded.org/g/openembedded-core/message/155206 Mute This Topic: https://lists.openembedded.org/mt/85087396/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
