On 2022-09-08 03:06, Richard Purdie wrote:
On Wed, 2022-09-07 at 20:27 +0530, pgowda....@gmail.com wrote:
From: pgowda <pgowda....@gmail.com>
The patch implements Rust testing framework similar to other selftest,
specifically the gcc selftest in OE. It uses the client and server
based method to test the binaries for cross-target on the image.
The test framework is a wrapper around the Rust build system as ./x.py
test. It tests many functionalities of Rust distribution like tools,
documentation, libraries, packages, tools, Cargo, Crater etc.
Please refer the following link for detailed description of Rust
testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
To support the rust tests in oe-core, the following functions were
added:-
setup_cargo_environment(): Build bootstrap and some early stage tools.
do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemu target image.
Approximate Number of Tests Run in the Rust Testsuite :- 18000
Approximate Number of Tests that FAIL in bitbake environment :- 100-150
Normally majority of the testcases are present in major folder "test/"
It contributes to more than 80% of the testcases present in Rust test
framework. These tests pass as expected on any Rust versions without
much fuss. The tests that fail are of less important and contribute to
less than 2% of the total testcases. These minor tests are observed to
work on some versions and fail on others. They have to be added, ignored
or excluded for different versions as per the behavior.
These tests have been ignored or excluded in the Rust selftest
environment to generate success of completing the testsuite.
These tests work in parallel mode even in the skipped test mode as
expected. Although the patch to disable tests is large, it is very simple
in that it only disables tests. When updating to a newer version of Rust,
the patch can usually be ported in a day.
Signed-off-by: pgowda <pgowda....@gmail.com>
Signed-off-by: Vinay Kumar <vinay.m.e...@gmail.com>
---
meta/lib/oeqa/selftest/cases/rust.py | 54 ++
meta/recipes-devtools/rust/rust-source.inc | 3 +-
meta/recipes-devtools/rust/rust.inc | 1 +
.../rust/rust/rust-oe-selftest.patch | 508 ++++++++++++++++++
meta/recipes-devtools/rust/rust_1.63.0.bb | 5 +
5 files changed, 570 insertions(+), 1 deletion(-)
create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
create mode 100644 meta/recipes-devtools/rust/rust/rust-oe-selftest.patch
I ran this through some testing on the autobuilder. The first time was
ok, it only ran on qemux86 and qemux86-64 as it was only enabled for
toolchain-system but ran successfully.
Yay, progress!
On the second run I added @OETestTag("toolchain-user") as well which
meant it ran for arm/mips/powerpc. It looks like there was one worrying
intermittent failure:
qemux86-64 intermittent issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5788/steps/19/logs/stdio
A copy failed so Naveen should figure out why and fix the root cause.
This is the top priority to deal with and if you fix that, you should
re-submit
before spending significant time on the non-x86 tests.
and then some arch specific failures:
qemuppc failed in the test:
https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/5790/steps/19/logs/stdio
qemumips64 failed in the test:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/5786/steps/20/logs/stdio
qemumips failed in the test:
https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/5804/steps/20/logs/stdio
qemuarm64, qemuarm and qemux86 passed on this run.
Naveen,
Does qemuriscv64 pass all tests? If not summarize as described below.
We need to understand and fix that qemux86-64 intermittent issue before
this can merge. I wasn't sure what we want to do with mips and ppc.
It's tempting to just skip ppc, mips* since they are 2rd tier platforms for
Rust which means that:
"Rust's continuous integration checks that tier 2 targets will
always build,
but they may or may not pass tests."
- https://doc.rust-lang.org/nightly/rustc/target-tier-policy.html
- https://doc.rust-lang.org/nightly/rustc/platform-support.html
Also, in practice, they are not top tier targets for Yocto.
Regardless , I'd like Naveen to figure out what the problems are
and summarize them here. Naveen you don't need to look at more tha
say the first 5 or 10 test failures since we're just trying to get a sense
of whether these are problems that are unique to Yocto or known upstream.
If any of the issues are known issues upstream, please provide a link
that we could track.
../Randy
Cheers,
Richard
--
# Randy MacLeod
# Wind River Linux
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170489):
https://lists.openembedded.org/g/openembedded-core/message/170489
Mute This Topic: https://lists.openembedded.org/mt/93526674/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
