On 2021-08-20 10:48 p.m., Randy MacLeod wrote:
On 2021-08-20 6:06 a.m., Randy MacLeod wrote:
Added Trevor who may be working on upgrading python-cryptography.
Added Anatol since he was interested last time we worked on merging.
Again, let me know if you want to be dropped by replying privately.
No one from the meta-rust community responded last time so I'm
assuming that y'all are just on vacation! ;-)
On 2021-08-17 10:52 a.m., Randy MacLeod wrote:
On 2021-08-13 11:22 a.m., Vinay Kumar wrote:
Hi Randy,
Vinay or I will look into glibc/qemuppc64
if no one else is working on that.
The failure is due to the below error in libstd-rs package build.
"error: unrecognized arch "powerpc64le" in target specification"
The reason for this is "arch": "powerpc64le" from the below target
specific file.
,,,
Thanks Vinay, applied.
I'll push a new branch to poky-contrib later today.
I fixed and worked-around a few issues and pushed a new branch:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=rmacleod/rust-redo-aug-17-2021-a
1. There are only a few new commits including Vinay's:
757da4cbf1e rust: remove Rust version 1.51.0 toolchain
a4808f787ca cargo_common: remove http_proxy
b6941135c00 rust-common.inc: Fix build failure with qemuppc64.
If anyone wants to help by doing test builds, trying to generate and use
and SDK with rust in it or anything else, that'd be great.
Alex also ran a test build on the Autobuilder:
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/2439
but I have yet to look at the failed stages in the run.
Testing logs and debugging notes below for those interested.
../Randy
The test:
oe-selftest -r distrodata.Distrodata.test_maintainers
is fixed by creating maintainer records for all rusty packages
and by removing the older version of the rust toolchain:
757da4cbf1e rust: remove Rust version 1.51.0 toolchain
I suppose that if the meta-rust users want to keep an older TC
it can live in meta-rust.
Then, these tests:
oe-selftest -r sstatetests.SStateTests.test_sstate_noop_samesigs
oe-selftest -r sstatetests.SStateTests.test_sstate_sametune_samesigs
were failing. One issue was fixed by the work-around of:
a4808f787ca cargo_common: remove http_proxy
Debugging the next step was a challenge since I hadn't worked
with diffsigs before. Then to simplify, I removed all rusty .bb package
( by mv foo.bb
<https://urldefense.com/v3/__http://foo.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWpsy_PUvWQ$>
foo.bb.hide)
until I was down to the first recipe in the dependency tree:
rust-llvm-native.
and then running:
oe-selftest -r sstatetests.SStateTests.test_sstate_noop_samesigs \
sstatetests.SStateTests.test_sstate_sametune_samesigs \
2021-08-19 17:54:12,878 - oe-selftest - INFO
- Ran 2 tests in 142.010s
- OK
- RESULTS:
- RESULTS - sstatetests.SStateTests.test_sstate_noop_samesigs:
PASSED (41.45s)
- RESULTS - sstatetests.SStateTests.test_sstate_sametune_samesigs:
PASSED (98.89s)
- SUMMARY:
- oe-selftest () - Ran 2 tests in 142.012s
- oe-selftest - OK - All required tests passed
(successes=2, skipped=0, failures=0, errors=0)
adding the next self consistent group of package:
cargo/cargo-cross-canadian_1.54.0.bb
<https://urldefense.com/v3/__http://cargo-cross-canadian_1.54.0.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWpvCxIrDwA$>
cargo/cargo_1.54.0.bb
<https://urldefense.com/v3/__http://cargo_1.54.0.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWptRWUAvyA$>
rust/libstd-rs_1.54.0.bb
<https://urldefense.com/v3/__http://libstd-rs_1.54.0.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWpuNZ1zxTA$>
rust/rust-cross-canadian_1.54.0.bb
<https://urldefense.com/v3/__http://rust-cross-canadian_1.54.0.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWpvu8Kgohw$>
rust/rust-cross_1.54.0.bb
<https://urldefense.com/v3/__http://rust-cross_1.54.0.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWpuYvgjUig$>
rust/rust-tools-cross-canadian_1.54.0.bb
<https://urldefense.com/v3/__http://rust-tools-cross-canadian_1.54.0.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWpv42uDIpQ$>
rust/rust_1.54.0.bb
<https://urldefense.com/v3/__http://rust_1.54.0.bb__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWps4C8-W7g$>
I see another http_proxy failure that seems to require a feature
drop in cargo:
https://github.com/rust-lang/cargo/blob/master/src/cargo/ops/registry.rs#L524
Patching out a feature doesn't seem like a good idea but
maybe someone else has a better one?
To see if there were other problems lurking menacingly behind the proxy
error, I disabled the http_proxy test in:
meta/lib/oeqa/selftest/cases/sstatetests.py
<https://urldefense.com/v3/__http://sstatetests.py__;!!AjveYdw8EvQ!IgwYWcWmCub5aQLbkRB6aDq3QRH6tsJ6nak3hdCaB0jy1kp9MmL2hQHzPNhpWptCntmNRQ$>
-http_proxy = "http://example.com/";
+http_proxy = ""
and ran again to find:
- RESULTS - sstatetests.SStateTests.test_sstate_noop_samesigs:
PASSED (50.63s)
- RESULTS - sstatetests.SStateTests.test_sstate_sametune_samesigs:
FAILED (160.05s)
so that's somewhat encouraging but there is still the error:
File ".../meta/lib/oeqa/selftest/cases/sstatetests.py", line 449,
in test_sstate_sametune_samesigs
self.assertCountEqual(files1, files2)
AssertionError: Element counts were not equal:
First has 1, Second has 0:
'/ala-lpggp31/rmacleod/src/distro/yocto/b/aug-19-a-st/\
tmp-sstatesamehash/stamps/x86_64-linux/\
lib32-rust-cross-i686/1.54.0-r0.do_rust_gen_targets.sigdata.\
fc54579cc139c777172242c3da3e0'
First has 1, Second has 0:
'/ala-lpggp31/rmacleod/src/distro/yocto/b/aug-19-a-st/\
tmp-sstatesamehash/stamps/x86_64-linux/\
lib32-rust-cross-i686/1.54.0-r0.do_compile.sigdata.\
86cb5a1346b0615cb21fab3c0110cc964ecced'
... lots of similar lines of diffsig info ...
Hacking the test to not clean up and
running bitbake-dumpsigs on the two sig files for the first error,
I get the two files attached. I also attached the
tmp/log/oe-selftest-results.log file.
That's it for now.
I think I've gotten to the bottom of a chain of sigs that differ in
part due to:
"Hash for dependent task" differing and found that on leaf node is:
stamps/x86_64-linux/rust-cross-i686/1.54.0-r0.do_rust_gen_target
See below for my notes and my assertion/question about the next step
needing
to be deterministically ordering the elements used in rust_gen_target.
../Randy
With a patch from Richard, and the http_proxy check removed,
$ git diff | grep "^[+-]"
--- a/meta/lib/oeqa/selftest/cases/sstatetests.py
+++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
-http_proxy = "http://example.com/";
+http_proxy = ""
we have fixed the oe-selftest that was failing.
Any volunteers to stub out the http[s]_proxy code in cargo ?
I've written up a commit log that could use some review and pushed an
update to:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rmacleod/rust-redo-aug-17-2021-a
I starting to work on the YP autobuilder issues.
I can build:
$ bitbake cargo-cross-canadian-x86-64
Adding this:
require conf/multilib.conf
MULTILIBS = 'multilib:lib32'
DEFAULTTUNE:virtclass-multilib-lib32 = 'x86'
to my local.conf, I see that:
$ bitbake lib32-rust
fails with the error:
---
failed to execute command:
"/ala-lpggp31/rmacleod/src/distro/yocto/b/aug-19-a/tmp/work/x86-pokymllib32-linux/lib32-rust/1.54.0-r0/lib32-recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config"
"--bindir"
| error: No such file or directory (os error 2)
---
Khem, a MUSL build fails:
$ TCLIBC=musl bitbake rust-hello-world
as shown below (1) but I'll see if that's reproducible with a clean build.
I've also rebased to poky master:
3fd2dff4eb3 (origin/master, origin/HEAD, master) ref-manual: Fix
reference to bbappend section of dev-manual
and the two key oe-selftests still pass with the http_proxy work-around
as does core-image-minimal+rust-hello-world (2) so I've pushed a new
branch:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rmacleod/rust-redo-aug-21-2021-a
Richard, Alex,
Is it worth another AB run?
../Randy
1)
$ TCLIBC=musl bitbake rust-hello-world
ERROR: rust-cross-x86_64-1.54.0-r0 do_prepare_recipe_sysroot: Error
executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure
was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:extend_recipe_sysroot(d)
0003:
File:
'/ala-lpggp31/rmacleod/src/distro/yocto/poky-contrib.git/meta/classes/staging.bbclass',
lineno: 582, function: extend_recipe_sysroot
0578: if "/bin/" in l or "/sbin/" in l:
0579: # defer /*bin/* files until last in
case they need libs
0580: binfiles[l] = (targetdir, dest)
0581: else:
*** 0582: staging_copyfile(l, targetdir, dest,
postinsts, seendirs)
0583:
0584: # Handle deferred binfiles
0585: for l in binfiles:
0586: (targetdir, dest) = binfiles[l]
File:
'/ala-lpggp31/rmacleod/src/distro/yocto/poky-contrib.git/meta/classes/staging.bbclass',
lineno: 149, function: staging_copyfile
0145: if os.path.islink(c):
0146: linkto = os.readlink(c)
0147: if os.path.lexists(dest):
0148: if not os.path.islink(dest):
*** 0149: raise OSError(errno.EEXIST, "Link %s already
exists as a file" % dest, dest)
0150: if os.readlink(dest) == linkto:
0151: return dest
0152: raise OSError(errno.EEXIST, "Link %s already
exists to a different location? (%s vs %s)" % (dest, os.readlink(dest),
linkto), dest)
0153: os.symlink(linkto, dest)
Exception: FileExistsError: [Errno 17] Link
/ala-lpggp31/rmacleod/src/distro/yocto/b/aug-19-a/tmp/work/x86_64-linux/rust-cross-x86_64/1.54.0-r0/recipe-sysroot/usr/include/c++/11.2.0/x86_64-poky-linux
already ex'
ERROR: Logfile of failure stored in:
/ala-lpggp31/rmacleod/src/distro/yocto/b/aug-19-a/tmp/work/x86_64-linux/rust-cross-x86_64/1.54.0-r0/temp/log.do_prepare_recipe_sysroot.192760
ERROR: Task
(/ala-lpggp31/rmacleod/src/distro/yocto/poky-contrib.git/meta/recipes-devtools/rust/rust-cross_1.54.0.bb:do_prepare_recipe_sysroot)
failed with exit code '1'
2) There is a warning for rust-native:
grep WARNING
tmp/work/x86_64-linux/rust-native/1.54.0-r0/temp/log.do_populate_sysroot
| fmt
WARNING: File
'/ala-lpggp31/rmacleod/src/distro/yocto/b/aug-21-a/tmp/work/x86_64-linux/rust-native/1.54.0-r0/recipe-sysroot-native/usr/lib/rustlib/x86_64-linux/bin/rust-llvm-dwp'
from rust-native was already stripped, this will prevent future debugging!
I did learn a bit about this test code that is failing:
https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/sstatetests.py#n404
as the comment explains:
|
The sstate checksums of two identical machines (using the same tune)
should be the
same, apart from changes within the machine specific stamps directory.
We use the
qemux86copy machine to test this. Also include multilibs in the test.|
so the good news is that there are: 23185files being checked and
only 43 are different and if you look at what packages and arches
differ it's:
x86_64-linux/lib32-rust-cross-i686
x86_64-linux/rust-cross-i686
core2-32-poky-linux/rust
core2-32-poky-linux/libstd-rs
x86-pokymllib32-linux/lib32-libstd-rs
x86-pokymllib32-linux/lib32-rust
Note that these are all 32bit multilib builds.
Maybe that's not so important for the initial merge to oe-core... Richard?
Looking at core2-32-poky-linux/rust the stages that have different
signatures are:
do_configure
do_populate_sysroot
do_package_qa
do_install
do_package
do_packagedata
do_compile
do_prepare_recipe_sysroot
do_package_write_rpm
9 of the 19 stages. Ok so maybe it's not great news after all. ;-)
I'll kept digging down through the tree of dependant diffsigs
since when looking at do_prepare_recipe_sysroot, it was clear
that a key difference in it's signatures was due to:
"Hash for dependent task ..."
differing. Finally I found a set of signatures that did not have a
diff in the dependent task:
stamps/x86_64-linux/rust-cross-i686/1.54.0-r0.do_rust_gen_target*
The simplified diff is below and the files are attached.
There are a variety of variables that differ only in order, such as:
-List of dependencies for variable TUNE_FEATURES is
{'TUNE_FEATURES:tune-core2-32', 'DEFAULTTUNE'}
+List of dependencies for variable TUNE_FEATURES is {'DEFAULTTUNE',
'TUNE_FEATURES:tune-core2-32'}
I assume the next step is to find out where those variables are produced:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/rust/rust-common.inc?h=rmacleod/rust-redo-aug-10-2021-a#n292
and to introduce deterministic order when building them.
../Randy
diff -Naur x86_64-rust-cross-i686-rust-gen-targets/* | grep "^[+-]" |
cut -c -166
---
x86_64-rust-cross-i686-rust-gen-targets/1.rust-cross-i686.do_rust_gen_target
2021-08-20 22:19:19.883320107 -0400
+++
x86_64-rust-cross-i686-rust-gen-targets/2.rust-cross-i686.do_rust_gen_target
2021-08-20 22:19:19.651319015 -0400
-basewhitelist: {'CCACHE_TOP_DIR', 'SOURCE_DATE_EPOCH',
'SDKPKGSUFFIX', 'BB_WORKERCONTEXT', 'FILESEXTRAPATHS', 'HOME',
'OMP_NUM_THREADS', 'PRSERV_DUMPDIR', 'PRSERV_HO
+basewhitelist: {'PWD', 'SSTATE_HASHEQUIV_OWNER', 'STAMPS_DIR',
'CCACHE', 'BB_WORKERCONTEXT', 'COREBASE', 'BBPATH', 'HOME', 'TMPDIR',
'ERROR_QA', 'SSTATE_HASHEQUIV_ME
-basehash:
bd66629631e757dcccfc103c13e7de6ac3568677df3ef5ad47dd99e62427caab
+basehash:
d697cf2cbb5807e83f8b55de8012d5bb9ab8674f4bbf95587fc3ca822a47269b
-List of dependencies for variable TUNE_ARCH is {'X86ARCH32',
'X86ARCH64'}
+List of dependencies for variable TUNE_ARCH is {'X86ARCH64',
'X86ARCH32'}
-List of dependencies for variable TUNE_FEATURES is
{'TUNE_FEATURES:tune-core2-32', 'DEFAULTTUNE'}
+List of dependencies for variable TUNE_FEATURES is {'DEFAULTTUNE',
'TUNE_FEATURES:tune-core2-32'}
-List of dependencies for variable llvm_cpu is {'PACKAGE_ARCH',
'TUNE_FEATURES', 'TRANSLATED_TARGET_ARCH'}
-List of dependencies for variable llvm_features is
{'llvm_features_from_cc_arch', 'llvm_features_from_tune',
'llvm_features_from_target_fpu'}
+List of dependencies for variable llvm_cpu is {'PACKAGE_ARCH',
'TRANSLATED_TARGET_ARCH', 'TUNE_FEATURES'}
+List of dependencies for variable llvm_features is
{'llvm_features_from_target_fpu', 'llvm_features_from_cc_arch',
'llvm_features_from_tune'}
-List of dependencies for variable llvm_features_from_tune is
{'target_is_armv7', 'TUNE_FEATURES', 'ARM_THUMB_OPT', 'MACHINEOVERRIDES'}
+List of dependencies for variable llvm_features_from_tune is
{'ARM_THUMB_OPT', 'TUNE_FEATURES', 'target_is_armv7', 'MACHINEOVERRIDES'}
-List of dependencies for variable rust_gen_target is {'sys_for',
'arch_to_rust_target_arch', 'RUST_PANIC_STRATEGY', 'prefix_for'}
+List of dependencies for variable rust_gen_target is {'sys_for',
'prefix_for', 'arch_to_rust_target_arch', 'RUST_PANIC_STRATEGY'}
-List of dependencies for variable target_is_armv7 is {'TARGET_ARCH',
'TUNE_FEATURES', 'MACHINEOVERRIDES'}
+List of dependencies for variable target_is_armv7 is
{'TUNE_FEATURES', 'TARGET_ARCH', 'MACHINEOVERRIDES'}
-Variable MACHINE value is qemux86
-Variable MACHINEOVERRIDES value is
${@bb.utils.contains('TUNE_FEATURES', 'mx32', 'x86-x32:', '',
d)}${@bb.utils.contains('TUNE_FEATURES', 'm32', 'x86:', '', d)}qemua
+Variable MACHINE value is qemux86copy
+Variable MACHINEOVERRIDES value is
${@bb.utils.contains('TUNE_FEATURES', 'mx32', 'x86-x32:', '',
d)}${@bb.utils.contains('TUNE_FEATURES', 'm32', 'x86:', '', d)}qemua
-Computed base hash is
bd66629631e757dcccfc103c13e7de6ac3568677df3ef5ad47dd99e62427caab and
from file
bd66629631e757dcccfc103c13e7de6ac3568677df3ef5ad47dd99e62427caab
-Computed task hash is
448a8113230a16ac5d606de18989f98b3700edeb3e7956c6f02207aa76a5d4f3
+Computed base hash is
d697cf2cbb5807e83f8b55de8012d5bb9ab8674f4bbf95587fc3ca822a47269b and
from file
d697cf2cbb5807e83f8b55de8012d5bb9ab8674f4bbf95587fc3ca822a47269b
+Computed task hash is
83d1a26ebe7e2df3f496024a297ba782e145e6982457e09102d5e7c4fbf1b3b7
../Randy
../Randy
On Fri, Aug 13, 2021 at 8:50 PM Vinay Kumar
<vinay.m.e...@gmail.com> wrote:
rust-common.inc: Fix build failure with qemuppc64.
The glibc build of "rust-hello-world" throws error in libstd-rs
package.
error: unrecognized arch "powerpc64le" in target specification
The same got fixed by changing the arch to "powerpc64".
Signed-off-by: Vinay Kumar <vinay.m.e...@gmail.com>
---
meta/recipes-devtools/rust/rust-common.inc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-devtools/rust/rust-common.inc
b/meta/recipes-devtools/rust/rust-common.inc
index 350517c723..f574152815 100644
--- a/meta/recipes-devtools/rust/rust-common.inc
+++ b/meta/recipes-devtools/rust/rust-common.inc
@@ -248,6 +248,8 @@ def arch_to_rust_target_arch(arch):
return "mips64"
elif arch == "armv7":
return "arm"
+ elif arch == "powerpc64le":
+ return "powerpc64"
else:
return arch
--
2.31.1
--
# Randy MacLeod
# Wind River Linux
--
# Randy MacLeod
# Wind River Linux
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155068):
https://lists.openembedded.org/g/openembedded-core/message/155068
Mute This Topic: https://lists.openembedded.org/mt/85017687/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
