On 7/5/22 6:18 AM, Sundeep KOKKONDA wrote:
Hello Richard,
-----Original Message-----
From: Richard Purdie <richard.pur...@linuxfoundation.org>
Sent: 04 July 2022 14:20
To: Sundeep KOKKONDA <sundeep.kokko...@gmail.com>; openembedded-
c...@lists.openembedded.org
Subject: Re: [OE-core] [kirkstone] Binutils, Glibc: Updated with latest commits
On Mon, 2022-07-04 at 00:51 -0700, Sundeep KOKKONDA wrote:
The regression test for Binutils and Glibc is performed.
1) Binutils test shows no regression after update.
2) Glibc test result:
Test summary before glibc update on Kirkstone:
204 FAIL
4320 PASS
43 UNSUPPORTED
12 XFAIL
6 XPASS
Test summary after glibc update on Kirkstone:
204 FAIL
4339 PASS
41 UNSUPPORTED
12 XFAIL
6 XPASS
Test results analysis:
Total 4585 test cases before Glibc update & Total 4602 test cases
after Glibc update.
17 new test cases added (out of those 16 PASS & 1 FAIL)
12 new FAIL test cases after glibc update
14 new PASS test cases after glibc update
Do we know why some of the failing test cass now pass and why some of the
passing test cases now fail? Is this some issue with the way we're testing
things?
I'm wondering if this is "normal" noise between test runs and if there is some
underlying instability or glibc changes do cause changes in pass/fails of tests
like
this in general?
Hello Richard,
The test results are impacted by Execution procedure OR by Changes in glibc.
Execution procedure:
I tested the glibc with the cross-compiler method given in
https://sourceware.org/glibc/wiki/Testing/Testsuite for the aarch64
architecture on qemu.
However, based on our past test experience, we're following a few additional
steps like copying the 'glibc-locale/2.28-r0/image/usr/*' to booted qemu image
/usr/ directory (This step is not mentioned the glibc wiki) etc.
Also, the glibc test is aborted when the -Werror=format-security flag is used
and to get the test done we've removed this error flag and compiled glibc
sources.
Since the glibc test not performed very frequently, I am not sure the
additional steps what we followed are obsolete or not. I am now working on
adhering to the test procedure mentioned in the glibc test wiki and will see
the results.
Perhaps run few iterations on unmodified setup and see if the results
vary. If they do then we know that there are some jitter to look into.
otherwise, its perhaps a combination of updates which would also be good
to know. Failing one's are more important to look into as they may cause
unwanted regressions.
Changes in glibc:
To comment on the code change impact on the test failures I've to look into
each failed test case by case and what are the changes in glibc and how it is
impacting the test.
I'll keep you posted on this and since the glibc test execution is taking a lot
of time my response may be delayed.
Thanks,
Sundeep K.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167656):
https://lists.openembedded.org/g/openembedded-core/message/167656
Mute This Topic: https://lists.openembedded.org/mt/92160650/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
