Hi Richard, For oe-selftest -r runqemu,meta_ide, configuration we are using is only changing the machine type.
For adding "who" identifier in resulttool command, we can take it up as an enhancement. Thanks & Regards, Sangeeta Jain >-----Original Message----- >From: richard.pur...@linuxfoundation.org <richard.pur...@linuxfoundation.org> >Sent: Wednesday, 3 April, 2019 2:32 AM >To: Jain, Sangeeta <sangeeta.j...@intel.com>; 'yocto@yoctoproject.org' ><yocto@yoctoproject.org>; Eggleton, Paul <paul.eggle...@intel.com>; 'Michael >Halstead' <mhalst...@linuxfoundation.org>; Erway, Tracey M ><tracey.m.er...@intel.com>; sjolley.yp...@gmail.com; Burton, Ross ><ross.bur...@intel.com> >Cc: OTC Embedded Malaysia <otc.embedded.malay...@intel.com> >Subject: Re: QA cycle report for 2.7 M3 RC1 > >Hi Sangeeta/Ee Peng, > >On Tue, 2019-04-02 at 05:02 +0000, Jain, Sangeeta wrote: >> QA cycle report for 2.7 M3 RC1: >> >> No high milestone defects. >> This is completion of first QA cycle after adoption of new QA process. >> Test results are available at following location: >> · For results of all automated tests, please refer to results >> at public AB [1]. >> · For other test results, refer to attachment [2]. >> · For test report for test cases run by Intel and WR team, >> refer attachment [3] >> · For full test report, refer attachment [4] >> 2 new defects are found in this cycle, oe-selftest [5] and qemu- >> shutdown[6]. >> Number of existing issues observed in this release is 3- toaster [7], >> Build-appliance [8] and Beaglebone [9] For ptest, 3 package tests are >> failing in this release which were passing in previous release: python >> [10], python3[11] and strace[12]. >> 3 packages are facing timeout issues: lttng-tools [13], openssh [14] >> and python3 [15]. > >Thanks for this. I have some questions/observations. > >Looking at the Intel/WR report, I see tests which start oeselftest_XXX for sdk- >extras and qemu-shutdown. I think this means you have some QA test >infrastructure running "oe-selftest -r runqemu,meta_ide", iterating over >MACHINE=qemuppc, qemumips, qemuarm, qemux86? > >Is this some automation we're missing in the autobuilder configuration? >We all other tests covered and this is the only remaining automated piece we >haven't covered? What configuration are we missing exactly for the missing >tests? Just run for MACHINE=qemuppc /qemumips / qemuarm / qemux86 > >I'm partly wondering how you found the shutdown issue but this would explain >that. > >I think going forward we need to add some kind of "who" identifier to the test >results so we can say where/who they came from (main autobuilder, Intel, WR, >anywhere else). We could add a resulttool command to "stamp" a set of test >results with this, maybe as a parameter to the store command? Will work on resulttool to add this enhancement. Though not sure about timeline as of now. > >Of the bugs, I didn't see any which I think should block M3, I think we >probably >should release that and move forward to M4. I do want to see some of these >fixed before we build M4 though. I've made comments on them below. > >> New Bugs : >> >> [5] Bug 13237 - [2.7 M3 RC1][OE-selftest][Opensuseleap 15.0][Public >> AB]:"test_wic_cp_ext" failing for Opensuseleap 15.0. >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13237 > >Ross: Are you looking into that one? > >> [6] Bug 13234 - [2.7 M3 RC1] qemumips & qemumips64: failed to shutdown >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13234 > >This needs looking into and fixing, as well as figuring out why the autobuilder >doesn't see this (see above for my hypothesis). > >> Previous Bugs observed in this release: >> >> [7] Bug 13191 – [Test Case 1439] Build Failure after adding or >> removing packages with and without dependencies >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13191 >> >> [8] Bug 12991 - [Bug] [2.6 M4 RC1][Build-Appliance] Bitbake build- >> appliance-image getting failed during building image due to webkitgtk >> package. >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12991 > >We need to do something about this. I believe its related to resource >starvation in >the VM. > >> [9] Bug 13153 – [2.7 M2 RC1] Systemtap doesn't work on beaglebone >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13153 > >I'm hoping a SRCREV bump for beaglebone-yocto fixes this. > >> ptest Bugs: >> >> [10] Bug 13249 - [2.7 M3 RC1] python ptest failure >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13249 > >I'm hoping a patch from Ross in master fixes this, need to retest ptest with >this >applied. > >> [11] Bug 13253 - [2.7 M3 RC1] python3 ptest failure >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13253 > >I'm hoping a patch from Ross in master fixes this, need to retest ptest with >this >applied. > >> [12] Bug 13254 - [2.7 M3 RC1] strace ptest failure >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13254 > >No ideas on this one yet. > >> [13] Bug 13255 - [2.7 M3 rc1] lttng-tools ptest facing timeout issue >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13255 > >There are patches which are due to arrive soon which should address this. > >> [14] Bug 13256 - [2.7 M3 rc1] openssh ptest facing timeout issue >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13256 > >No ideas on this one yet. > >> [15] Bug 13257 - [2.7 M3 rc1] python3 ptest facing timeout issue >> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13257 > >I'm hoping a patch from Ross in master fixes this, need to retest ptest with >this >applied. > >Cheers, > >Richard -- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
