On Thu, 2017-07-20 at 10:09 +0800, Robert Yang wrote:
>
> On 07/19/2017 04:16 PM, Robert Yang wrote:
> >
> > Hello,
> >
> > These patches can make ptest test case work, RP has suggested we
> > write a tool to
> > do the regression check on ptest result, I think that the use case
> > is like:
> >
> > $ bitbake <image> -ctestiamge # Suppose we add ptest to default
> > test cases in the future
> > # Upgrade a recipe form V1.0 to V1.1
> > $ bitbake <image> -ctestiamge # Run the test again
> >
> > Then the regression check tool can report what's different (passed,
> > failed,
> > skipped) between V1.0 and V1.1.
> >
> > Currently, I'm not sure about where to save the ptest results, I
> > saved it to
> > ${WORKDIR}/testimage/ptest_log atm, e.g.:
> > tmp/work/qemux86-poky-linux/core-image-minimal/1.0-
> > r0/testimage/ptest_log
> I still have to think about more about where to put the test result.
Would buildhistory make sense for these since we already have a
mechanism to store results there between builds?
> > But it will be removed after -cclean, then no regression check can
> > be made, so
> > I'd like to save the ptest results to DEPLOY_DIR_IMAGE if no
> > objections, and
> > make -cclean not remove them. (Or only keep the latest 2 results).
> >
> > And I'm not sure where to add the regression check/tool, maybe one
> > of:
> > 1) Add a separate tool in oe-core/scripts, this can make it easy to
> > do regression
> > check among different build directories, and
> > runtime/cases/_ptest.py can invoke it.
> I'm leaning to add a script called ptest-regression-check which is
> more flexible, the cases/ptest.py or buildhistory (also the user) can
> run it when needed.
I'd either add a script, or perhaps an option to the buildhistory-diff?
Paul might have some adivce on that. I would like to understand if
there is a good reason we can't use buildhistory to at least help with
some of this?
Cheers,
Richard
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
