Hi, > -----Original Message----- > From: openembedded-core-boun...@lists.openembedded.org > [mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf > Of Markus Boos > Sent: Friday, October 24, 2014 12:37 > To: 'openembedded-core@lists.openembedded.org' > Subject: [OE-core] Discussion about architecure of ptest-runner patch > > Hi > > We patched the ptest-runner script to deliver a xUnit file as a rough > overview/indicator over the package tests. > Having such a report allows us to use the file in Jenkins and in our ALM tool > to > display the results of the ptest run. > > Before we send the patch upstream I like to start the discussion about the > architecture of the patch because I think it's worth to extend the ptest- > runner to deliver the results in readable text format (xUnit, JSON, html > table, > csv, whateverformat). > I think of something similar like the automated deployment mechanism > where yocto delivers the base class and you can write your own extension > with this class.
The concept of ptest is to run the tests using the minimum possible appropriate resource of a target to be tested. As per wiki page (https://wiki.yoctoproject.org/wiki/Ptest): "One major point of ptest is to consolidate the output format of all tests into a single common format. The format selected is the automake "simple test" format result: testname" As per automake manual (http://www.gnu.org/software/automake/manual/automake.html#Simple-Tests). "The possible results (whose meanings should be clear from the previous Generalities about Testing) are PASS, FAIL, SKIP, XFAIL, XPASS and ERROR" While I acknowledge that an automated mechanism to process and display the test results would be a nice piece of work for an automated testing framework, I don't think that this belongs to ptest (ptest-runner). The processing of the test results of an embedded tested device should not be done by the embedded device itself (inside ptest-runner in particular) but rather by the testing infrastructure (Jenkins? autobuild?) that eventually spawn ptest-runner. The next steps to improve ptest is to provide parallelism (that is to run parallel-tests for each feature tested and to run ptest for many feature in parallel). The work to be done to process the results (e.g. for a readable text format) should take into consideration this aspect. Regards, Tudor. > > What is the best way to add such a feature? > > Simply extend the ptest-runner script with the preferred output formatter? > > Or > > Write a bitbake class with an interface where you can attach your own > preferred output formatter? > > I'm sure there are other solutions possible as well. If you have one in mind, > please share it. > > Thank you. > > Best regards > Markus > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
