Hi Tudor Thank you for your explanation.
We made a small sed/awk script to format the googletest output to the ptest format. We published it on the yocto automated-testing list: https://lists.yoctoproject.org/pipermail/automated-testing/2014-November/000091.html BR Markus > -----Ursprüngliche Nachricht----- > Von: Tudor Florea [mailto:tudor.flo...@enea.com] > Gesendet: Samstag, 25. Oktober 2014 02:57 > An: Boos Markus; 'openembedded-core@lists.openembedded.org' > Betreff: RE: Discussion about architecure of ptest-runner patch > > Hi, > > > -----Original Message----- > > From: openembedded-core-boun...@lists.openembedded.org > > [mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of > > Markus Boos > > Sent: Friday, October 24, 2014 12:37 > > To: 'openembedded-core@lists.openembedded.org' > > Subject: [OE-core] Discussion about architecure of ptest-runner patch > > > > Hi > > > > We patched the ptest-runner script to deliver a xUnit file as a rough > > overview/indicator over the package tests. > > Having such a report allows us to use the file in Jenkins and in our > > ALM tool to display the results of the ptest run. > > > > Before we send the patch upstream I like to start the discussion about > > the architecture of the patch because I think it's worth to extend the > > ptest- runner to deliver the results in readable text format (xUnit, > > JSON, html table, csv, whateverformat). > > I think of something similar like the automated deployment mechanism > > where yocto delivers the base class and you can write your own > > extension with this class. > > The concept of ptest is to run the tests using the minimum possible > appropriate > resource of a target to be tested. > As per wiki page (https://wiki.yoctoproject.org/wiki/Ptest): > "One major point of ptest is to consolidate the output format of all tests > into a single > common format. The format selected is the automake "simple test" format > result: testname" > As per automake manual > (http://www.gnu.org/software/automake/manual/automake.html#Simple-Tests). > "The possible results (whose meanings should be clear from the previous > Generalities about Testing) are PASS, FAIL, SKIP, XFAIL, XPASS and ERROR" > While I acknowledge that an automated mechanism to process and display the > test > results would be a nice piece of work for an automated testing framework, I > don't > think that this belongs to ptest (ptest-runner). > The processing of the test results of an embedded tested device should not be > done > by the embedded device itself (inside ptest-runner in particular) but rather > by the > testing infrastructure (Jenkins? autobuild?) that eventually spawn > ptest-runner. > > The next steps to improve ptest is to provide parallelism (that is to run > parallel-tests > for each feature tested and to run ptest for many feature in parallel). The > work to be > done to process the results (e.g. for a readable text format) should take > into > consideration this aspect. > > Regards, > Tudor. > > > > > What is the best way to add such a feature? > > > > Simply extend the ptest-runner script with the preferred output formatter? > > > > Or > > > > Write a bitbake class with an interface where you can attach your own > > preferred output formatter? > > > > I'm sure there are other solutions possible as well. If you have one > > in mind, please share it. > > > > Thank you. > > > > Best regards > > Markus > > -- > > _______________________________________________ > > Openembedded-core mailing list > > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
