Am Samstag, 3. August 2013 um 03:54:46, schrieb Scott Kostyshak <skost...@lyx.org> > On Fri, Aug 2, 2013 at 5:55 AM, Kornel Benko <kor...@lyx.org> wrote: > > Am Freitag, 2. August 2013 um 11:10:33, schrieb Vincent van Ravesteijn > > <v...@lyx.org> > >> What about testing everything, and marking these tests as "known to fail". > > > >> In that way, if suddenly, LuaTex supports such a language, the tests will > > > >> tell us. By hardcoding that these languages are not tested, this > >> hardcoding > > > >> will be there for ever, even if some of the languages might get supported. > > > > > > > > The list could be read in CMakeLists.txt, then we can add a parameter to > > export.cmake > > > > to revert the meaning of success. > > I like these ideas. I created the file > development/expectedTestFailures partly for this reason (to know when > a test fails because of a regression and when a test file fails > because of something we already know), but I didn't think about > programmatically inputting such a file. > > My only hesitation with the inversion is that it is not intuitive: if > a Hebrew LuaTeX test passes, that means that it is broken, and if it > suddenly fails, that means that the export actually worked. This is > fine for you and me because we would know about the inversion. But if > anyone else ends up running the tests, how will they know? We could > have the LastTest.log explain this but I don't know if they would know > to look there. This could also be explained in > development/autotests/README, but again I don't know if someone will > read that. > > To be clear, I am in favor of this. But I wanted to explain my hesitation. > > Scott
We could mark the test as "shouldFail" or "shouldNotSucceed" (in the testname).
like
# ctest -R export/doc/attic/shouldNotSucceed_DocStyle_pdf
or, to check all (to be) unsuccesfull tests
#ctest -R shouldNotSucceed
Kornel
signature.asc
Description: This is a digitally signed message part.
