On 2015-12-17, Kornel Benko wrote: > Am Donnerstag, 17. Dezember 2015 um 01:25:35, schrieb Scott Kostyshak > <skost...@lyx.org> >> A big thanks to Kornel and Günter for all of their work on the export >> tests. These will catch regressions and make us more confident in our >> releases going forward.
... >> What is a reasonable expectation of the output of 'ctest' with the >> following? >> -DLYX_ENABLE_EXPORT_TESTS=ON > Yesterday I had 0 failures with the command > #ctest -j12 -L export -E "xhtml|lyx16" > (about 2800 tests) > Today, after the many changes, I don't know. These are now reverted. Generally, as it is quite easy to invert a test or mark it as unreliable, the goal should be 0 test-failures. No test-failures means that all export-failures are recognized as one of * correct fail (e.g. TeX limitations prevent export to a certain format) * tolerable (e.g. failures for "exotic" documents with non-default output format) * unreliable (e.g. failures due to non-standard requirements, failures depending on local configuration, "erratic" behaviour (dependong on phase of the moon) * TODO item (e.g. LyX bugs that wait for someone to fix them or test-failures that still need proper categorization) It does not mean that there are no bugs or errors. Günter
