Am Freitag, 30. Oktober 2015 um 19:23:02, schrieb Scott Kostyshak <skost...@lyx.org> > On Fri, Oct 30, 2015 at 11:11:52PM +0000, Guenter Milde wrote: > > On 2015-10-28, Scott Kostyshak wrote: > > > On Mon, Oct 26, 2015 at 03:33:22PM +0000, Guenter Milde wrote: > > > > ... > > > > >> However, if the exit status of an export test changes (from fail to > > >> pass or vice versa), we should check whether this is due to a new bug, > > >> a fix or just exposing previously hidden problems. > > > > > Agreed. In fact, sometimes it is an improvement when a test fails. When > > > I check manually, it was passing before but showing garbled text in a > > > PDF output. Now it might fail with a clear message of "language xyz not > > > supported". It is always good to check manually why something fails and > > > if it passes if the PDF output is good. > > > > And, instead of re-establishing this with every status change, we could > > have "tags" for inverted tests, distinguishing: > > > > * failure because of known permanent incompatiblity (good failure) > > e.g. lyx -e latex ... for a document using non-TeX fonts > > > > * failure because of ERT or preamble code (not LyX's fault) > > > > * failure because of upstream bugs > > > > * failure because of known LyX bugs > > > > * failure for unknown reason with non-standard export route > > e.g. XeTeX and TeX-fonts > > Yes this would be nice. Right now I just try to put that information as > a comment for why we invert a test, but it would be nice to have that > information more easily available in the test summary.
I don't know how such an info can go to a summary.
Besides we have ATM about 200 failing export test cases.
The summary contains lines like:
Label Time Summary:
export = 59316.83 sec (3753 tests)
key = 0.26 sec (1 test)
reverted = 5631.52 sec (312 tests)
Even if we label some tests, the summary does not specify
how mane tests went wrong for a specified label.
> Scott
Kornel
signature.asc
Description: This is a digitally signed message part.
