On 2015-10-30, Kornel Benko wrote: > Am Freitag, 30. Oktober 2015 um 19:23:02, schrieb Scott Kostyshak > <skost...@lyx.org> >> On Fri, Oct 30, 2015 at 11:11:52PM +0000, Guenter Milde wrote: >> > On 2015-10-28, Scott Kostyshak wrote: >> > > On Mon, Oct 26, 2015 at 03:33:22PM +0000, Guenter Milde wrote: >> > >> > ... >> > >> > >> However, if the exit status of an export test changes (from fail >> > >> to pass or vice versa), we should check whether this is due to a >> > >> new bug, a fix or just exposing previously hidden problems. >> > >> > > Agreed. In fact, sometimes it is an improvement when a test fails. >> > > When I check manually, it was passing before but showing garbled >> > > text in a PDF output. Now it might fail with a clear message of >> > > "language xyz not supported". It is always good to check manually >> > > why something fails and if it passes if the PDF output is good. >> > >> > And, instead of re-establishing this with every status change, we >> > could have "tags" for inverted tests, distinguishing: >> > >> > * failure because of known permanent incompatiblity (good failure) >> > e.g. lyx -e latex ... for a document using non-TeX fonts >> > >> > * failure because of ERT or preamble code (not LyX's fault) >> > >> > * failure because of upstream bugs >> > >> > * failure because of known LyX bugs >> > >> > * failure for unknown reason with non-standard export route >> > e.g. XeTeX and TeX-fonts
>> Yes this would be nice. Right now I just try to put that information >> as a comment for why we invert a test, but it would be nice to have >> that information more easily available in the test summary. > I don't know how such an info can go to a summary. We could have separate files for the tests that we revert for different reasons. Or use a set of keywords. > Besides we have ATM about 200 failing export test cases. Too many to be helpfull. > The summary contains lines like: > Label Time Summary: > export = 59316.83 sec (3753 tests) > key = 0.26 sec (1 test) > reverted = 5631.52 sec (312 tests) > Even if we label some tests, the summary does not specify > how mane tests went wrong for a specified label. How many of these are for the obscure combination of Xetex and Tex fonts? While there is a use case for LuaTeX and TeX fonts, I can't see a reason to use Xetex instead of pdflatex with TeX fonts! Solving all issues that arise from this combination is diverting attention and ressources from more important tasks. Also, we identified some generic problems with this combination that are not solvable in the short term: third party packages well as documents not prepared for this use case. Just reverting failing Xetex export tests for the moment would allow us to concentrfate on the remaining failing test and get the test suite in a usable state again. Günter
