On Thu, Jan 28, 2016 at 07:07:43AM +0000, Guenter Milde wrote:
> On 2016-01-27, Kornel Benko wrote:
> 
> > The following tests FAILED:
> >     772 - INVERTED.MANUALS.TODO_export/doc/es/Customization_dvi3_systemF 
> > (Failed)
> >     777 - INVERTED.MANUALS.TODO_export/doc/es/Customization_pdf4_systemF 
> > (Failed)
> >     779 - INVERTED.MANUALS.TODO_export/doc/es/Customization_pdf5_systemF 
> > (Failed)
> 
> Good news, seems like the translation solved some issues.
> See the comment in Suspici...
> 
>   # language nesting (may disappear after completed translation)
> 
> We still have the failing dedicated sample.
> 
> >     1228 - MANUALS_export/doc/ja/UserGuide_pdf3 (Failed)
> 
> Not related to siggraph.
> 
> >     3365 - TEMPLATES_export/templates/ACM-siggraph_dvi (Failed)
> ...
> 
> It works with the new version. However, as these templates are for a
> not-on-CTAN class, I vote to add them to
>   unreliable:nonstandard
> in any case.

Fine with me. When you add the comment, to be consistent, should we say
"not-on-TeXLive class" ?

The documentation currently states:

  nonstandard: Documents with additional requirements, e.g. a class or
               package file not in TeXLive.

They are probably closely related, but apparently not equivalent:

http://tex.stackexchange.com/questions/185495/how-can-a-package-be-listed-on-ctan-but-not-be-available-in-tex-live

I don't have a strong preference on this so if you think something else
makes more sense, go ahead.

Scott

> The docs are docs, if using them directly as test samples makes problems,
> we should use a copied version for the tests.

We should write down the advantages and disadvantages of this. One
advantage is that we are more likely to catch LyX bugs. For example,
when Uwe added English to the Japanese docs, they started to fail (I
think you refer to this above). This is probably a LyX bug that we would
not catch if we just fixed the documents that we test. Some
disadvantages are that there are more noise and frustration.

My opinion is that we should not use fixed versions of the documentation
to test. We should have unit tests that have fixed versions, and I hope
that we simulataneously build up our unit tests. But for these export
tests I think we will miss bugs if we only test the fixed versions. I
think it is worth the extra noise and that in the long-run we will
develop a policy that is agreed upon, not too invasive, and thus we will
eventually improve the signal-to-noise ratio.

Scott

Attachment: signature.asc
Description: PGP signature

Reply via email to