Am Donnerstag, 3. Dezember 2015 um 22:08:35, schrieb Guenter Milde <mi...@users.sf.net> > On 2015-12-02, Kornel Benko wrote: > > Am 2. Dezember 2015 um 22:07:36, schrieb Guenter Milde <mi...@users.sf.net> > > Dear Kornel, > > ... > > >> It would make it easier to work with the labels, if we write the > >> complete "hierarchy" into sublabels. > > > Let my last commit settle a little. I try to implement, but only step > > by step. ATM the labels are hierarchical. > > A step in the right direction ;-) > > However, now we have 26 labels. > > Generally, labels are sensible to select subsets that cannot easily be > selected via regular expressions. > > Are labels for the (sub-)locations really required? > > Why not use, e.g. `-R "export/examples/"` > instead of `-L export:examples` in case there is need to select just > examples (and not manuals or templates). >
Because -R does not categorize.
If you want to test examples, simply use '-L examples'
Since sublabels are are (and hopefully stay) unique, there is no problem.
Using '-R examples' dos all test having 'examples' in testname. This is not the
same.
E.g. (hypothetica) test 'check_example_urls'.
> If you prefer to keep them, I'd prefer separate labels for place and
> status and also clearly mark that "suspended" is a subset of "inverted"
> (this is especially important for the output: clearly indicate whether a
> fail is an export fail or an fail to fail).
It is.
> cmplyx
> export
> export:autotests
> export:examples
> export:manuals
> export:mathmacros
> export:templates
> inverted
> inverted:suspended
> suspended:suspended:chemgreek
What is this?
Sublabels should not be doubled.
> layout
> load
> lyx2lyx
> lyxlyx
> module
> roundtrip
> unreliable:erratic
> unreliable:nonstandard
>
>
> If the labels "inverted" and "unreliable" are only intended for export
> tests, the could be used with "export:" suffix:
No, also other tests may be inverted. This is a feature from ctest, not ours.
To our convenience the tests are also labeled.
> export:inverted
> export:inverted:chemgreek
> export:inverted:suspended
> export:inverted:suspended:chemgreek
> export:unreliable:erratic
> export:unreliable:nonstandard
>
> Then, it becomes easier find the description in the documentation.
Yes, 'suspended' means also inverted. But we do NOT WANT to test 'suspended' if
we test only '-L inverted'
Therefore it is NOT labeled as 'inverted'
> ...
>
>
>
> >> >> A test matching a regexp in UnreliableTests and
> >> - InvertedTests, say, would get the labels "unreliable" and
> >> "inverted".
> >> + InvertedTests, say, would get the labels "export:unreliable"
> >> + and "export:inverted".
>
>
> > No, any setting for "unreliable" is wrong.
> > For a test in "unreliable" may for me be good to set to inverted, but for
> > you
> > it may be exactly reverse. That's why the test is unreliable.
>
> I know its unreliable and will fail at some place or time.
>
> However, we may still want to invert a test, if we know the result
> to be wrong even if it compiles without error.
>
Sure, but then it will be another class. Not implemented yet.
If it is unreliable, then it is not tested in '-L export'.
And for us it is completely irrelevant if the test succeeds or not.
Setting inverted does not change this situation. Moreover, tests with '-L
inverted' would
also test this one. I don't want it to.
> For this "double tagging" to work, you would have to change the logic:
> instead of applying the filters in a chain, the pattern files would be
> applied to all (non-ignored) tests: test that match get the corresponding
> label.
>
This can always be done. I don't understand.
...
>
> > Everyone is free to add sublabels.
>
> Yes. But my idea is to use "minor" and/or "wontfix" *instead* of the
> "suspended" pattern file.
>
Not my idea. For me 'suspended' means don't care, rare use case, ignore for a
while.
> > But used sublabels for example in
> > 'suspicious' should not match sublabel names in 'suspended'.
Why? As I already wrote more than once, 'suspended' is meant as a more general
filter.
Please look into the file.
We have many tests failing because of combination pdf4_texF. We do not want to
bother with them now.
So in suspendedTests we choose '.*pdf4_texF'.
We normally do not choose a single test here.
ATM there are exacly 2 (two) regexes there.
...
> >> >> Do we want a label for all tests with low signal/noise ratio?
>
> >> > I want.
>
> >> >> How to name this label?
>
> >> Suggestions: fragile, delicate, unstable
>
> ... fussy, shaky, vulnerable
>
> IMV this should be decided by a native English speaker.
>
OK
> Günter
Kornel
signature.asc
Description: This is a digitally signed message part.
