On 2016-01-28, Kornel Benko wrote:
> Am Donnerstag, 28. Januar 2016 um 08:48:48, schrieb Guenter Milde
> <mi...@users.sf.net>
...
>> back to a place for failing test samples¹.
>> Alternatives:
>> a) keep them local (this is what most developers currently do).
>> +1 simple, good if you work alone on an issue
>> -1 no collaboration
>> b) put to autotests/export/ and add a commented inversion pattern
>> +1 no need to move the sample once it works
>> +1 problem description can be seen without opening a lyx file,
>> at the same place like other todo issues (without dedicated test file)
>> -1 mix of working and failing files in the sample directory
>> c) put in a new directory and move once it works
>> -1 problem description canot be seen without opening the lyx sample
>> +1 no mix of working and failing files in the sample directory
> We could combine b) + c)
> The mix of working and failing files would be still there, but clearly
> separated.
>> +1 problem description can be seen without opening a lyx file,
>> at the same place like other todo issues (without dedicated test file)
>> +1 no mix of working and failing files in the sample directory
-1 after solving the issue, there are two actions to take: move the
file, remove the "inversion pattern".
I don't see the advantage over just b) but could live with this.
Finding failing/working samples would be simple with e.g.
ctest -N -L inverted -R /samples/
ctest -N -LE inverted -R /samples/
>> We should also consider that there are already "autotest" files scattered
>> at several places:
>> development/autotests/
> Historical reasons. I confess that moving the export cmake files from
> there makes sense. This is not really difficult, I already separated
> the tasks for keytests and export tests.
Are the keytests automatic tests? If yes, they would be suitable placed
in a subdir under the "autotests" dir.
.../autotests/keytests
.../autotests/export
...
>> autotests/
>> Testing/ # Logs written to Testing/Temporary/
> This is in the build-dir, no way to change the path.
>> Suggestions:
>> * collect the autotest files under one of
>> Testing/autotests/ or
>> development/autotests/
> There are some mails from Georg explaining why the /autotests dir is
> created.
If I recall correctly this was when looking for the best name of the
directory, reasons were
* this is how it is called at KDE
* we want separate dirs for automatic tests and interactive tests
Both do not speak against Testing/autotests/
>> * rename (Testing|development)/autotests/export/ to
>> (Testing|development)/autotests/samples/
>> * eventually add .../autotests/failing-samples/
>> or .../autotests/samples/failing/
> +1
>> Add a README in the samples/ and failing(-samples)? directory, explaining
>> for all to place working samples here and failing samples there.
>> ¹ mind, that the "longest labeling label" problem is not a *regression* -
>> is is there from the beginning of LyX (well actually only since the
>> support for "longest label").
> Once it is corrected it will be part of regressions.
Only if something works and then fails (again), it is a regression.
So, once it is corrected it will be no regression.
Once it is corrected it may become a regressions - if it fails later on.
With plan c), this would mean we have to move the sample file again:
for the time of failure to failing-samples, after solving the regression
back to samples :-(
Günter
