Re: categorizing export tests

2015-11-26 Thread Kornel Benko
Am Donnerstag, 26. November 2015 um 22:18:18, schrieb Guenter Milde > On 2015-11-26, Kornel Benko wrote: > > Am 26. November 2015 um 11:23:46, schrieb Guenter Milde > > >> The following proposal for an export test case categorisation tries to > >> avoid the controversial terms "inverted/reverte

Re: categorizing export tests

2015-11-26 Thread Guenter Milde
On 2015-11-26, Kornel Benko wrote: > Am 26. November 2015 um 11:23:46, schrieb Guenter Milde >> The following proposal for an export test case categorisation tries to >> avoid the controversial terms "inverted/reverted", "suspended", and >> "ignored". >> Instead, the basic distinction is between

Re: categorizing export tests

2015-11-26 Thread Kornel Benko
Am Donnerstag, 26. November 2015 um 11:23:46, schrieb Guenter Milde > On 2015-11-23, Kornel Benko wrote: > > Am 23. November 2015 um 16:38:20, schrieb Guenter Milde > > ... > > > Maybe you could propose different names? > > The following proposal for an export test case categorisation tries t

Re: categorizing export tests

2015-11-26 Thread Guenter Milde
On 2015-11-23, Kornel Benko wrote: > Am 23. November 2015 um 16:38:20, schrieb Guenter Milde ... > Maybe you could propose different names? The following proposal for an export test case categorisation tries to avoid the controversial terms "inverted/reverted", "suspended", and "ignored". Inst

Re: categorizing export tests

2015-11-23 Thread Kornel Benko
Am Montag, 23. November 2015 um 16:38:20, schrieb Guenter Milde ... > >> Up to here, I hope we can agree. > > > Partly. I made the comments where I think it should be more precise. > > Which of the follwing is true? > > a) whether a combination of document, output format, and scripted change

Re: categorizing export tests

2015-11-23 Thread Guenter Milde
On 2015-11-23, Kornel Benko wrote: > Am Montag, 23. November 2015 um 13:09:40, schrieb Guenter Milde > >> On 2015-11-22, Kornel Benko wrote: >> > We apparently don't understand each other. >> Indeed, there is a misunderstanding, but I believe to be a bit wiser now. >> Please correct me where I

Re: categorizing export tests

2015-11-23 Thread Kornel Benko
Am Montag, 23. November 2015 um 13:09:40, schrieb Guenter Milde > Dear Kornel and Scott (and everyone else interested in export tests), > > On 2015-11-22, Kornel Benko wrote: > > > > We apparently don't understand each other. > > Indeed, there is a misunderstanding, but I believe to be a bit

Re: categorizing export tests

2015-11-23 Thread Guenter Milde
Dear Kornel and Scott (and everyone else interested in export tests), On 2015-11-22, Kornel Benko wrote: > We apparently don't understand each other. Indeed, there is a misunderstanding, but I believe to be a bit wiser now. Please correct me where I am still wrong. > For me, ignored test is

Re: categorizing export tests (was: export test failures 2)

2015-11-21 Thread Kornel Benko
Am Samstag, 21. November 2015 um 18:54:51, schrieb Scott Kostyshak > On Sat, Nov 21, 2015 at 11:38:54PM +, Guenter Milde wrote: > > On 2015-11-20, Kornel Benko wrote: > > > > To make it clear: Everything ignored cannot be tested. > > > > Then, there is a fundamental flaw in the test machin

Re: categorizing export tests (was: export test failures 2)

2015-11-21 Thread Kornel Benko
Am Samstag, 21. November 2015 um 23:38:54, schrieb Guenter Milde > > If we want to see, if anything > > changed (like XeTeX), we should be able to retest. > > like 'ctest -L suspended' > > We should be able to retest like 'ctest -L ignored' We apparently don't understand each other. For m

Re: categorizing export tests (was: export test failures 2)

2015-11-21 Thread Scott Kostyshak
On Sat, Nov 21, 2015 at 11:38:54PM +, Guenter Milde wrote: > On 2015-11-20, Kornel Benko wrote: > > To make it clear: Everything ignored cannot be tested. > > Then, there is a fundamental flaw in the test machinery: > > We need a category and rule-set for tests where: > > * we don't care f

categorizing export tests (was: export test failures 2)

2015-11-21 Thread Guenter Milde
On 2015-11-20, Kornel Benko wrote: > Am 20. November 2015 um 11:20:54, schrieb Guenter Milde >> My suggestion would be to have the following categories and sub-categories: >> * export # we expect the export to succeed > this is also a candidate for regressions >> * r

categorizing export tests (was: Cmake export tests: More tests to be reverted.)

2015-11-21 Thread Guenter Milde
On 2015-11-20, Kornel Benko wrote: > Am 18. November 2015 um 18:56:59, schrieb Guenter Milde >> Generally, I don't think it is a good idea to "massage" the documents >> shipping with lyx in the test machinery until the tests pass. >> * If the document can be made more robust (i.e. working with m