Re: test infrastructure ideas appreciated ... - Help / Doc screenshots ...

2016-11-15 Thread Stephan Bergmann
On 06/05/2015 02:26 PM, Thorsten Behrens wrote: Michael Meeks wrote: * Find some way to automate the creation of (translated) screenshots for help and documentation + this can be used to keep the help up-to-date + and also to test significant dia

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-17 Thread Michael Meeks
On Tue, 2015-06-16 at 23:45 +0200, Zolnai Tamás wrote: > I didn't find any example code in this topic. It was just a sudden > idea deducted from the general rule: If you are bored doing things > manually do it automatically. :) Fair enough =) > I'm not sure what clang plugins capable of.

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-17 Thread Zolnai Tamás
2015-06-16 23:52 GMT+02:00 Zolnai Tamás : > 2015-06-16 9:15 GMT+02:00 Stephan Bergmann : >> On 06/15/2015 03:51 PM, Zolnai Tamás wrote: >>> >>> I've got an idea too: automatically generated unit tests. >>> I think with a clang plugin we can generate unit tests for all >>> class/struct in the source

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-16 Thread Zolnai Tamás
2015-06-16 9:15 GMT+02:00 Stephan Bergmann : > On 06/15/2015 03:51 PM, Zolnai Tamás wrote: >> >> I've got an idea too: automatically generated unit tests. >> I think with a clang plugin we can generate unit tests for all >> class/struct in the source code in build time. Build system can link >> it

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-16 Thread Zolnai Tamás
2015-06-15 21:34 GMT+02:00 Michael Meeks : > Hi Tamas, > > On Mon, 2015-06-15 at 15:51 +0200, Zolnai Tamás wrote: >> I've got an idea too: automatically generated unit tests. > > Sounds interesting :-) are there some existing examples of clang > plugins that do this - or some papers / exist

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-16 Thread Stephan Bergmann
On 06/15/2015 03:51 PM, Zolnai Tamás wrote: I've got an idea too: automatically generated unit tests. I think with a clang plugin we can generate unit tests for all class/struct in the source code in build time. Build system can link it as a simple unit test and run it directly after generation.

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-15 Thread Michael Meeks
Hi Tamas, On Mon, 2015-06-15 at 15:51 +0200, Zolnai Tamás wrote: > I've got an idea too: automatically generated unit tests. Sounds interesting :-) are there some existing examples of clang plugins that do this - or some papers / existing code that does this ? I imagine we would want anno

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-15 Thread Zolnai Tamás
Hi guys, I've got an idea too: automatically generated unit tests. I think with a clang plugin we can generate unit tests for all class/struct in the source code in build time. Build system can link it as a simple unit test and run it directly after generation. If it returns with no error it can b

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-11 Thread Thorsten Behrens
Kohei Yoshida wrote: > FWIW, I've found officetron to be much pickier and generally more useful > than the official SDK from Microsoft when trying to figure out why a > certain OOXML file generated from LibreOffice wouldn't open correctly in > MS Office. At least that was the case when I was debug

Re: test infrastructure ideas appreciated ...

2015-06-10 Thread Michael Meeks
On Wed, 2015-06-10 at 09:35 -0400, Kohei Yoshida wrote: > I am happy to see that you and I are on the same term on the majority of the > points. But let me nitpick on a few points below. :-) > > My yard-stick would be that if writing the unit-test takes longer than > > finding & fixi

Re: test infrastructure ideas appreciated ...

2015-06-10 Thread Kohei Yoshida
Hi Michael, I am happy to see that you and I are on the same term on the majority of the points. But let me nitpick on a few points below. > On June 10, 2015 at 6:47 AM Michael MeeksSo - AFAICS -every- bug-fix is potentially unit-testable - it is just a > matter of the cost to implement t

Re: test infrastructure ideas appreciated ...

2015-06-10 Thread Michael Meeks
Hi Kohei, On Tue, 2015-06-09 at 21:00 -0400, Kohei Yoshida wrote: > On Wed, 2015-06-03 at 14:33 +0100, Michael Meeks wrote: > > Having said that writing unit tests is every > > developers' responsibility where feasible > > So, this "where feasible" is quite often subject to a wide range of > inte

Re: test infrastructure ideas appreciated ...

2015-06-09 Thread Kohei Yoshida
On Wed, 2015-06-03 at 14:33 +0100, Michael Meeks wrote: > Having said that writing unit tests is every > developers' responsibility where feasible So, this "where feasible" is quite often subject to a wide range of interpretation. For some, writing a unit test is often not "feasible" if a bug fi

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-09 Thread Kohei Yoshida
On Tue, 2015-06-09 at 18:44 +0200, Thorsten Behrens wrote: > Michael Meeks wrote: > > What of this don't we cover with the crashtester validation ? > > > I guess, as I said, but: > > > On Fri, 2015-06-05 at 18:59 +0200, Thorsten Behrens wrote: > > > Same applies to OOXML, having one box runni

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-09 Thread Thorsten Behrens
Michael Meeks wrote: > What of this don't we cover with the crashtester validation ? > I guess, as I said, but: > On Fri, 2015-06-05 at 18:59 +0200, Thorsten Behrens wrote: > > Same applies to OOXML, having one box running the Open XML SDK there & > > its validator would be awesome > > > (h

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-09 Thread Michael Meeks
On Fri, 2015-06-05 at 14:46 +0200, Thorsten Behrens wrote: > > So I have never been able to get it stable across more than my > > machines and gandalf. I suppose you can increase the number of > > systems that support it if you really know what you are > > doing. > > With the above - pretty much r

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-09 Thread Michael Meeks
On Fri, 2015-06-05 at 18:59 +0200, Thorsten Behrens wrote: > another angle to tests - the various file format compliance checkers. > For example ODF: ... > Same applies to OOXML, having one box running the Open XML SDK there & > its validator would be awesome > (http://openxmldeveloper.org/blog/b/

Re: test infrastructure ideas appreciated ...

2015-06-08 Thread Stephan Bergmann
On 06/04/2015 09:12 PM, Robert Antoni Buj i Gelonch wrote: $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/check-depends.pl $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/lib/tools.pm $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtim

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-05 Thread Thorsten Behrens
Hi, another angle to tests - the various file format compliance checkers. For example ODF: - have an up-to-date rng file in our tree, that needs to have extensions added the moment we start writing that out - make --with-export-validation pickup that file instead - have a host (e.g. the crasht

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-05 Thread Thorsten Behrens
Markus Mohrhard wrote: > So I have never been able to get it stable across more than my > machines and gandalf. I suppose you can increase the number of > systems that support it if you really know what you are > doing. > With the above - pretty much rules it out for me at this stage. I have not so

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-05 Thread Markus Mohrhard
On Fri, Jun 5, 2015 at 2:21 PM, Thorsten Behrens wrote: > Michael Meeks wrote: > > I'm no expert on the current state here, but my feeling is that > there > > are a lot of document / chart layout issues that are a real problem to > > regression test. > > > Curious to hear Moggi's input on t

Re: test infrastructure ideas appreciated ... - Help / Doc screenshots ...

2015-06-05 Thread Thorsten Behrens
Michael Meeks wrote: > * Find some way to automate the creation of (translated) > screenshots for help and documentation > + this can be used to keep the help up-to-date > + and also to test significant dialog open/close with it > + and to add

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-05 Thread Thorsten Behrens
Michael Meeks wrote: > I'm no expert on the current state here, but my feeling is that there > are a lot of document / chart layout issues that are a real problem to > regression test. > Curious to hear Moggi's input on that one - what I recall, was that Chart layout is just too randomly uns

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Robert Antoni Buj i Gelonch
$ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/check-depends.pl $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/lib/tools.pm $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/lib/Platform.pm $ curl -OL https://llvm.org/svn/llvm-

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Michael Stahl
On 04.06.2015 18:32, Michael Meeks wrote: > Hi Robert, > > On Wed, 2015-06-03 at 15:44 +0200, Robert Antoni Buj i Gelonch wrote: >> We could add an automatic validation test for checking the discovery >> of dynamic library dependencies on OS X & Linux. >> * OS X: otool -L file >> * Lin

Re: test infrastructure ideas appreciated ... - a11y based testing

2015-06-04 Thread Matthew J. Francis
On 05/06/2015 00:39, Michael Meeks wrote: On Thu, 2015-06-04 at 16:14 +0200, Noel Grandin wrote: mjayfrancis(IRC) is doing some interesting automated-UI testing work using the UNO accessibility API - perhaps get him on contract to finish it up and make it nice? Ah - it'd be great to

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-04 Thread Michael Meeks
I'm no expert on the current state here, but my feeling is that there are a lot of document / chart layout issues that are a real problem to regression test. I imagine that funding an investigation of the best of breed of whatever we have, plus some combination of custom fonts, bu

Re: test infrastructure ideas appreciated ... - Help / Doc screenshots ...

2015-06-04 Thread Michael Meeks
This was mostly a Norbert idea from the ESC call :-) * Find some way to automate the creation of (translated) screenshots for help and documentation + this can be used to keep the help up-to-date + and also to test significant dialog open/close wit

Re: test infrastructure ideas appreciated ... - junit ...

2015-06-04 Thread Michael Meeks
On Thu, 2015-06-04 at 16:14 +0200, Noel Grandin wrote: > Perhaps a contract to convert the Java unit tests to C++? Might be interesting - I wonder if that can be partially automated in some way ;-> At a minimum for JUnit it'd be good to further accelerate the tests by processing

Re: test infrastructure ideas appreciated ... - a11y based testing

2015-06-04 Thread Michael Meeks
On Thu, 2015-06-04 at 16:14 +0200, Noel Grandin wrote: > mjayfrancis(IRC) is doing some interesting automated-UI testing work using > the UNO accessibility API - perhaps get him on contract to finish it up and > make it nice? Ah - it'd be great to use our UNO API directly, rather than vi

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Michael Meeks
Hi Robert, On Wed, 2015-06-03 at 15:44 +0200, Robert Antoni Buj i Gelonch wrote: > We could add an automatic validation test for checking the discovery > of dynamic library dependencies on OS X & Linux. > * OS X: otool -L file > * Linux: ldd file Sounds interesting - the conce

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Noel Grandin
On 2015-06-03 03:33 PM, Michael Meeks wrote: The ESC are interested in improving unit (and other automated) testing and are interested in concrete ideas for implementing new automated tests to prevent regressions. Hi mjayfrancis(IRC) is doing some interesting automated-UI testing wo

Re: test infrastructure ideas appreciated ...

2015-06-03 Thread Terrence Enger
Thanks to Michael Meeks for asking for suggestions for test infrastructure. I suggest a bunch of virtual partitions provisioned with all the world's freely available database engines. This would enpower interested testers to work on Base bugs without having to have the database engine referenced

Re: test infrastructure ideas appreciated ...

2015-06-03 Thread Robert Antoni Buj i Gelonch
We could add an automatic validation test for checking the discovery of dynamic library dependencies on OS X & Linux. - OS X: otool -L file - Linux: ldd file 2015-06-03 15:33 GMT+02:00 Michael Meeks : > Hi guys, > > The ESC are interested in improving unit (and other automated) >