Re: test infrastructure ideas appreciated ... - Help / Doc screenshots ...

2016-11-15 Thread Stephan Bergmann
On 06/05/2015 02:26 PM, Thorsten Behrens wrote: Michael Meeks wrote: * Find some way to automate the creation of (translated) screenshots for help and documentation + this can be used to keep the help up-to-date + and also to test significant dia

Re: Anytime builder VMs for devs (was: test infrastructure ideas appreciated ...)

2015-06-26 Thread Christian Lohmaier
Hi Björn, *, On Thu, Jun 18, 2015 at 11:14 AM, Bjoern Michaelsen wrote: > On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: >> Constructive thoughts appreciated in reply here. > > Budget for 2-5 of our Hackfest VMs[1] to always be on standby for someone > wanting to do a fast b

track and present status of individual tests (was: test infrastructure ideas appreciated ...)

2015-06-18 Thread Bjoern Michaelsen
Hi, On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: > Constructive thoughts appreciated in reply here. This has some synergies with the dashboard proposal[1] but it is sufficiently distict to be included as an ideas on its own: We lack a good representation of test status. t

Re: Anytime builder VMs for devs (was: test infrastructure ideas appreciated ...)

2015-06-18 Thread Norbert Thiebaud
On Thu, Jun 18, 2015 at 4:14 AM, Bjoern Michaelsen wrote: > Hi, > > On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: >> Constructive thoughts appreciated in reply here. > > Budget for 2-5 of our Hackfest VMs[1] to always be on standby for someone > wanting to do a fast build, a

Anytime builder VMs for devs (was: test infrastructure ideas appreciated ...)

2015-06-18 Thread Bjoern Michaelsen
Hi, On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: > Constructive thoughts appreciated in reply here. Budget for 2-5 of our Hackfest VMs[1] to always be on standby for someone wanting to do a fast build, and provision for Cloph to be able to hand out log-ins at a quick ping

Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-17 Thread Norbert Thiebaud
On Wed, Jun 17, 2015 at 6:17 AM, Bjoern Michaelsen wrote: > Hi, > > On Wed, Jun 10, 2015 at 03:04:41PM +0200, Bjoern Michaelsen wrote: >> Having master-tested might help us a lot in having more of the first and less >> of the second. > > I learned today that I could have spared myself the work of

Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-17 Thread Bjoern Michaelsen
Hi, On Wed, Jun 10, 2015 at 03:04:41PM +0200, Bjoern Michaelsen wrote: > Having master-tested might help us a lot in having more of the first and less > of the second. I learned today that I could have spared myself the work of describing this, as git comes with a best practices advice to do pret

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-17 Thread Michael Meeks
On Tue, 2015-06-16 at 23:45 +0200, Zolnai Tamás wrote: > I didn't find any example code in this topic. It was just a sudden > idea deducted from the general rule: If you are bored doing things > manually do it automatically. :) Fair enough =) > I'm not sure what clang plugins capable of.

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-17 Thread Zolnai Tamás
2015-06-16 23:52 GMT+02:00 Zolnai Tamás : > 2015-06-16 9:15 GMT+02:00 Stephan Bergmann : >> On 06/15/2015 03:51 PM, Zolnai Tamás wrote: >>> >>> I've got an idea too: automatically generated unit tests. >>> I think with a clang plugin we can generate unit tests for all >>> class/struct in the source

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-16 Thread Zolnai Tamás
2015-06-16 9:15 GMT+02:00 Stephan Bergmann : > On 06/15/2015 03:51 PM, Zolnai Tamás wrote: >> >> I've got an idea too: automatically generated unit tests. >> I think with a clang plugin we can generate unit tests for all >> class/struct in the source code in build time. Build system can link >> it

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-16 Thread Zolnai Tamás
2015-06-15 21:34 GMT+02:00 Michael Meeks : > Hi Tamas, > > On Mon, 2015-06-15 at 15:51 +0200, Zolnai Tamás wrote: >> I've got an idea too: automatically generated unit tests. > > Sounds interesting :-) are there some existing examples of clang > plugins that do this - or some papers / exist

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-16 Thread Stephan Bergmann
On 06/15/2015 03:51 PM, Zolnai Tamás wrote: I've got an idea too: automatically generated unit tests. I think with a clang plugin we can generate unit tests for all class/struct in the source code in build time. Build system can link it as a simple unit test and run it directly after generation.

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-15 Thread Michael Meeks
Hi Tamas, On Mon, 2015-06-15 at 15:51 +0200, Zolnai Tamás wrote: > I've got an idea too: automatically generated unit tests. Sounds interesting :-) are there some existing examples of clang plugins that do this - or some papers / existing code that does this ? I imagine we would want anno

Re: test infrastructure ideas appreciated ... -- automatically generated unit tests

2015-06-15 Thread Zolnai Tamás
Hi guys, I've got an idea too: automatically generated unit tests. I think with a clang plugin we can generate unit tests for all class/struct in the source code in build time. Build system can link it as a simple unit test and run it directly after generation. If it returns with no error it can b

Re: regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Bjoern Michaelsen
Hi, On Sat, Jun 13, 2015 at 02:07:14PM -0500, Norbert Thiebaud wrote: > if you want to take ownership of that, I'll be more than happy to give > you the needed auth on jenkins and on tb31 I'd love to invest some time into tracking long term trends there, yes. It should be reasonably easy to get s

Re: regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Norbert Thiebaud
On Sat, Jun 13, 2015 at 1:40 PM, Bjoern Michaelsen wrote: > > Whereto? As in: In which repo is the script for the job hosted? there is not tools that I am aware of that do lcov trending... so nowhere. but patch welcome... dev-tools.git or buildbot.git for instance > > Its great that we have it

Re: regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Bjoern Michaelsen
Hi, On Sat, Jun 13, 2015 at 12:10:31PM -0500, Norbert Thiebaud wrote: > > What kind of config does it run? Something "-O0" or "-fno-inline" and thus > > providing useful info, or does it reuse something else. > > Running ./configure with '--enable-debug' '--enable-python=internal' > '--disable-on

Re: regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Norbert Thiebaud
On Sat, Jun 13, 2015 at 10:23 AM, Bjoern Michaelsen wrote: > Hi, > > On Sat, Jun 13, 2015 at 10:05:52AM -0500, Norbert Thiebaud wrote: >> why didn't I think of that.. oh wait > > Hey, this is not about what we think about, but about what we intend to > implement It is not an 'intention'. A volunt

Re: regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Bjoern Michaelsen
Hi, On Sat, Jun 13, 2015 at 10:05:52AM -0500, Norbert Thiebaud wrote: > why didn't I think of that.. oh wait Hey, this is not about what we think about, but about what we intend to implement and document so that it is available for others to use. ;) Care to update https://wiki.documentfoundati

Re: regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Norbert Thiebaud
On Sat, Jun 13, 2015 at 10:05 AM, Norbert Thiebaud wrote: > On Sat, Jun 13, 2015 at 9:46 AM, Bjoern Michaelsen > wrote: >> Hi, >> >> On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: >>> It is noticeable that where there are existing tests & infrastructure, >>> new tests are ea

Re: regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Norbert Thiebaud
On Sat, Jun 13, 2015 at 9:46 AM, Bjoern Michaelsen wrote: > Hi, > > On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: >> It is noticeable that where there are existing tests & infrastructure, >> new tests are easier to create and often there are more of them - so are >> there pl

regular lcov reports (was: test infrastructure ideas appreciated ...)

2015-06-13 Thread Bjoern Michaelsen
Hi, On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: > It is noticeable that where there are existing tests & infrastructure, > new tests are easier to create and often there are more of them - so are > there places where we should work to create infrastructure ? > > Is

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-12 Thread Wols Lists
On 12/06/15 09:41, Jacobo Aragunde Pérez wrote: > El 11/06/15 a las 08:44, David Ostrovsky escribió: >> On Wed, Wed Jun 10 12:22:53 PDT 2015, Norbert Thiebaud wrot >> >>> All that being said, none of that matter if the culture does not >>> follow. no amount of CI can make people care.. what set the

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-12 Thread Bjoern Michaelsen
On Fri, Jun 12, 2015 at 11:34:00AM +0300, Tor Lillqvist wrote: > No need to bring up ridiculous things like that, when even for Linux, there > are hundreds of non-ridiculous slightly different configurations. Thats irrelevant as the goal for now is to have the Linux CI-builder to succeed. Thats on

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-12 Thread Jacobo Aragunde Pérez
El 11/06/15 a las 08:44, David Ostrovsky escribió: > On Wed, Wed Jun 10 12:22:53 PDT 2015, Norbert Thiebaud wrot > >> All that being said, none of that matter if the culture does not >> follow. no amount of CI can make people care.. what set the tone is >> the core developer group, the rest of us

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-12 Thread Tor Lillqvist
> If someone wants master to be always green on AmigaOS on VAX, No need to bring up ridiculous things like that, when even for Linux, there are hundreds of non-ridiculous slightly different configurations. Just recall what Stephan mentioned in this or some other thread some day ago about wanting

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-12 Thread Bjoern Michaelsen
Hi, On Fri, Jun 12, 2015 at 03:06:19AM -0500, Norbert Thiebaud wrote: > Gee, that is so easy.. why didn't I think of that... Indeed its not. But thats beside the point. Bjoern wrote: > > Right now, the important and achievable goal is to get master to be always > > green for our CI-builders.

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-12 Thread Norbert Thiebaud
On Fri, Jun 12, 2015 at 2:31 AM, Bjoern Michaelsen wrote: > If it is very important to someone to support some uncommon build scenario, > there is a way to get that: Add a CI-enabled tinderbox. Gee, that is so easy.. why didn't I think of that... Seriously: NO. people can setup all the tinderbo

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-12 Thread Bjoern Michaelsen
Hi, On Fri, Jun 12, 2015 at 09:58:34AM +0300, Tor Lillqvist wrote: > How can master always work, if, as you said yourself, not all developers > can test all configs, what makes you think there WOULD magically exist > tinderbox slaves for all configs then? Right. Its unavoidable that there might b

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Tor Lillqvist
> Because not all developers can test all configs, we then have the branch > "bleeding" which is explicitly "works for me, does it break someone else?". > > That way, master always works, which is what novice (and I guess most > experienced, too) devs want AND EXPECT, but we have a branch dedicated

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread julien2412
David Ostrovsky-3 wrote > ... > I expect that master is always green. My definition of green is: > > $ make check > > with --enable-werror is passing on all three platforms: Linux|Mac|Win > 64. Hello, Master seems to me the ref code for: - debugging,(to know if a bugtracker shows an existing

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Wols Lists
On 11/06/15 15:23, Bjoern Michaelsen wrote: > On Thu, Jun 11, 2015 at 05:09:08PM +0300, Tor Lillqvist wrote: >>> So what happens if I write a patch that works fine on linux, so I apply >>> it to master, and the Windows build promptly blows up ... >> >> Instead of "master", read "bleeding". Does tha

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Bjoern Michaelsen
On Thu, Jun 11, 2015 at 10:28:52AM -0500, Norbert Thiebaud wrote: > > http://www.documentfoundation.org/grant-request/ > > ? Michaels original mail said: > We hope to turn this into a proposal for the Board, which if approved -could- > turn into a tender to fund more work in this area. So, if t

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Norbert Thiebaud
On Thu, Jun 11, 2015 at 6:45 AM, Bjoern Michaelsen wrote: > > As we seem to be agreeing on the technical part, anyone caring about this is > invited to boil down the technical (and non-controversial) part of this into a > concrete proposal for TDF to consider: > > http://www.documentfoundation.or

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Bjoern Michaelsen
On Thu, Jun 11, 2015 at 05:09:08PM +0300, Tor Lillqvist wrote: > > So what happens if I write a patch that works fine on linux, so I apply > > it to master, and the Windows build promptly blows up ... > > Instead of "master", read "bleeding". Does that make it OK? Not if we care about testing and

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Tor Lillqvist
> > So what happens if I write a patch that works fine on linux, so I apply > it to master, and the Windows build promptly blows up ... > Instead of "master", read "bleeding". Does that make it OK? --tml ___ LibreOffice mailing list LibreOffice@lists.fr

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Wols Lists
On 11/06/15 12:19, Tor Lillqvist wrote: > > Let's have a branch called "lo-next", or "bleeding", or something like > that. I don't have access to Mac, and don't build on Win. How hard is it > to push all changes to "bleeding", and then either cherry-pick or bulk > push all changes

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Bjoern Michaelsen
Hi, On Thu, Jun 11, 2015 at 11:14:29AM +0100, Wols Lists wrote: > Let's have a branch called "lo-next", or "bleeding", or something like > that. I don't have access to Mac, and don't build on Win. How hard is it > to push all changes to "bleeding", and then either cherry-pick or bulk > push all ch

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Tor Lillqvist
> Let's have a branch called "lo-next", or "bleeding", or something like > that. I don't have access to Mac, and don't build on Win. How hard is it > to push all changes to "bleeding", and then either cherry-pick or bulk > push all changes to master when they pass on all relevant test boxes. > I a

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Tor Lillqvist
> let me guess... the core developer was a certain Finnish guy with a > certain reputation for sarcastic remarks? > That was my suspicion, too, but I could not find the quoted text in my IRC logs... --tml ___ LibreOffice mailing list LibreOffice@lists.f

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Michael Stahl
On 11.06.2015 08:44, David Ostrovsky wrote: > Nothing causes more pain, frustration and disappointment than > unfulfilled expectations. > > I expect that master is always green. My definition of green is: > > $ make check > > with --enable-werror is passing on all three platforms: Linux|Mac|Wi

Re: Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-11 Thread Wols Lists
On 11/06/15 07:44, David Ostrovsky wrote: > On Wed, Wed Jun 10 12:22:53 PDT 2015, Norbert Thiebaud wrot > >> All that being said, none of that matter if the culture does not >> follow. no amount of CI can make people care.. what set the tone is >> the core developer group, the rest of us looks aro

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-11 Thread Thorsten Behrens
Kohei Yoshida wrote: > FWIW, I've found officetron to be much pickier and generally more useful > than the official SDK from Microsoft when trying to figure out why a > certain OOXML file generated from LibreOffice wouldn't open correctly in > MS Office. At least that was the case when I was debug

Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-11 Thread Thorsten Behrens
Bjoern Michaelsen wrote: > However, there are _some_ things that can be improved and can be > done so with infra/money. While we cant have machines write good > tests, we can have machines _run_ the tests we have regularly in the > first place. Looking at the time-broken number from our tinderboxes

Changing mindset of core LO developers to the status of master -- was test infrastructure ideas appreciated ...

2015-06-10 Thread David Ostrovsky
On Wed, Wed Jun 10 12:22:53 PDT 2015, Norbert Thiebaud wrot > All that being said, none of that matter if the culture does not > follow. no amount of CI can make people care.. what set the tone is > the core developer group, the rest of us looks around how it is done > and emulate the behavior. N

Re: why a master-tested branch is important and incentives for CI over no-CI (was: test infrastructure ideas appreciated ...)

2015-06-10 Thread Bjoern Michaelsen
Hi, On Wed, Jun 10, 2015 at 06:01:54PM -0500, Norbert Thiebaud wrote: > No today you cannot, as there is no guarantee that the intersection of > the set of the commit built by each tinderbox is non-empty. > In fact the odds that that set is non empty during a given work-day is > pretty low. Yes,

Re: why a master-tested branch is important and incentives for CI over no-CI (was: test infrastructure ideas appreciated ...)

2015-06-10 Thread Norbert Thiebaud
On Wed, Jun 10, 2015 at 3:40 PM, Bjoern Michaelsen wrote: > > Of course, you can collect the data of last-known-good manually No today you cannot, as there is no guarantee that the intersection of the set of the commit built by each tinderbox is non-empty. In fact the odds that that set is non em

Re: why a master-tested branch is important and incentives for CI over no-CI (was: test infrastructure ideas appreciated ...)

2015-06-10 Thread Bjoern Michaelsen
On Wed, Jun 10, 2015 at 10:40:29PM +0200, Bjoern Michaelsen wrote: > Having an easy-to-pull and CI-verified-only pull branch called > 'master-tested' should help setting these. Eh, easy-to-pull and CI-verified-only push branch of-course. Best, Bjoern _

why a master-tested branch is important and incentives for CI over no-CI (was: test infrastructure ideas appreciated ...)

2015-06-10 Thread Bjoern Michaelsen
Hi, On Wed, Jun 10, 2015 at 02:22:53PM -0500, Norbert Thiebaud wrote: > We can get all that merely with git-notes I think. > iow instead of a separate brnch, just annotate or even maiybe maintain > a tag on master to indicate the last 'green master' in the sens you gave. Yes, and I consider tinde

Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-10 Thread Norbert Thiebaud
On Wed, Jun 10, 2015 at 8:04 AM, Bjoern Michaelsen wrote: > Hi, > As such, here is one idea for infrastructure: > - Create a branch master-tested We can get all that merely with git-notes I think. iow instead of a separate brnch, just annotate or even maiybe maintain a tag on master to indicate t

Re: test infrastructure ideas appreciated ...

2015-06-10 Thread Michael Meeks
On Wed, 2015-06-10 at 09:35 -0400, Kohei Yoshida wrote: > I am happy to see that you and I are on the same term on the majority of the > points. But let me nitpick on a few points below. :-) > > My yard-stick would be that if writing the unit-test takes longer than > > finding & fixi

Re: test infrastructure ideas appreciated ...

2015-06-10 Thread Kohei Yoshida
Hi Michael, I am happy to see that you and I are on the same term on the majority of the points. But let me nitpick on a few points below. > On June 10, 2015 at 6:47 AM Michael MeeksSo - AFAICS -every- bug-fix is potentially unit-testable - it is just a > matter of the cost to implement t

test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-10 Thread Bjoern Michaelsen
Hi, On Wed, Jun 03, 2015 at 02:33:23PM +0100, Michael Meeks wrote: > Constructive thoughts appreciated in reply here. Soo, looking at tests, regressions and infra for that, lets first have a look at the "regression pipeline". There are four stages a regression goes through: 1/ regression g

Re: test infrastructure ideas appreciated ...

2015-06-10 Thread Michael Meeks
Hi Kohei, On Tue, 2015-06-09 at 21:00 -0400, Kohei Yoshida wrote: > On Wed, 2015-06-03 at 14:33 +0100, Michael Meeks wrote: > > Having said that writing unit tests is every > > developers' responsibility where feasible > > So, this "where feasible" is quite often subject to a wide range of > inte

Re: test infrastructure ideas appreciated ...

2015-06-09 Thread Kohei Yoshida
On Wed, 2015-06-03 at 14:33 +0100, Michael Meeks wrote: > Having said that writing unit tests is every > developers' responsibility where feasible So, this "where feasible" is quite often subject to a wide range of interpretation. For some, writing a unit test is often not "feasible" if a bug fi

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-09 Thread Kohei Yoshida
On Tue, 2015-06-09 at 18:44 +0200, Thorsten Behrens wrote: > Michael Meeks wrote: > > What of this don't we cover with the crashtester validation ? > > > I guess, as I said, but: > > > On Fri, 2015-06-05 at 18:59 +0200, Thorsten Behrens wrote: > > > Same applies to OOXML, having one box runni

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-09 Thread Thorsten Behrens
Michael Meeks wrote: > What of this don't we cover with the crashtester validation ? > I guess, as I said, but: > On Fri, 2015-06-05 at 18:59 +0200, Thorsten Behrens wrote: > > Same applies to OOXML, having one box running the Open XML SDK there & > > its validator would be awesome > > > (h

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-09 Thread Michael Meeks
On Fri, 2015-06-05 at 14:46 +0200, Thorsten Behrens wrote: > > So I have never been able to get it stable across more than my > > machines and gandalf. I suppose you can increase the number of > > systems that support it if you really know what you are > > doing. > > With the above - pretty much r

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-09 Thread Michael Meeks
On Fri, 2015-06-05 at 18:59 +0200, Thorsten Behrens wrote: > another angle to tests - the various file format compliance checkers. > For example ODF: ... > Same applies to OOXML, having one box running the Open XML SDK there & > its validator would be awesome > (http://openxmldeveloper.org/blog/b/

Re: test infrastructure ideas appreciated ...

2015-06-08 Thread Stephan Bergmann
On 06/04/2015 09:12 PM, Robert Antoni Buj i Gelonch wrote: $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/check-depends.pl $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/lib/tools.pm $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtim

Re: test infrastructure ideas appreciated ... - format validity checks

2015-06-05 Thread Thorsten Behrens
Hi, another angle to tests - the various file format compliance checkers. For example ODF: - have an up-to-date rng file in our tree, that needs to have extensions added the moment we start writing that out - make --with-export-validation pickup that file instead - have a host (e.g. the crasht

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-05 Thread Thorsten Behrens
Markus Mohrhard wrote: > So I have never been able to get it stable across more than my > machines and gandalf. I suppose you can increase the number of > systems that support it if you really know what you are > doing. > With the above - pretty much rules it out for me at this stage. I have not so

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-05 Thread Markus Mohrhard
On Fri, Jun 5, 2015 at 2:21 PM, Thorsten Behrens wrote: > Michael Meeks wrote: > > I'm no expert on the current state here, but my feeling is that > there > > are a lot of document / chart layout issues that are a real problem to > > regression test. > > > Curious to hear Moggi's input on t

Re: test infrastructure ideas appreciated ... - Help / Doc screenshots ...

2015-06-05 Thread Thorsten Behrens
Michael Meeks wrote: > * Find some way to automate the creation of (translated) > screenshots for help and documentation > + this can be used to keep the help up-to-date > + and also to test significant dialog open/close with it > + and to add

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-05 Thread Thorsten Behrens
Michael Meeks wrote: > I'm no expert on the current state here, but my feeling is that there > are a lot of document / chart layout issues that are a real problem to > regression test. > Curious to hear Moggi's input on that one - what I recall, was that Chart layout is just too randomly uns

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Robert Antoni Buj i Gelonch
$ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/check-depends.pl $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/lib/tools.pm $ curl -OL https://llvm.org/svn/llvm-project/openmp/trunk/runtime/tools/lib/Platform.pm $ curl -OL https://llvm.org/svn/llvm-

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Michael Stahl
On 04.06.2015 18:32, Michael Meeks wrote: > Hi Robert, > > On Wed, 2015-06-03 at 15:44 +0200, Robert Antoni Buj i Gelonch wrote: >> We could add an automatic validation test for checking the discovery >> of dynamic library dependencies on OS X & Linux. >> * OS X: otool -L file >> * Lin

Re: test infrastructure ideas appreciated ... - a11y based testing

2015-06-04 Thread Matthew J. Francis
On 05/06/2015 00:39, Michael Meeks wrote: On Thu, 2015-06-04 at 16:14 +0200, Noel Grandin wrote: mjayfrancis(IRC) is doing some interesting automated-UI testing work using the UNO accessibility API - perhaps get him on contract to finish it up and make it nice? Ah - it'd be great to

Re: test infrastructure ideas appreciated ... - layout tests ...

2015-06-04 Thread Michael Meeks
I'm no expert on the current state here, but my feeling is that there are a lot of document / chart layout issues that are a real problem to regression test. I imagine that funding an investigation of the best of breed of whatever we have, plus some combination of custom fonts, bu

Re: test infrastructure ideas appreciated ... - Help / Doc screenshots ...

2015-06-04 Thread Michael Meeks
This was mostly a Norbert idea from the ESC call :-) * Find some way to automate the creation of (translated) screenshots for help and documentation + this can be used to keep the help up-to-date + and also to test significant dialog open/close wit

Re: test infrastructure ideas appreciated ... - junit ...

2015-06-04 Thread Michael Meeks
On Thu, 2015-06-04 at 16:14 +0200, Noel Grandin wrote: > Perhaps a contract to convert the Java unit tests to C++? Might be interesting - I wonder if that can be partially automated in some way ;-> At a minimum for JUnit it'd be good to further accelerate the tests by processing

Re: test infrastructure ideas appreciated ... - a11y based testing

2015-06-04 Thread Michael Meeks
On Thu, 2015-06-04 at 16:14 +0200, Noel Grandin wrote: > mjayfrancis(IRC) is doing some interesting automated-UI testing work using > the UNO accessibility API - perhaps get him on contract to finish it up and > make it nice? Ah - it'd be great to use our UNO API directly, rather than vi

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Michael Meeks
Hi Robert, On Wed, 2015-06-03 at 15:44 +0200, Robert Antoni Buj i Gelonch wrote: > We could add an automatic validation test for checking the discovery > of dynamic library dependencies on OS X & Linux. > * OS X: otool -L file > * Linux: ldd file Sounds interesting - the conce

Re: test infrastructure ideas appreciated ...

2015-06-04 Thread Noel Grandin
On 2015-06-03 03:33 PM, Michael Meeks wrote: The ESC are interested in improving unit (and other automated) testing and are interested in concrete ideas for implementing new automated tests to prevent regressions. Hi mjayfrancis(IRC) is doing some interesting automated-UI testing wo

Re: test infrastructure ideas appreciated ...

2015-06-03 Thread Terrence Enger
Thanks to Michael Meeks for asking for suggestions for test infrastructure. I suggest a bunch of virtual partitions provisioned with all the world's freely available database engines. This would enpower interested testers to work on Base bugs without having to have the database engine referenced

Re: test infrastructure ideas appreciated ...

2015-06-03 Thread Robert Antoni Buj i Gelonch
We could add an automatic validation test for checking the discovery of dynamic library dependencies on OS X & Linux. - OS X: otool -L file - Linux: ldd file 2015-06-03 15:33 GMT+02:00 Michael Meeks : > Hi guys, > > The ESC are interested in improving unit (and other automated) >

test infrastructure ideas appreciated ...

2015-06-03 Thread Michael Meeks
Hi guys, The ESC are interested in improving unit (and other automated) testing and are interested in concrete ideas for implementing new automated tests to prevent regressions. We hope to turn this into a proposal for the Board, which if approved -could- turn into a tender to fund more wo