Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-17 Thread Norbert Thiebaud
On Wed, Jun 17, 2015 at 6:17 AM, Bjoern Michaelsen wrote: > Hi, > > On Wed, Jun 10, 2015 at 03:04:41PM +0200, Bjoern Michaelsen wrote: >> Having master-tested might help us a lot in having more of the first and less >> of the second. > > I learned today that I could have spared myself the work of

Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-17 Thread Bjoern Michaelsen
Hi, On Wed, Jun 10, 2015 at 03:04:41PM +0200, Bjoern Michaelsen wrote: > Having master-tested might help us a lot in having more of the first and less > of the second. I learned today that I could have spared myself the work of describing this, as git comes with a best practices advice to do pret

Re: test-infra proposal: master-tested branch

2015-06-11 Thread Michael Stahl
On 10.06.2015 23:36, Bjoern Michaelsen wrote: > On Wed, Jun 10, 2015 at 10:02:12PM +0100, Wols Lists wrote: >> Would it make sense to have a server dedicated to Heisenbugs? >> If a test triggers a heisenbug, disable it on most of them >> ... but try to >> instrument the heisenbug tester up the waz

Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-11 Thread Thorsten Behrens
Bjoern Michaelsen wrote: > However, there are _some_ things that can be improved and can be > done so with infra/money. While we cant have machines write good > tests, we can have machines _run_ the tests we have regularly in the > first place. Looking at the time-broken number from our tinderboxes

Re: test-infra proposal: master-tested branch

2015-06-10 Thread Bjoern Michaelsen
Hi, On Wed, Jun 10, 2015 at 10:02:12PM +0100, Wols Lists wrote: > Would it make sense to have a server dedicated to Heisenbugs? > If a test triggers a heisenbug, disable it on most of them That makes sense -- thus the suggestion to have 'master-tested' run e.g. make check multiple -- say 10 time

Re: test-infra proposal: master-tested branch

2015-06-10 Thread Wols Lists
On 10/06/15 20:22, Norbert Thiebaud wrote: > - a test auto-induced one: when a test is unstable and produce random > failures based on circumstances... the infamous 'heisenbugs' > and heisenbug can be a systemic/design problem or can be a real bug > that is hard to trigger. either way these are not

Re: test-infra proposal: master-tested branch (was: test infrastructure ideas appreciated ...)

2015-06-10 Thread Norbert Thiebaud
On Wed, Jun 10, 2015 at 8:04 AM, Bjoern Michaelsen wrote: > Hi, > As such, here is one idea for infrastructure: > - Create a branch master-tested We can get all that merely with git-notes I think. iow instead of a separate brnch, just annotate or even maiybe maintain a tag on master to indicate t