It's a pretty useful technique especially from CI perspective. I use it via JUnit's assumptions. Failed assumption is shown as an ignored test.
On Sat, May 5, 2012 at 2:29 AM, Chris Hostetter <hossman_luc...@fucit.org>wrote: > > Dawid: > > With the new test runner you created, would it be possible to setup an > annotation that we could use instead to indicate that a test should in fact > be run, and if it fails, include the failure info in the build report, but > do not fail the build? > > I'm thinking in particular about some of the test that multi-threaded > tests that are currently marked @Ignore or @AwaitsFix because they > sporadically fail on jenkins in our jail -- but that people haven't been > able to reproduce consistently on local dev machines (or that some people > have been able to reproduce, but not the people who understand the > tests/code well enough to try and fix) > > As it stands right now, if somone wants to try and fix a complicated test > that's disabled, they have to make a guess at the fix, un-@Ignore, then > watch the next few/several builds patiently to see if / how-often it fails, > then commit the @Ignore back, and repeat. > > If we could leave these tests running on every build, then we could at > least monitor the relative frequency of the failures -- ie: "last week > testFoo failed in 10% of the builds, this week it fails in every build, so > somebody definiteily broke something" or "last week testFoor failed in 10% > of the builds, and after my attempted hardening it only fails in 5% of the > builds so i may be on to something." > > what do folks think? > > -Hoss > > ------------------------------**------------------------------**--------- > To unsubscribe, e-mail: > dev-unsubscribe@lucene.apache.**org<dev-unsubscr...@lucene.apache.org> > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- Obviousnessly Yours Captain <http://www.griddynamics.com> <mkhlud...@griddynamics.com>
