----- Original Message ----- > From: "Steve Ebersole" <st...@hibernate.org> > To: "Gail Badner" <gbad...@redhat.com> > Cc: "Hardy Ferentschik" <ha...@hibernate.org>, "Hibernate hibernate-dev" > <hibernate-dev@lists.jboss.org> > Sent: Monday, August 6, 2012 7:02:54 AM > Subject: Re: [hibernate-dev] Unit test failures using the new metamodel > > > On 08/02/2012 07:39 PM, Gail Badner wrote: > > > > IIRC, there were some tests that were annotated with > > @FailureExpected that were only expected to fail using some > > dialects. This was causing what looked like failures on the > > dialects where the test was expected to pass. When we were trying > > to clean up the test failures for the tested dialects, either > > hibernate.test.validatefailureexpected was introduced or the > > default for hibernate.test.validatefailureexpected was changed to > > false. Steve might remember exactly why this was done. > > > > In any case, this is hindering more than helping when we're trying > > to get things working with the new metamodel. > > Like I said in my initial response that flag has always been there. > The > reason had more to do with different audiences running the tests IIRC > . > But like I said, it was years ago, so the exact specifics escape me. > I'll search the dev list when i get a few minutes and see what I can > dig up. > > I think you are perhaps getting caught up in making changes for a > transitory set of conditions. We currently have a slew of failures > mostly due to the tests using code they were never designed to test.
I'm not sure I understand what you mean? Do you mean that the FailureExpected code is not intended to expect failures in @Before, @BeforeClass, and @BeforeClassOnce callbacks? > And even worse, that code is known to be incomplete. The way > "failure > expected" is supposed to work is that tests which are marked as > @FailureExpected are supposed to be handled the same as if they were > marked with @Ignore (standard JUnit annotation) *if* > `hibernate.test.validatefailureexpected` is not set. If > `hibernate.test.validatefailureexpected` is set that changes the > logic > up a bit. However, again the problem is that we are leveraging tests > to > test something they were never designed to test. In fact, they are > so > much not testing it that the code (now) under test occurs before the > test really even begins. There is supposed to be some code that > skips > before/after callbacks if all tests methods are being ignored. It > would > seem that code is no longer working (or > @FailureExpectedWithNewMetamodel > was not hooked into that code completely) if the tests are marked for > expected failure but before callbacks still happen. Yes, this should be fixed. > > Am I against changing `hibernate.test.validatefailureexpected` to > default to true *for this branch, for the time being*? No. I > already > said I would be willing to discuss that. But I'd also prefer to > figure > out why what is supposed to be happening in the hibernate-testing > code > is no longer happening. +1 > > > When hibernate.test.validatefailureexpected is false (the default), > > the result from FailureExpected tests is simply ignored. > >> :-( > Not understanding how this is not desirable? If the result is simply ignored, then we don't know if changes would result in a @FailureExpectedWithNewMetamodel test to actually pass. With the validation flag set to true, a failure due to FailureExpectedTestPassedException would (hopefully) mean that the @FailureExpectedWithNewMetamodel annotation can be removed. Once it's removed, we have a way to see if a future change breaks it. > > > > > Is there a way to shift the failure so it happens during test > > execution so it would be handled properly? > > Sure. You would not use @Before, @BeforeClass, @BeforeClassOnce > callbacks. Instead you would build the metamodel during the test > method > execution. But again, see above about tests testing code they were > not > designed to test... > I agree that this is not the way to fix this. Thanks, Gail > > -- > st...@hibernate.org > http://hibernate.org > > _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
