+1 Sergi idea Create/destroy cache with unique names should be enough to isolated test logic. If failed test case depends which tests were executed (or weren't executed) earlier the it means a bug or non-determenistic test code.
---------- Forwarded message ---------- From: Dmitriy Setrakyan <dsetrak...@apache.org> Date: Sun, Sep 20, 2015 at 1:46 PM Subject: Re: Nodes starts in tests To: dev@ignite.apache.org On Sun, Sep 20, 2015 at 1:34 PM, Branko Čibej <br...@apache.org> wrote: > On 20.09.2015 10:11, Yakov Zhdanov wrote: > > Very cool idea! However this will not be working in 100% cases since > > sometimes we need custom grid configurations, SPIs, etc, but for > > significant number of tests this will work. > > This is actually, IMO, a very bad idea. One of the most important > principles when designing tests is that each test case should be > independent of others. In practice that means that if you run test cases > in random order, their results should always be the same. If a test case > depends on a particular way the grid is started up, then moving the grid > startup outside the test case will introduce dependencies between test > cases that could hide bugs. > > The primary goal of a test suite is not to be efficient and fast but to > be accurate. > Brane, if we test cache behavior (which we do a lot), it is enough to create/destroy caches using the same node in each test, rather than start/stop the node itself. Every test should use its own instance of cache, thus not depending on other tests or test order. I believe that this will speed up test execution in many cases. D. > > -- Brane > > > > 2015-09-20 11:00 GMT+03:00 Sergi Vladykin <sergi.vlady...@gmail.com>: > > > >> Guys, > >> > >> A little observation. We have really many tests which start and stop > nodes > >> for each test method. > >> Example is IgniteCacheAbstractQuerySelfTest, it contains about 30 test > >> methods and 3 subclasses. > >> In beforeTest method it starts nodes in afterTest it stops them, while > >> cache setup is the same. > >> > >> What I'm trying to say is that each test method itself takes > milliseconds > >> to run, but grid start/stop takes* more than 5 seconds* for each test > >> method. So from the standpoint of time we are testing grid start/stop > all > >> the time. It is not surprising that our tests take very long time to > >> finish. > >> > >> Since we already support dynamic cache start/destroy which are much more > >> fast than grid start/stop, > >> I think we should go through a refactoring and use existing nodes > within a > >> suite as much as possible. > >> I believe this can reduce run time of cache related tests like 10 times. > >> > >> Sergi > >> > > -- Sergey Kozlov
