Thanks Artem! I think this test suite should go a long way towards removing issues with different configuration property permutations.
Is this suite turned on by default? D. On Mon, Feb 29, 2016 at 7:07 AM, Artem Shutak <ashu...@gridgain.com> wrote: > Igniters, > > I'm glad to announce that Configuration Variations test framework has been > merged to master and is ready to use. > > The framework provides an opportunity to write a test-case once and to run > it in many Ignite and Cache configuration variations, for different Ignite > topologies (with and without clients) and execute the same test scenario > from different Ignite nodes (server and client) automatically. > > Another useful feature is "runInAllDataModes" functionality that allows to > run test-case and run it in all supported by framework data modes (like > Serializable, Externalizable, plane, mixed and etc. objects). > > I've added the framework description on "Implementing Tests" [1] page. > Please, feel free to comment. > > As proof-of-concept I've > added IgniteCacheBasicConfigVariationsFullApiTestSuite and add it on TC. > > It does not mean that the work on IGNITE-2521 > <https://issues.apache.org/jira/browse/IGNITE-2521> is completed and we > have coverage for full Ignite API, but I think it is a good improvement for > Ignite's test framework and Igniters can test functionality better and > simpler. > > [1] https://cwiki.apache.org/confluence/display/IGNITE/Implementing+Tests > (see "Configuration variations test framework" section). > > Thanks, > -- Artem -- > > On Thu, Feb 11, 2016 at 12:29 AM, Dmitriy Setrakyan <dsetrak...@apache.org > > > wrote: > > > Agree about separation. I think we need to define some internal > > permutations that keep coming up with bugs. I can start the list here: > > > > 1. Serializable, Externalizable, neither. > > 1. We should run the whole suite 3 times, once for each serialization > > mode. Having 2 or 3 nodes in the cluster here should be enough. > > No need to > > test it on the changing cluster. > > 2. On-heap and off-heap with entry processor and peer-deployment > > 1. on-heap, entry-processor, peer-deployment=true/false > > 2. off-heap, entry-processor, peer-deployment=true/false > > 3. On-heap and off-heap with eviction policies > > 1. eviction policy, memory-mode=on-heap > > 2. eviction policy, memory-mode=off-heap > > 3. eviction policy, memory-mode=off-heap-values > > 4. On-heap and off-heap with continuous queries and > > peer-deployment=true/false > > 5. … > > > > I think we can come up with about 20 most important permutations. > Thoughts? > > > > D. > > > > > > On Wed, Feb 10, 2016 at 9:24 AM, Alexey Goncharuk < > > alexey.goncha...@gmail.com> wrote: > > > > > Dmitriy, the size of the circular buffer can be decreased by setting > > > IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE system property. > > > > > > I was wondering what our next steps will be after implementing the > > > framework. From the initial discussion I thought we want to convert > some > > > older tests to this new framework and run all new tests using this > > > framework. However, from what Artem already has written, this sounds > > > unrealistic because adding even a simple test case will induce hours of > > run > > > time. I think we still need to separate more important and less > important > > > configurations, run important ones on a daily basis, and use all others > > for > > > after-code-freeze runs, for example. > > > > > > Thoughts? > > > > > >
