Geir Magnusson Jr wrote:
> Stepan Mishura wrote:
>> Did anybody think of creating a 'global' (i.e. shared by all modules)
>> exclude list or every module will have its own exclude list? Or Harmony
>> tests will always pass and we don't need it at all :-)
>
> That would be the goal :)
>
>> I see at least the following benefits of creating 'global' exclude
>> list: all
>> know issues are kept in one well known place (they don't spread between
>> several private lists)
>
> That's true, but.... I always imagined that people would be working in
> the modules anyway, so there isn't much gain.
Modules should be as self-sufficient as we can make them, so devolving
tests and their exclude lists into a module makes sense to me too. They
can be linked into a global view quite easily (read on...)
> also it is easier to create an exclude list for a
>> target platform, for example, linux.exclude or win.exclude.
>
> Yes, that could be. Interesting idea.
The mechanism that George contributed in HARMONY-57 uses a decorator to
implement an exclusion list on regular JUnit tests. (I happen to know
that George is off enjoying himself for the next few days, so I hope he
doesn't mind me describing it here.)
The exclusion list is implemented as a (declarative) XML file read when
the tests run -- in HARMONY-57 you can see one in
Harmony_Tests/src/test/resources/config/jcltest-excludes.xml
Picking an entry at random from there:
...
<hy:type id="tests.api.java.io.FileTest">
<hy:exclude id="test_Constructor_String_String_112270"
shouldfix="true">
<hy:reason>Undiagnosed failure</hy:reason>
</hy:exclude>
<hy:exclude id="test_Constructor_File_String_112270"
shouldfix="true">
<hy:reason>Undiagnosed failure</hy:reason>
</hy:exclude>
<hy:exclude id="test_deleteOnExit" shouldfix="true">
<hy:reason>Needs investigation...</hy:reason>
</hy:exclude>
</hy:type>
...
You can see how it works -- some tests are excluded because they fail
and should be fixed, others (not shown here) can be excluded because
they don't make sense on a particular platform, VM, etc.
Applying a style sheet makes it easy to read the exclusions list in
glorious technicolor, either in an individual module or as a combined
global view.
Take a look at the incoming contribution and see if it fits your needs.
Regards,
Tim
> However, I did imagine that we'd give the modules a bit of freedom and
> independence for testing - a global exclude list might impact that?
>
> geir
>
>>
>> Thoughts?
>>
>> Thanks,
>> Stepan Mishura
>> Intel Middleware Products Division
>>
>
--
Tim Ellison ([EMAIL PROTECTED])
IBM Java technology centre, UK.
