On 2/28/12 6:55 AM, "Michael A. Labriola" <labri...@digitalprimates.net>
wrote:
>> Correct, but as I understand them, Mustella tests are pretty specific to
>> components (e.g., display objects). Does the rest of the framework have some
>> sort of test coverage, or am I misunderstanding Mustella?
>
> You can think of mustella tests as executing a series of actions or steps
> against an object and having the ability to wait for event to do property or
> even image comparison. So far as I know, the only 'test coverage' numbers
> that Adobe used to define their test coverage came from running the mustella
> suite... so any other APIs were exercised by the use of mustella component
> interactions. Further, and maybe Alex can comment on this as I do not know for
> sure, I believe the test coverage information quoted by Deepa and others used
> the coverage plugin on labs they built for Flash Builder. This used the
> profiler API and was effectively fairly useless method coverage.
>
> For example:
>
> function doThing( val:* ):void {
>
> if (!val) {
> return;
> }
>
> //5000 more lines in this method
>
> }
>
> Therefore if this method was called with a null, the whole method and all the
> code that did not belong in it was considered 'covered'
>
> Mike
>
>
Mustella's goal was to test every line of code in the framework. There are
very few public methods on our classes, especially the display classes, so
we would fake events to get into the entry points. There is a mustella test
step that will call a method though. Non-display classes like validators
and formatters and resource manager have tests, but many times those tests
are more like integration tests where the formatter or validator is wired up
to something, events are faked, and results are verified.
My understanding (needs verification) is that we used a tool that matched
the output of flash.trace.Trace to the debugline opcodes in a SWF and that
is the code coverage number. Unless I'm missing something, that is
sufficient, and the example above would not show 100% coverage.
For sure, there aren't enough mustella tests otherwise there would be far
fewer bugs in the shipping framework, and few of these tests are unit-tests
under the strict definition, but they do a pretty good job of catching
regressions.
I am behind schedule prepping Mustella for donation. I'm shooting for
mid-March.
--
Alex Harui
Flex SDK Team
Adobe Systems, Inc.
http://blogs.adobe.com/aharui
