Re: Testing Framework Site Update

2012-02-28 Thread Alex Harui
On 2/28/12 10:44 AM, "Michael A. Labriola" wrote: >> My understanding (needs verification) is that we used a tool that matched the >> output of flash.trace.Trace to the debugline opcodes in a SWF and that is the >> code >coverage number. Unless I'm missing something, that is sufficient, >> a

RE: Testing Framework Site Update

2012-02-28 Thread Michael A. Labriola
>My understanding (needs verification) is that we used a tool that matched the >output of flash.trace.Trace to the debugline opcodes in a SWF and that is the >code >coverage number. Unless I'm missing something, that is sufficient, and >the example above would not show 100% coverage. Thanks A

Re: Testing Framework Site Update

2012-02-28 Thread Alex Harui
On 2/28/12 6:55 AM, "Michael A. Labriola" wrote: >> Correct, but as I understand them, Mustella tests are pretty specific to >> components (e.g., display objects). Does the rest of the framework have some >> sort of test coverage, or am I misunderstanding Mustella? > > You can think of mustel

RE: Testing Framework Site Update

2012-02-28 Thread Michael A. Labriola
>Hey Mike, > >I remember you talking about generating test stubs for all the framework >classes last year on the Spoon list. Did you ever manage to finish this? >And if so, do you think that these stubs are still something that can be of >value? > >cheers, > >Roland Not 100% finished but also no

RE: Testing Framework Site Update

2012-02-28 Thread Michael A. Labriola
>Are those "surgical changes" in your whiteboard space? Sounds extremely >valuable. There predate Apache. The ideas may, in some cases, be valuable, however, we are now operating under different constraints to in many places I was proposing changes only because of those limitations. Example, in

Re: Testing Framework Site Update

2012-02-28 Thread Roland Zwaga
> > The other concern is that we are likely to make more modification to > existing code than to write brand new code for a while, IMO... to that end > we are then trying to write tests against something that is very coupled > and hence fundamentally untestable in units. When spoon first started, I

Re: Testing Framework Site Update

2012-02-28 Thread Scott Delamater
For anyone else picking up this thread, my earlier question is also answered and documented here under "Enable SDK Testability" http://www.spoon.as/core-values/ sD

Re: Testing Framework Site Update

2012-02-28 Thread Scott Delamater
On Tue, Feb 28, 2012 at 8:30 AM, Michael A. Labriola < labri...@digitalprimates.net> wrote: > The other concern is that we are likely to make more modification to > existing code than to write brand new code for a while, IMO... to that end > we are then trying to write tests against something that

RE: Testing Framework Site Update

2012-02-28 Thread Michael A. Labriola
>I agree, full coverage would be ideal. But I share the same concern. >Writing unit tests at this point is mostly "oh shit does this really work" >tests... not really the same as driving the development with them, which is >why I don't see it as a high priority (for myself) to spend time on. Again

Re: Testing Framework Site Update

2012-02-28 Thread Omar Gonzalez
> > I'd love to see full coverage asap, but in practice I don't know how > valuable the tests will be when writing them is decoupled from either using > or learning the unit under test. My worry is that we'd create tests that > gave us good coverage, but little meaning. > > sD > I agree, full cov

Re: Testing Framework Site Update

2012-02-28 Thread Scott Delamater
On Tue, Feb 28, 2012 at 7:58 AM, Omar Gonzalez wrote: > No. It does not. That is why one of the biggest outstanding tasks is to > get unit tests written for the Flex SDK. And a big reason why that hasn't > been tackled is the highly coupled nature of the framework which will take > quite a bit of

Re: Testing Framework Site Update

2012-02-28 Thread Omar Gonzalez
> > > * For existing code, write tests to verify changes > * For new code, target 100% coverage > > sD > > That's exactly the approach I'm taking with code in my whiteboard area: http://svn.apache.org/viewvc/incubator/flex/whiteboard/s9tpepper/validators/ . Of course, that's not to say that if som

Re: Testing Framework Site Update

2012-02-28 Thread Scott Delamater
Thanks Mike. I saw heaps of discussion around Mustella and regarding the selection of a test framework, but nothing specifically about the (non)existence of unit tests for the framework. I'm more than a little frightened that we don't have test coverage. Without seeing the tests, I'm guessing that

Re: Testing Framework Site Update

2012-02-28 Thread Omar Gonzalez
On Tue, Feb 28, 2012 at 6:50 AM, Scott Delamater wrote: > Correct, but as I understand them, Mustella tests are pretty specific to > components (e.g., display objects). Does the rest of the framework have > some sort of test coverage, or am I misunderstanding Mustella? > > sD > > No. It does not.

RE: Testing Framework Site Update

2012-02-28 Thread Michael A. Labriola
>Correct, but as I understand them, Mustella tests are pretty specific to >components (e.g., display objects). Does the rest of the framework have some >sort of test coverage, or am I misunderstanding Mustella? You can think of mustella tests as executing a series of actions or steps against an

Re: Testing Framework Site Update

2012-02-28 Thread Scott Delamater
Correct, but as I understand them, Mustella tests are pretty specific to components (e.g., display objects). Does the rest of the framework have some sort of test coverage, or am I misunderstanding Mustella? sD On Tue, Feb 28, 2012 at 7:44 AM, Martin Heidegger wrote: > We are waiting for Mustell

RE: Testing Framework Site Update

2012-02-28 Thread Michael A. Labriola
>I assume I'm missing something, but I don't see *any* unit tests in the repo. >Are we still waiting on those? Look back through the list, there was a lot of discussion on this. There are no unit tests for the Flex framework. There are mustella tests, which is proprietary testing framework used

Re: Testing Framework Site Update

2012-02-28 Thread Martin Heidegger
We are waiting for Mustella tests by adobe. I will start working on FlexUnit tests next time I see fit. yours Martin. On 28/02/2012 23:43, Scott Delamater wrote: On Fri, Feb 17, 2012 at 12:25 PM, Alex Harui wrote: Yes the Marshall Plan is not about units, but I assume there is some vision f

Re: Testing Framework Site Update

2012-02-28 Thread Scott Delamater
On Fri, Feb 17, 2012 at 12:25 PM, Alex Harui wrote: > > Yes the Marshall Plan is not about units, but I assume there is some vision > for how to do "integration testing" or "functional testing". I thought > FlexUnit had the ability to do more than just "unit tests". Or do you think > we can ship

Re: Testing Framework Site Update

2012-02-17 Thread Alex Harui
Mustella has a bitmap compare feature. We get anti-aliasing differences on different platforms. On 2/17/12 11:35 AM, "David Francis Buhler" wrote: > I'm curious to know the differences Mustella measures when comparing > test-results for the Mac OS vs. Windows OS. How much do the SDK classes >

RE: Testing Framework Site Update

2012-02-17 Thread Michael A. Labriola
>Yes the Marshall Plan is not about units, but I assume there is some vision >for how to do "integration testing" or "functional testing". I thought >FlexUnit had the ability to do more than just "unit tests". Or do you think we >can ship >a valid release just by unit tests? Alex, I don't thi

Re: Testing Framework Site Update

2012-02-17 Thread David Francis Buhler
I'm curious to know the differences Mustella measures when comparing test-results for the Mac OS vs. Windows OS. How much do the SDK classes vary in their behavior on different Operating Systems? On Fri, Feb 17, 2012 at 2:25 PM, Alex Harui wrote: > > > > On 2/17/12 9:59 AM, "Michael A. Labriola"

Re: Testing Framework Site Update

2012-02-17 Thread Alex Harui
On 2/17/12 9:59 AM, "Michael A. Labriola" wrote: > > FlexUnit won't be able to support that as nothing about the Marshall Plan is a > unit. It is running a whole additional application in a whole additional > domain. It is sort of the antithesis of what FlexUnit is trying to accomplish. > Y

RE: Testing Framework Site Update

2012-02-17 Thread Michael A. Labriola
>At one point, there was a discussion with the FlexUnit folks of taking some >aspects of Mustella and enhancing FlexUnit to support things like Marshall >Plan. It might be worth looking into that again. FlexUnit won't be able to support that as nothing about the Marshall Plan is a unit. It is

Re: Testing Framework Site Update

2012-02-17 Thread Omar Gonzalez
On Thursday, February 16, 2012, Alex Harui wrote: > > > > On 2/16/12 6:02 PM, "Omar Gonzalez" > > wrote: > > > We use [FlexUnit][4] for unit testing and [Mockolate][5] for mocks and > test > > spies. > Are we actually using or planning to use? > > I have not researched any of these test frameworks

RE: Testing Framework Site Update

2012-02-17 Thread Michael A. Labriola
>@Michael Labriola Is there any progress? Open Questions? Heard back from the mentor yesterday. There will be many open questions, still working through the requirements side of the IP clearance Mike

Re: Testing Framework Site Update

2012-02-16 Thread Omar Gonzalez
On Thursday, February 16, 2012, Alex Harui wrote: > > > > On 2/16/12 6:02 PM, "Omar Gonzalez" > > wrote: > > > We use [FlexUnit][4] for unit testing and [Mockolate][5] for mocks and > test > > spies. > Are we actually using or planning to use? > > I have not researched any of these test frameworks

Re: Testing Framework Site Update

2012-02-16 Thread Martin Heidegger
On 17/02/2012 15:18, Alex Harui wrote: Are we actually using or planning to use? I have not researched any of these test frameworks, but I didn't think any of them handled Marshall Plan configurations. At one point, there was a discussion with the FlexUnit folks of taking some aspects of Mustel

Re: Testing Framework Site Update

2012-02-16 Thread Alex Harui
On 2/16/12 6:02 PM, "Omar Gonzalez" wrote: > We use [FlexUnit][4] for unit testing and [Mockolate][5] for mocks and test > spies. Are we actually using or planning to use? I have not researched any of these test frameworks, but I didn't think any of them handled Marshall Plan configurations.