Ok, just so we're on the same topic: I'm talking about functional tests of the output of the compiler. This means that I want to be able to test if a button with an event that changes the text on a label that we've created using the FlexJS AS framework does, after being put through the FalconJX compiler, actually works using the FlexJS JS framework in IE, Firefox and Chrome. This means that the test framework needs to be able to take a MXML/AS application, compile it, launch the output in a browser, fake a click on the button and check the text of the label to see if it has the correct value.
EdB On Fri, Apr 12, 2013 at 8:31 PM, Alex Harui <aha...@adobe.com> wrote: > > > > On 4/12/13 11:12 AM, "Erik de Bruin" <e...@ixsoftware.nl> wrote: > >>> I don't know much about testing frameworks other than mustella, so I'm >>> mostly asking dumb questions. I don't have a fully-formed plan on how to >>> test all of this stuff, so I'm not saying you made the wrong decision. I'm >>> just curious. How do I test the AS Side? Does this mean I have two >>> versions of each test? >> >> With this solution, you don't test the AS side (yet). Unless we write >> a meta-layer that is interpreted by both an AS functional testing >> framework (mustella?) and a JS functional testing framework >> (marmotinni?), there is no framework Google knows about that does >> functional testing on both AS and JS based on the same test data/file. > I would like to explore what that layer looks like before we author a bunch > of tests that have to be duplicated to test the AS side. I still think > FlexUnit can do functional testing just like Junit can, but I could be > wrong. >> >>> I was just looking at the compiler.jx.tests trying to figure out where to >>> add more tests. These seem like feature tests to me, do you agree? Yet we >>> are using Junit to run them. I haven't used FlexUnit but I assume it can do >>> unit and functional testing just like Junit. And we own the code so we can >>> make it do that if needed. >> >> Feature tests? If by that you mean functional tests, then no, I don't >> agree. The tests in compiler.jx.tests are unit tests that test if the >> output string from FalconJX matches the expected value, given a >> certain input. It in no way tests whether the input or the output >> actually DOES anything. I doesn't check if given the code for a click >> on a button, the code correctly fires an event that causes the text of >> a specified label to change to the correct value. That is functional >> testing, IMHO. > Yeah, I meant functional testing. IMHO, these are functional tests. They > test that the compiler is functioning as a whole. It isn't testing the > individual units like a set of tests for the JSEmitter.java class. And a > separate suite of functional tests for ASJS would probably live in the asjs > repo, not the falcon repo. But I could certainly be wrong here. >> >>> I do think I just want to write the "source" for a test once and then have >>> it run "everywhere". >> >> That would be ideal, and maybe we can use the format that mustella >> uses (MXML if I'm correct?) and make marmotinni use the same format, >> making if possible to use the same functional tests for both the AS >> (Flash Player) side and the JS (browser) side. > IMO, MXML or AS. We're in the business of translating either to JS aren't > we? >> >>> As I've been going through the compiler.jx.tests, it makes me want to >>> refactor those tests as well so they can be used to test Falcon's AS >>> handling too. >> >> The tests in 'org.apache.flex.compiler.internal.codegen.as' test >> exactly that. They test AS to AS compilation of most language features >> (some work needs to be done writing tests for the missing feature). >> All other ASJS test 'suites' (amd, goog, flexjs) subclass these tests >> and only override them where the expected output differs from that of >> the parent. It was set up this way to ensure that starting with >> testing all possible AS output, all the language features would also >> be tested in the subclasses, i.e. the various JS output formats. > Yeah, but it seems like I have to override by copying the entire test > including its source instead of having a set of source inputs and an > overridable set of expected outputs. That's what I was thinking of > refactoring. >> >>> At one point several years ago, Mike L and I discussed authoring FlexUnit >>> tests in MXML. I still like the idea as it more tightly controls the way >>> tests are authored. >> >> As I mentioned above, we might look into mustella's MXML formatted >> tests and see if we can leverage that in marmotinni. I don't think >> FlexUnit (being a unit testing framework) will be able to run >> functional tests in both the Flash Player and the various browsers... > Well, we own the code for FlexUnit, so at the point it takes the compiler > output and launches the player and tears it down (I am just assuming it does > that) maybe we could change it to use Selenium? In looking a bit at your > commits, it appears I need a .java file for each test or set of tests. That > may not be helpful for the AS-side so just being able to declare in MXML or > AS what the expected output is and then have an engine or engines that > compile and run in different configs and test against those outputs seems > like it would enable more folks to write tests. I think it might be > possible to tweak your TextButton.java to pick up its expected output from > some external file and thus become part of that engine. > -- > Alex Harui > Flex SDK Team > Adobe Systems, Inc. > http://blogs.adobe.com/aharui > -- Ix Multimedia Software Jan Luykenstraat 27 3521 VB Utrecht T. 06-51952295 I. www.ixsoftware.nl
