I want testing to be as painless and as reliable as possible. Perhaps the IaaS route is a good first step. If we can get the suite running in 10 minutes or less on an IaaS grid, that's a lot of runs that could happen. In addition, if we ever move to Git and use Gerrit for code reviews, there's a Jenkins plugin that will run builds based on any given changeset, which would effectively let anyone do it by way of submitting to the code review. https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Plugin
Is there a Jenkins plugin to apply an attached patch to a JIRA issue? I did a quick scan and didn't see one. There is a JIRA plugin that can update JIRA issues based on issue ID's mentioned in commit comments, though. https://wiki.jenkins-ci.org/display/JENKINS/JIRA+Plugin Also, what's 'hurricane'? I saw it mentioned in the mustella source yesterday, but the word hurricane is too generic for Google to be of any help. Every phrase I thought of didn't turn up anything useful to answer that question on my own. Jeff On Tue, Aug 14, 2012 at 12:10 PM, Peter Ent <p...@adobe.com> wrote: > Hi Jeff, > > You are suggesting that if someone has the horsepower available (i.e., > access to an IaaS cloud), the mustella tests should run, in their entirety > in 10 mins or less - versus - anyone being able to do this, correct? I > think I'm reading more into your 10-minutes-or-less suggestion than > intended. > > I do think the tests can be broken down into better units as they are all > kind of lumped together, so that's a particularly good goal. > > --peter > > On 8/14/12 12:01 PM, "Jeff Conrad" <jeff...@gmail.com> wrote: > > >Hi, > > > >I'd like to help the project get to a point to where we can run the entire > >test suite for the sdk in 10 minutes or less. I think that's a worthy > >goal, and I'm willing to help make that a reality. > > > >If we get the testing time down to being that fast, that will mean we can > >review a lot more contributions, and it could also mean that potential > >contributors can submit patches that they know don't fail any tests before > >submitting the patch. It also means that if one of those tests fail, the > >person writing the code still has the context of what they did fresh in > >their mind and can fix it quickly. > > > >For reference, I ran the entire Mustella suite last night, and it took 4 > >hours, 3 minutes, and 50 seconds to run ./mini_run.sh -createImages -all > >last night on a quad-core machine running Windows 7 using the Git Bash > >(yay! no cygwin!). To make it work with the Git Bash, I changed the shell > >variable in mustella/build.xml to just sh.exe. As a plus, shellrunner.sh > >somewhat intelligently parallel-ized the compilation of all the test swfs > >so it was compiling 4 swfs at a time, and then the ant script ran all the > >test swfs one at a time. I know mustella is more a functional / > >integration test suite, so by definition it's going to run slower than a > >suite of unit tests. > > > >I also know that at least a few people on this list want to refactor the > >sdk so it's unit-testable. I definitely support this and would like to > >help in any way I can on that front. > > > >I want the entire suite: unit, integration, and functional to be able to > >run in 10 minutes or less. I have two ideas as to how we can make this > >happen, and I'm open to more. > > > >One idea would be to intelligently look at the files that a given patch / > >changeset affects and only build and run the tests that test that > >functionality. > > > >The other idea that comes to mind is spin up a group of virtual servers in > >an IaaS cloud and split up the workload. If we can evenly split this up, > >and my original number is right, it means 24+ servers. There are easy > >ways > >to get the files across that many servers quickly, so the file transfer > >part wouldn't be that hard. > > > >What do you guys think? Where would you start? > > > >Jeff > >
