How would the master read the 'results.txt' from the slave's workspace? EdB
On Wed, Sep 4, 2013 at 8:56 PM, OmPrakash Muppirala <bigosma...@gmail.com> wrote: > On Wed, Sep 4, 2013 at 11:49 AM, Erik de Bruin <e...@ixsoftware.nl> wrote: > >> The way I read that article is that each slave runs a separate job. So >> we'd still have to split a Mustella run up in a lot of separate jobs >> (i.e. flex-sdk_mustella-spark.components.Label or something). I don't >> see (yet) how we would collect the results from all the runs and >> combine them into a report we can send to the list(s). >> >> EdB >> >> > We will need to modify the mustella script to take in a list of tests. > Just like how when we pass it a -failures option, it loads the tests from > failures.txt. > > This way, the mustella script can be kept generic, only the list of tests > given to it would vary per VM. The master job will run a simple > test-distributer.sh. The job 'should' have access to Jenkins' env. > variables such as number of slaves, slave names, etc. > > Thoughts? > > Om > > > > >> >> >> On Wed, Sep 4, 2013 at 7:40 PM, OmPrakash Muppirala >> <bigosma...@gmail.com> wrote: >> > On Wed, Sep 4, 2013 at 10:32 AM, Alex Harui <aha...@adobe.com> wrote: >> > >> >> I don't know anything about Jenkins, but how would you get multiple >> >> instances of FP writing to separate flashlog.txt files? Are you saying >> >> there'd be different Vms with different logins/usernames? >> >> >> >> >> > Yes, that is correct. Right now there is a master/slave setup on the one >> > VM we use. In this instance, the slave is local. But theoretically, we >> > could have any number of remote slaves. Each slave would be on its own >> VM >> > (assuming we get committers to subscribe to the free Azure VMs and make >> > them available) >> > >> > More details here [1] >> > >> > Thanks, >> > Om >> > >> > [1] https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds >> > >> > >> >> -Alex >> >> >> >> On 9/4/13 10:24 AM, "OmPrakash Muppirala" <bigosma...@gmail.com> wrote: >> >> >> >> >On Wed, Sep 4, 2013 at 10:15 AM, Alex Harui <aha...@adobe.com> wrote: >> >> > >> >> >> Thanks for trying. The compile should have distributed to the extra >> >> >> cores, but the run phase doesn't. One reason is because the >> >> >>flashlog.txt >> >> >> is captured so only one player instance should run at a time. In >> >> >>theory, >> >> >> we don't need that any more since it was mainly used to catch >> uncaught >> >> >> exceptions, but I don't have any quick and easy ideas on how to make >> it >> >> >> work in parallel. >> >> >> >> >> > >> >> >Map-reducing it using one Master Jenkins and multiple slaves is >> probably >> >> >our best bet. Since almost all tests run independent of each other, >> the >> >> >mapping part should not be very hard. We could just distribute the >> number >> >> >of tests equally among the available slaves. Reducing it would just be >> >> >collecting the output from each flashlog.txt file and dumping it >> >> >somewhere. >> >> > >> >> >Jenkins has built in support of this kind of stuff. >> >> > >> >> >If someone has bandwidth now, they could probably run with this idea >> and >> >> >test it out. If not, I will get to it when my workload becomes a bit >> >> >lighter. >> >> > >> >> >Thanks, >> >> >Om >> >> > >> >> > >> >> >> >> >> >> On 9/4/13 10:07 AM, "Erik de Bruin" <e...@ixsoftware.nl> wrote: >> >> >> >> >> >> >I upgraded the VM, it now runs on 4 cores instead of 2 and has 7 GB >> of >> >> >> >memory. This shaved roughly an hour, hour and a half from the run >> >> >> >time. Not sure if my Azure account "free credits" will stretch to a >> >> >> >month with this config, if not, I'll go back to the 'old' config as >> >> >> >the improvement is not as spectacular as I hoped. >> >> >> > >> >> >> >EdB >> >> >> > >> >> >> > >> >> >> > >> >> >> >On Wed, Sep 4, 2013 at 6:24 PM, Alex Harui <aha...@adobe.com> >> wrote: >> >> >> >> On 9/4/13 9:17 AM, "Kessler CTR Mark J" < >> mark.kessler....@usmc.mil> >> >> >> >>wrote: >> >> >> >> >> >> >> >>>Is the test for [1] misspelled? "Sesision" vs "Session"? >> >> >> >> There are lots of spelling mistakes in test names, but shouldn't >> >> >>affect >> >> >> >> the test itself. >> >> >> >> >> >> >> >> >> >> >> >> -Alex >> >> >> >> >> >> >> > >> >> >> > >> >> >> > >> >> >> >-- >> >> >> >Ix Multimedia Software >> >> >> > >> >> >> >Jan Luykenstraat 27 >> >> >> >3521 VB Utrecht >> >> >> > >> >> >> >T. 06-51952295 >> >> >> >I. www.ixsoftware.nl >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> Ix Multimedia Software >> >> Jan Luykenstraat 27 >> 3521 VB Utrecht >> >> T. 06-51952295 >> I. www.ixsoftware.nl >> -- Ix Multimedia Software Jan Luykenstraat 27 3521 VB Utrecht T. 06-51952295 I. www.ixsoftware.nl
