Works like a charm. Well done. ----- Original Message ----- From: "Richard Pappalardo" <[email protected]> To: "Sam Penrose" <[email protected]> Cc: "Jared Hirsch" <[email protected]>, "No-Jun Park" <[email protected]>, "Edwin Wong" <[email protected]>, [email protected] Sent: Wednesday, May 14, 2014 11:31:22 AM Subject: Re: FxA automation
Hi guys, Please pull down the new testcase Makefile into your gaia dir. I've simplified the test menu and added ability to specify the fxa server. Things work a bit differently, but hopefully the 'Usage' comment will suffice. Let me know, however, if it's confusing. $ cp Makefile Makefile.BAK; curl -O https://raw.githubusercontent.com/rpappalax/gaia/bug-989368-fxa-auto/Makefile $ make fxa-test To download the latest testcases, run: $ make fxa-test-new FYI: the "smoke_settings_test" will test both Settings > Firefox Accounts and Settings > Find My Device user flows. Still working on adding in other scenarios. Thanks, -Richard On 5/8/14, 12:42 PM, Sam Penrose wrote: So, this is fantastic! I've been doing 90%/10% Gecko/Gaia development using a combination of unit and manual testing -- all the inaccuracy of the former plus all the tedium and slow turnaround of the latter. Richard's work lets me have the convenience of unit tests (kickoff shell command, refill coffee cup, inspect result) with the value of integration testing. To test gecko specifically, just replace the b2g directory in gaia/ with a link to "dist/B2GDebug.app" inside your Gecko B2G desktop build. This looks looks like: # in .mozconfig mk_add_options MOZ_OBJDIR=</path/from/root/to/>build_unified_b2g # shell command ln -s </path/from/root/to/>build_unified_b2g/dist/B2GDebug.app </path/to/gaia/>b2g Next steps are: 1) Add test cases to make fxa-test. Right now it covers only a part of the state space. 2) Add controls so we can specify which cases to run per invocation. 3) Think about server usage. In my dream world, we run against prod, with some sort of flags set on our data to prevent contaminating metrics. This is obviously an organizational issue, and I'd love to get everyone together to talk about the best way to do effective TDD. ----- Original Message ----- From: "Richard Pappalardo" <[email protected]> To: "Jared Hirsch" <[email protected]> , "Sam Penrose" <[email protected]> , "No-Jun Park" <[email protected]> Cc: "Edwin Wong" <[email protected]> , [email protected] Sent: Tuesday, May 6, 2014 2:58:08 PM Subject: FxA automation Hey folks, Had a minor setback with a marionette js bug in the latest master, but thanks to Gareth Aye, I was able to put a workaround in place. If you'd like to begin using the automation to validate your work, you can now grab the tests easily from my branch. cd to your gaia directory and run the following: $ cp Makefile Makefile.BAK; curl -O https://raw.githubusercontent.com/rpappalax/gaia/bug-989368-fxa-auto/Makefile then to download files into your gaia dir: $ make fxa-update run automation $ make fxa-test Let me know if you have any problems with it. make fxa-test is currently set to run one (smoke) test for you to do a quick validation, but other tests can be specified in FXA_TEST_MANIFEST No-Jun and I have already been working together to put other tests in place. We'll be working directly from the FxA product backlog, but please let us know if there are other features that we should give priority to for automation that would help you. I know Sam mentioned something about an Oath test last week and Jed had requested tests for the RP API, which I still need to look into. Our primary goal is to create tests you can use right now to validate your work, but I'll also be extracting from those simple feature tests to ultimately be landed in gaia, where possible. Thanks! -R -- Cloud Services QA Mozilla Corporation Phone: 314.374.7249 _______________________________________________ Dev-fxacct mailing list [email protected] https://mail.mozilla.org/listinfo/dev-fxacct
