Personally, I never use mobile-spec because it is buggy on windows+wp8, and nearly impossible to tell if a failure is a bad test or bad code, (this may be better now, but I've already lost faith). I needed a way to easily test a particular plugin in isolation, which is why I wrote paramedic.
The cordova-plugin-test-framework will automagically run the auto-tests if there is a medic.json file in the app package. I leveraged this to get the test results posted back to the command line in paramedic. Ultimately the intent of paramedic is to run jasmine-2 based tests written according to cordova-plugin-test-frameworks's idioms for every pull request to every core plugin and display the results back in github. I will add the ability to specify multiple plugins as an arg, the rest I am somewhat indifferent about. Manual tests will always have a place because many APIs require user interaction. mobile-spec is still good at this, flakey UI aside. Ultimately though, paramedic is designed for non-interactive testing, like what would be done by TravisCI and hopefully soon, Appveyor. @purplecabbage risingj.com On Fri, Feb 27, 2015 at 2:30 PM, Dmitry Blotsky <dblot...@microsoft.com> wrote: > Thanks for the feedback, Michal! The issue we had with using the > checkboxes was that it was not easily automatable. A more simple solution > was to use a parameter to specify the plugins you want installed, instead > of installing all of them and then disabling them. Also the > dependencies-plugin in mobilespec is a hardcoded list, which requires more > manipulation to modify than specifying a list as a parameter. Using it as > the default is a great idea, but being able to override it during > invocation would be great too I think. What do you think? > > Also: Jesse, and others, do you think it might be worthwhile to > collaborate and incorporate the features of paramedic (like specifying the > plugins to run) into mobilespec? That way we will have one tool for testing > throughout. I've definitely been sold on the idea of having mobilespec be > the main point for running tests. That way we can use it in all our testing > scenarios: local, Medic, Travis, etc. > > Kindly, > Dmitry > > -----Original Message----- > From: mmo...@google.com [mailto:mmo...@google.com] On Behalf Of Michal > Mocny > Sent: Wednesday, February 25, 2015 12:42 PM > To: dev > Subject: Re: Cordova Automated Testing > > I haven't followed this full thread, so sorry if this is out of context, > but wanted to point out: > > - Jason added support for testing a subset of plugins in mobile-spec by > way of checkboxes in the automated test runner. It defaults to all, but > you can change that. > - By design, the cordova-plugin-test-framework (and thus mobile-spec) will > only run tests for the plugins you have actually installed (actually, for > the nested plugin tests you have explicitly installed). The > createmobilespec.js script will by default install all core plugins, but > you can easily change that. > > -Michal > > On Wed, Feb 25, 2015 at 3:20 PM, Dmitry Blotsky <dblot...@microsoft.com> > wrote: > > > Thanks for the feedback, Jesse and Dmitriy! > > > > Judging from the responses, it definitely sounds like modifying > > mobilespec, or just plain using paramedic would be the way to go. I'm > > very much swinging in that direction. Medivac really only exists on > > its own because I wasn't comfortable enough with our git repos to make > > a branch off of mobilespec; because it really is just mobilespec with > > some things stripped away. > > > > Jesse: to me the primary value add was actually just simplicity. This > > guy doesn't depend on plugin-test-framework and has none of the > > heavier features like local storage and test segmentation. It's > > basically just a Cordova app with a single page to run Jasmine tests > > and an optional URI to post the results to, and the value is > > simplicity. It has very little code and has very few moving parts. > > > > Dmitriy Barkalov: I don't see what the use case is for adding > > plugin-test-framework to any Cordova app other than a blank one, and > > that is basically just doing manually what mobilespec does automatically. > > > > Kindly, > > Dmitry > > > > -----Original Message----- > > From: Jesse [mailto:purplecabb...@gmail.com] > > Sent: Wednesday, February 25, 2015 11:36 AM > > To: dev@cordova.apache.org > > Subject: Re: Cordova Automated Testing > > > > Thanks for contributing, however, I do not see where the value add is > > in this. > > mobile-spec does much of this already, as does > > cordova-plugin-test-framework. > > > > Adding an argument to test an individual plugin is a good idea, I > > would definitely support adding this to mobile-spec. > > > > My own cordova-paramedic is also designed for simplified plugin > > testing, so you can test an individual plugin, and see the results > > back on the command line. Primarily this was intended for CI with > > Travis, so we could see immediately if a plugin pull request breaks a > test on a platform. > > > > We can discuss further here, but I think medivac would be better > > served as a fork+pull request to cordova-medic > > > > gh:purplecabbage/cordova-paramedic > > > > > > @purplecabbage > > risingj.com > > > > On Wed, Feb 25, 2015 at 9:35 AM, Dmitriy Barkalov (Akvelon) < > > v-dmb...@microsoft.com> wrote: > > > > > Dmitry, > > > > > > I like your idea of adding tests from different formats. I.e. you > > > search for standard Cordova plugin tests like in core plugins and > > > for non-standard tests like plane test.js file. I think it is a good > > > point to provide a way for plugin authors to either reformat their > > > tests to existing format or provide new test format parser. > > > Unfortunately, we can't run away from requirement to use Jasmin 2.0 > > > but still it is more > > freedom. > > > > > > It also seems that in many ways your application is doing the same > > > things as plugin testing framework. It provides a page to run tests, > > > holds Jasmine and puts results into CouchDB. But it is less flexible > > > because cannot be included into other application, while any Cordova > > > application can install plugin testing framework and get a page to > > > run > > manual and automated tests. > > > > > > So I vote for porting some of the new features from this tool back > > > to plugin testing framework and letting developers reuse this > functionality. > > > > > > > > > Regards, Dmitriy > > > > > > -----Original Message----- > > > From: Dmitry Blotsky [mailto:dblot...@microsoft.com] > > > Sent: Wednesday, February 25, 2015 11:47 AM > > > To: dev@cordova.apache.org > > > Subject: Re: Cordova Automated Testing > > > > > > Hi Shazron, > > > > > > Thank you for your feedback! It actually doesn’t work on iOS because > > > it seems like the most recent whitelist changes broke things. I ran > > > "npm > > test" > > > for the code in master and some of them fail because of whitelisting. > > > That’s just my intuition though: I don’t know exactly why it’s not > > > working yet. > > > > > > Regarding reporting though, reporting to localhost is actually just > > > the default - you can pass a host and port on the CLI and it will > > > report to a CouchDB server like mobilespec did. There is an > > > immediate next step in my development efforts to make it not tied to > > > CouchDB, or to at least allow the user to specify the URI format for > reporting. > > > Also, thanks for pointing out the ‘npm install’ requirement. I added > > > it to the docs. The reporting to console also happens, but it’s > > currently just limited to console.log. > > > Medivac uses all the standard Jasmine reporters (HTML, console, and > > JSAPI). > > > > > > And indeed, I will definitely drop a line to Jesse. Is it > > > well-received in the Apache community to schedule meetings to > > > discuss details, or should detailed discussions also be happening on > > > the mailing > > list? > > > > > > Kindly, > > > Dmitry > > > > > > > On Feb 24, 2015, at 3:32 PM, Shazron <shaz...@gmail.com> wrote: > > > > > > > > Also, the tool does not work on the device (I tried iOS), unlike > > > > mobile-spec since it's trying to connect to localhost for reporting. > > > > > > > > On Mon, Feb 23, 2015 at 4:30 PM, Dmitry Blotsky > > > > <dblot...@microsoft.com> > > > wrote: > > > >> Hi list, > > > >> > > > >> Over the past few weeks, I've developed an automated > > > >> mobilespec-like > > > tool in the process of resuscitating Cordova's CI. It's tentatively > > > called cordova-medivac, and it can currently be found here: > > > http://github.com/dblotsky/cordova-medivac. I'd like to pitch it to > > > the community as a potential alternative or replacement for > > > mobilespec. There have been several tools in the past few weeks on > > > the list, and I've tried to incorporate the features they have into > > > this > > tool. > > > >> > > > >> It contains the following new features: > > > >> > > > >> - Testing any list of plugins at a time - not just the core > > > ones; they're passed as a CLI argument > > > >> > > > >> - Support for "new-style" plugin-test-framework-only tests, > > as > > > well just pure Jasmine spec files that don't use test-framework > > > >> > > > >> - No manual interaction required - it is very basic and > just > > > runs the Jasmine tests as soon as it starts > > > >> > > > >> - Very little code - it basically just creates a simple > dummy > > > app with plugins (like mobilespec) > > > >> > > > >> - Cleaner code - some refactoring was done to remove a few > > > code smells > > > >> > > > >> - No flaky UI > > > >> > > > >> - Optionally reporting test results to a URI (currently > > > assumed to be a CouchDB server) > > > >> > > > >> Current shortcomings: > > > >> > > > >> - The special plugins inside cordova-mobile-spec that > contain > > > tests have not yet been copied over > > > >> > > > >> - The code is similar, so some functionality is not exactly > > > the same, so it's not 100% compatible with mobilespec > > > >> > > > >> - Some error-checking is still missing (like verifying that > > > CLI/JS/lib are locally linked instead of global) > > > >> > > > >> - Cordova-JS Grunt build task is not being automatically > run > > > on creation > > > >> > > > >> - Barebones hard-coded list of "core" plugins; doesn't > > include > > > convenience lists like the new ones in mobilespec > > > >> > > > >> - Doesn't support current manual tests > > > >> > > > >> I'd like to ask you if you would support any of the following > > > >> proposed > > > ideas: > > > >> > > > >> - Reach 100% parity and replace cordova-mobile-spec with > this > > > tool or, > > > >> > > > >> - If not replace, then port some of the code cleanup and > new > > > features from this tool back to mobilespec or, > > > >> > > > >> - Use this tool for the CI (cordova-medic) only, and keep > > > using mobilespec for other tasks > > > >> > > > >> Kindly, > > > >> Dmitry > > > >> > > > > > > > > ------------------------------------------------------------------ > > > > -- > > > > - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > > > > For additional commands, e-mail: dev-h...@cordova.apache.org > > > > > > > > > > > > > -------------------------------------------------------------------- > > > - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > > > For additional commands, e-mail: dev-h...@cordova.apache.org > > > > > > -------------------------------------------------------------------- > > > - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > > > For additional commands, e-mail: dev-h...@cordova.apache.org > > > > > >
