Excellent work Ben, this will really help everyone building integrations based upon the platform API.
Being able to auto-generate SDKs for most langauges is also something that would really benefit the project. Great stuff! On 13 July 2018 at 22:32, Ben Browning <bbrow...@redhat.com> wrote: > I've submitted a PR for the Swagger spec changes and validations at > https://github.com/apache/incubator-openwhisk/pull/3878 > > For this first iteration, I just focused on making the Swagger spec > match the behavior of our existing tests. As Rodric pointed out, that > may not exactly match the REST interface, but it at least gets us > closer. > > As a future second iteration, I'd like to investigate generating and > maintaining a new Java OpenWhisk client from the spec (see > https://github.com/apache/incubator-openwhisk/issues/2466). Using the > spec to generate and test a functioning Java client should help find > and fix more spec issues and pave the way for potentially moving other > hand-generated client libraries over to being at least partially > generated from the API spec. > > Ben > > > On Fri, Jul 6, 2018 at 12:55 AM, Rodric Rabbah <rod...@gmail.com> wrote: > > Ben, you might want to take note of this PR https://github.com/apache/ > > incubator-openwhisk/pull/3840 > > which removes a number of tests suites (redundant with unit tests). Also > > I've found that the WskRestOperations implementation in some places fixes > > behaviors to match the CLI instead of strictly implementing the > > narrower/strict REST interface. This may mean you have to edit tests (or > > remove some that are only valid for clients). > > > > -r > > > > On Thu, Jul 5, 2018 at 9:20 PM, Ben Browning <bbrow...@redhat.com> > wrote: > > > >> After several failed attempts, I have an approach that I believe will > >> work well enough for our needs. I pushed some in-progress code and > >> swagger spec fixes to a branch on a fork just so the general approach > >> can be shared - > >> https://github.com/projectodd/incubator-openwhisk/commit/5ad > >> b6be27188dcdc5b9519b57c11802a431b3e9c > >> > >> This approach uses the swagger-request-validator Java library. I'm > >> wiring that up to validate all requests and responses from the > >> WskRest* tests transparently, which you can see in > >> https://github.com/projectodd/incubator-openwhisk/commit/5ad > >> b6be27188dcdc5b9519b57c11802a431b3e9c#diff-5e514c6c3c27b9210 > >> d85a08d4ed3ed35 > >> (with some cruft and debug output that I'll clean up before submitting > >> a PR) > >> > >> This means that almost all of those tests are now failing due to > >> failed swagger spec validations. For now I'm going to concentrate on > >> making the spec match the reality and try to get all the existing > >> tests green with the validation in place and submit a PR. After that, > >> we can patch up any known but uncaught holes by increasing the test > >> coverage. > >> > >> Ben > >> > >> > >> > >> On Thu, Jun 28, 2018 at 11:23 PM, Carlos Santana <csantan...@gmail.com> > >> wrote: > >> > Thanks Ben for the quick update on this task > >> > > >> > -cs > >> > > >> > On Thu, Jun 28, 2018 at 5:15 PM Ben Browning <bbrow...@redhat.com> > >> wrote: > >> > > >> >> After doing some preliminary poking at this, I believe we'll want to > >> >> use either a tool like https://github.com/google/oatts to generate a > >> >> test suite from our Swagger spec OR use swagger-codegen to generate a > >> >> Scala client from our Swagger spec and try to plug that into the > >> >> existing WskRest tests. > >> >> > >> >> Using the oatts tool doesn't really fit well with the test setup in > >> >> the existing incubator-openwhisk repo (where the API spec lives) > >> >> because that generates Node.js tests. > >> >> > >> >> So, I'm leaning towards the second option, which is wiring in > >> >> generation of a Scala client into the gradle build and having the > >> >> current WskRest test client use this generated Scala client for > >> >> testing instead of directly invoking URLs. > >> >> > >> >> However, last time I played with Scala code generated from Swagger > >> >> specs it wasn't that usable. So, a bit more experimentation will > >> >> validate whether this option is viable or if other alternatives need > >> >> to be considered. I already have a handful of bugs in the API spec > >> >> that need to be fixed but I'm waiting to fix and push those until I > >> >> can get some kind of testing wired up to reproduce the bugs and > verify > >> >> the fix. > >> >> > >> >> Ben > >> >> > >> >> > >> >> On Thu, Jun 21, 2018 at 3:04 PM, Carlos Santana < > csantan...@gmail.com> > >> >> wrote: > >> >> > Thanks Ben for looking into this, having a good API doc/spec and > >> matching > >> >> > tests is very need it. > >> >> > > >> >> > +1 > >> >> > > >> >> > -cs > >> >> > > >> >> > On Thu, Jun 21, 2018 at 2:25 PM Ben Browning <bbrow...@redhat.com> > >> >> wrote: > >> >> > > >> >> >> Our Swagger spec > >> >> >> ( > >> >> >> > >> >> https://github.com/apache/incubator-openwhisk/blob/92a64c291 > >> 156a2cd3d6b304babc2a193a46d0699/core/controller/src/main/ > >> resources/apiv1swagger.json > >> >> >> ) > >> >> >> is incomplete and doesn't accurately reflect the actual Controller > >> >> >> API. It's manually updated without a full test suite which means > it's > >> >> >> easy for changes in the API to happen without the spec getting > >> >> >> updated. > >> >> >> > >> >> >> An accurate Swagger spec will not only better document the > OpenWhisk > >> >> >> API but also allow autogenerated clients in multiple languages to > >> >> >> supplement or eventually replace some of the existing client > >> >> >> implementations we have today. It also paves way for future > >> compatible > >> >> >> server implementations, whether they be rewrites of the existing > >> >> >> Controller or stub test harnesses to facilitate end-to-end > testing on > >> >> >> a developer's laptop. > >> >> >> > >> >> >> As I'm already working with autogenerating code from the Swagger > spec > >> >> >> for other purposes, I'm happy to take the lead on this effort. I'd > >> >> >> like to take a two-pronged approach for a test suite: > >> >> >> > >> >> >> * Generate a server stub from the spec and ensure the wsk CLI can > >> >> >> communicate with it. > >> >> >> > >> >> >> * Generate a client stub from the spec and ensure it can > communicate > >> >> >> with the existing API. > >> >> >> > >> >> >> There are a lot of details to figure out from those two > statements. > >> >> >> And, this approach won't guarantee 100% correctness of the spec. > The > >> >> >> only way to do that would be to generate all supported clients and > >> the > >> >> >> Controller API from the spec. But, this should get us started in > the > >> >> >> right direction. > >> >> >> > >> >> >> If anyone's gone down this path before and has some wisdom to > share, > >> >> >> please speak up! > >> >> >> > >> >> >> Thanks, > >> >> >> > >> >> >> Ben > >> >> >> > >> >> > >> > -- Regards, James Thomas
