On Wed, 2016-01-13 at 04:41 -0500, Tzu-Mainn Chen wrote: > Hey all, > > I realize now from the title of the other TripleO/Mistral thread [1] > that > the discussion there may have gotten confused. I think using Mistral > for > TripleO processes that are obviously workflows - stack deployment, > node > registration - makes perfect sense. That thread is exploring > practicalities > for doing that, and I think that's great work. > > What I inappropriately started to address in that thread was a > somewhat > orthogonal point that Dan asked in his original email, namely: > > "what it might look like if we were to use Mistral as a replacement > for the > TripleO API entirely" > > I'd like to create this thread to talk about that; more of a 'should > we' > than 'can we'. And to do that, I want to indulge in a thought > exercise > stemming from an IRC discussion with Dan and others. All, please > correct me > if I've misstated anything. > > The IRC discussion revolved around one use case: deploying a Heat > stack > directly from a Swift container. With an updated patch, the Heat CLI > can > support this functionality natively. Then we don't need a TripleO > API; we > can use Mistral to access that functionality, and we're done, with no > need > for additional code within TripleO. And, as I understand it, that's > the > true motivation for using Mistral instead of a TripleO API: avoiding > custom > code within TripleO.
The True motivation for investigating Mistral was to counter the assertion that we needed to build our own REST API for workflows in the TripleO API spec. This: "We need a REST API that supports the overcloud deployment workflow." https://review.openstack.org/#/c/230432/13/specs/mitaka/tripleo-overclo ud-deployment-api.rst In doing that I was trying to wrap some of the existing code in tripleo-common in Mistral actions and it occurred to me that some things (like the ability to deploy heat templates from Swift) might benefit others more if we put them in the respective client libraries (like heatclient) instead of carrying them in tripleo-common where they only benefit us. Especially since in the heatclient case it already had a --template-object option to begin with (https://bugs.launchpad.net/py thon-heatclient/+bug/1532326) This goes towards a similar pattern we follow with other components like Puppet which is that instead of trying to create our own functionality in puppet-tripleo we are trying to as much as possible add the features we need to the individual modules for each project (puppet-nova, puppet-swift, etc). In other words avoiding custom code within TripleO is just good practice in general, not something that is specific to the Mistral vs TripleO API discussion. When someone looks at TripleO as a project I would very much like them to admire our architecture... not what we've had to build to support it. As an example I'm very glad to see TripleO be a bit more friendly to config management tooling... rather than trying to build our own version (os-apply-config, etc.). And adding code to the right place usually works out better for everyone and can help build up the larger OpenStack community too. > > That's definitely a worthy goal... except from my perspective, the > story > doesn't quite end there. A GUI needs additional functionality, which > boils > down to: understanding the Heat deployment templates in order to > provide > options for a user; and persisting those options within a Heat > environment > file. TripleO API was previously described as what would be our "workflows API" for TripleO. A places for common workflows that are used for the CLI, and UI together (one code path for all!). What you describe here sounds like a different sort of idea which is a GUI helper sort of API. FWIW I think it is totally fine for a GUI to maintain its own API for caching reasons etc if the members of that team find it to be a requirement. Please don't feel that any of the workflow API comparisons block requirements for things to build a GUI though. > > Right away I think we hit a problem. Where does the code for > 'understanding > options' go? Much of that understanding comes from the capabilities > map > in tripleo-heat-templates [2]; it would make sense to me that > responsibility > for that would fall to a TripleO library. > > Still, perhaps we can limit the amount of TripleO code. So to give > API > access to 'getDeploymentOptions', we can create a Mistral workflow. > > Retrieve Heat templates from Swift -> Parse capabilities map I have no issues with this workflow if it helps the UI team to create say a cached copy of the capabilities map. Similar to the above heatclient discussion I would ask why put this into a TripleO library if we don't have to. Could this code perhaps be made into a more generic Heat feature? If not immediately, then perhaps eventually? What I mean is... it would be totally fine to write this code in say TripleO common with an eye towards moving it eventually to someplace better like Heat API (proper) or heatclient perhaps. Regardless of where the code for parsing the capabilities map lives though I think it would be a fine use of a Mistral workflow. > > Which is fine-ish, except from an architectural perspective > 'getDeploymentOptions' violates the abstraction layer between storage > and > business logic, a problem that is compounded because > 'getDeploymentOptions' > is not the only functionality that accesses the Heat templates and > needs > exposure through an API. And, as has been discussed on a separate > TripleO > thread, we're not even sure Swift is sufficient for our needs; one > possible > consideration right now is allowing deployment from templates stored > in > multiple places, such as the file system or git. > > Are we going to have duplicate 'getDeploymentOptions' workflows for > each > storage mechanism? If we consolidate the storage code within a > TripleO > library, do we really need a *workflow* to call a single > function? Is a > thin TripleO API that contains no additional business logic really so > bad > at that point? Do we really need our TripleO libaries to support multiple storage backends at all right now. As a cloud deployer I can store my heat templates wherever I like... git, svn, cvs... whatever. When I'm ready to deploy I would then use python-tripleoclient or the UI to "upload" my local copy of the templates into Swift. At this point I could either use the UI or python-tripleoclient to drive the rest of the deployment to completion. The fact that the TripleO workflows use Swift for storage is actually abstracted from me such that I don't really care much. > > My gut reaction is to say that proposing Mistral in place of a > TripleO API > is to look at the engineering concerns from the wrong direction. The > Mistral alternative comes from a desire to limit custom TripleO code > at all > costs. I think that is an extremely dangerous attitude that leads to > compromises and workarounds that will quickly lead to a shaky code > base > full of design flaws that make it difficult to implement or extend > any > functionality cleanly. Similar to what I said above limiting code in TripleO isn't necessarily the primary desire here I think. It is more of a question of can we use a generic workflow API like Mistral, or should we go and build our own. Whether more or less of the code lives in a TripleO common library is something we can debate and take a step at a time I think. > > I think the correct attitude is to simply look at the problem we're > trying to solve and find the correct architecture. For these get/set > methods that the API needs, it's pretty simple: storage -> some logic > -> > a REST API. Adding a workflow engine on top of that is unneeded, and > I > believe that means it's an incorrect solution. Totally agree on finding the correct architecture. What you suggest as a "dangerous attitude" is actually just trying to ask the question that to me have not yet been asked. Trying out tools and solutions we may not have tried yet. (FWIW I found the dangerous attitude bit to be a bit un-constructive to this conversation and would prefer to leave that verbiage elsewhere) If you feel strongly that this particular case of parsing the capabilities map and managing config settings is not suitably implemented with a generic workflow tool then perhaps we implement both and see how it works out. Continue work on creating TripleO API... I would however like to perhaps see us scale back selling TripleO API as our generic workflow API. Perhaps there still is a niche where it helps with some things and I've got no issue with pursuing this idea. I will likely continue to ask the questions about putting code into the right place (like could some of this be implemented as a Heat API feature instead). Perhaps this just comes in time after we've refined our deployment workflows a bit to accomidate but UI and CLI. Dan > > > Thanks, > Tzu-Mainn Chen > > > > [1] http://lists.openstack.org/pipermail/openstack-dev/2016-January/0 > 83757.html > [2] https://github.com/openstack/tripleo-heat-templates/blob/master/c > apabilities_map.yaml > > _____________________________________________________________________ > _____ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > cribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
