On Wed, Jan 27, 2016 at 10:22:54AM -0500, Dan Prince wrote: > On Wed, 2016-01-27 at 09:36 -0500, Dan Prince wrote: > > On Wed, 2016-01-27 at 14:32 +0100, Jiri Tomasek wrote: > > > On 01/26/2016 09:05 PM, Ben Nemec wrote: > > > > On 01/25/2016 04:36 PM, Dan Prince wrote: > > > > > On Mon, 2016-01-25 at 15:31 -0600, Ben Nemec wrote: > > > > > > On 01/22/2016 06:19 PM, Dan Prince wrote: > > > > > > > On Fri, 2016-01-22 at 11:24 -0600, Ben Nemec wrote: > > > > > > > > So I haven't weighed in on this yet, in part because I > > > > > > > > was > > > > > > > > on > > > > > > > > vacation > > > > > > > > when it was first proposed and missed a lot of the > > > > > > > > initial > > > > > > > > discussion, > > > > > > > > and also because I wanted to take some time to order my > > > > > > > > thoughts > > > > > > > > on > > > > > > > > it. > > > > > > > > Also because my initial reaction...was not conducive to > > > > > > > > calm and > > > > > > > > rational discussion. ;-) > > > > > > > > > > > > > > > > The tldr is that I don't like it. To explain why, I'm > > > > > > > > going to > > > > > > > > make > > > > > > > > a > > > > > > > > list (everyone loves lists, right? Top $NUMBER reasons we > > > > > > > > should > > > > > > > > stop > > > > > > > > expecting other people to write our API for us): > > > > > > > > > > > > > > > > 1) We've been down this road before. Except last time it > > > > > > > > was > > > > > > > > with > > > > > > > > Heat. > > > > > > > > I'm being somewhat tongue-in-cheek here, but expecting > > > > > > > > a > > > > > > > > general > > > > > > > > service to provide us a user-friendly API for our > > > > > > > > specific > > > > > > > > use > > > > > > > > case > > > > > > > > just > > > > > > > > doesn't make sense to me. > > > > > > > We've been down this road with Heat yes. But we are > > > > > > > currently > > > > > > > using > > > > > > > Heat for some things that we arguable should be (a > > > > > > > workflows > > > > > > > tool > > > > > > > might > > > > > > > help offload some stuff out of Heat). Also we haven't > > > > > > > implemented > > > > > > > custom Heat resources for TripleO either. There are mixed > > > > > > > opinions > > > > > > > on > > > > > > > this but plugging in your code to a generic API is quite > > > > > > > nice > > > > > > > sometimes. > > > > > > > > > > > > > > That is the beauty of Mistral I think. Unlike Heat it > > > > > > > actually > > > > > > > encourages you to customize it with custom Python actions. > > > > > > > Anything > > > > > > > we > > > > > > > want in tripleo-common can become our own Mistral action > > > > > > > (these get > > > > > > > registered with stevedore entry points so we'd own the > > > > > > > code) > > > > > > > and > > > > > > > the > > > > > > > YAML workflows just tie them together via tasks. > > > > > > > > > > > > > > We don't have to go off and build our own proxy deployment > > > > > > > workflow > > > > > > > API. The structure to do just about anything we need > > > > > > > already > > > > > > > exists > > > > > > > so > > > > > > > why not go and use it? > > > > > > > > > > > > > > > 2) The TripleO API is not a workflow API. I also largely > > > > > > > > missed > > > > > > > > this > > > > > > > > discussion, but the TripleO API is a _Deployment_ > > > > > > > > API. In > > > > > > > > some > > > > > > > > cases > > > > > > > > there also happens to be a workflow going on behind the > > > > > > > > scenes, > > > > > > > > but > > > > > > > > honestly that's not something I want our users to have to > > > > > > > > care > > > > > > > > about. > > > > > > > Agree that users don't have to care about this. > > > > > > > > > > > > > > Users can get as involved as they want here. Most users I > > > > > > > think > > > > > > > will > > > > > > > use python-tripleoclient to drive the deployment or the new > > > > > > > UI. > > > > > > > They > > > > > > > don't have to interact with Mistral directly unless they > > > > > > > really > > > > > > > want > > > > > > > to. So whether we choose to build our own API or use a > > > > > > > generic one > > > > > > > I > > > > > > > think this point is mute. > > > > > > Okay, I think this is a very fundamental point, and I believe > > > > > > it gets > > > > > > right to the heart of my objection to the proposed change. > > > > > > > > > > > > When I hear you say that users will use tripleoclient to talk > > > > > > to > > > > > > Mistral, it raises a big flag. Then I look at something like > > > > > > https://github.com/dprince/python-tripleoclient/commit/77ffd2 > > > > > > fa > > > > > > 7b1642 > > > > > > b9f05713ca30b8a27ec4b322b7 > > > > > > and the flag gets bigger. > > > > > > > > > > > > The thing is that there's a whole bunch of business logic > > > > > > currently > > > > > > sitting in the client that shouldn't/can't be there. There > > > > > > are > > > > > > historical reasons for it, but the important thing is that > > > > > > the > > > > > > current > > > > > > client architecture is terribly flawed. Business logic > > > > > > should > > > > > > never > > > > > > live in the client like it does today. > > > > > Totally agree here. In fact I have removed business logic from > > > > > python- > > > > > tripleoclient in this patch and moved it into a Mistral action. > > > > > Which > > > > > can then be used via a stable API from anywhere. > > > > > > > > > > > Looking at that change, I see a bunch of business logic > > > > > > around > > > > > > taking > > > > > > our configuration and passing it to Mistral. In order for us > > > > > > to do > > > > > > something like that and have a sustainable GUI, that code > > > > > > _has_ > > > > > > to > > > > > > live > > > > > > behind an API that the GUI and CLI alike can call. If we ask > > > > > > the GUI > > > > > > to > > > > > > re-implement that code, then we're doomed to divergence > > > > > > between > > > > > > the > > > > > > CLI > > > > > > and GUI code and we'll most likely end up back where we are > > > > > > with a > > > > > > GUI > > > > > > that can't deploy half of our features because they were > > > > > > implemented > > > > > > solely with the CLI in mind and made assumptions the GUI > > > > > > can't > > > > > > meet. > > > > > The latest feedback I've gotten from working with the UI > > > > > developers on > > > > > this was that we should have a workflow to create the > > > > > environment. That > > > > > would get called via the Mistral API via python-tripleoclient > > > > > and > > > > > any > > > > > sort of UI you could imagine and would essentially give us a > > > > > stable > > > > > environment interface. > > > > Anything that requires tripleoclient means !GUI though. I know > > > > the > > > > current GUI still has a bunch of dependencies on the CLI, but > > > > that > > > > seems > > > > like something we need to fix, not a pattern to repeat. I still > > > > think > > > > any sentence containing "call Mistral via tripleoclient" is > > > > indicative > > > > of a problem in the design. > > > > > > I am not sure I understand the argument here. > > > > > > Regardless of which API we use (Mistral API or TripleO API) GUI is > > > going > > > to call the API and tripleoclient (CLI) is going to call the API > > > (through mistralclient - impl. detail). > > > > > > GUI can't and does not call API through tripleoclient. This is why > > > the > > > work on extracting the common business logic to tripleo-common > > > happened. > > > So tripleo-common is the place which holds the business logic. > > > > > > The proposed API (in the spec) is supposed only to work as a thin > > > layer > > > that provides the tripleo-common functionality. > > > > > > > > > Most of the operations we do in TripleO are workflows that consists > > > of > > > several requests to various OpenStack services, e.g. Nodes > > > Introspection, deployment progress tracking, etc. Mistral fits as a > > > best > > > tool to use to manage this and provides an API that can be used > > > directly > > > by both CLI and GUI. The work involved is just about turning > > > tripleo-common into Mistral Actions. > > > Alternatives are to create TripleO API and implement our own > > > custom > > > workflows (reinvent the Mistral wheel) as part of tripleo-common. > > > Or > > > Shield the Mistral API behind a very thin TripleO API. Which in my > > > opinion is not worth it and just adds new layer to the project > > > (GUI/CLI > > > > TripleO API > mistralclient > Mistral API > tripleo-common > > > actions) > > > > > > But there are also some operations that do not require a workflow. > > > Such > > > as listing available environments, posting the parameter values > > > and > > > storing them in temporary environment... In those cases the Mistral > > > API > > > does not work as good as TripleO API because Mistral API is able > > > to > > > start a workflow but is not able to take a request and return an > > > immediate response to that request. > > > > What I did in my demo/prototype was to just use Mistral's generic API > > to update the environment directly. The things we want to store in > > the > > environment are actually quite simple: > > > > -container name (for heat templates) > > -set of selected heat environments > > -extra parameters > > -cached things to help UI select these things like a capabilities > > map > > > > Last time we talked on there was interest in creating an "interface" > > around managing creating/updating/deleting these things... and I > > suggested we could use a workflow to do it just the same. > > > > That isn't my preference, I'm fine with defining these as a loose > > data > > structure (a set of properties really). If we do it that we you can > > simply use the Mistral REST API to update an environment quite > > nicely. > > > > If we prefer a more rigidly defined data structure we could wrap the > > same logic with workflows to manage it. The UI/CLI could still read > > the > > information directly via the Mistral environments API, and just use a > > workflow to update it. Or we could use workflows for everything. Any > > combination would be possible here. > > > > Mistral environments are a useful abstraction for workflows. At the > > end > > of the day nothing would prevent an end user from using mistralclient > > to manage the same thing via a JSON file... But if we really want to > > wrap it with a workflow we really could do that. I think for the > > simple > > data construct we are talking about here it really is fine to manage > > as > > just a JSON POST request to Mistral's environments API: > > > > http://paste.openstack.org/show/485140/ > > > > Dan > > > > > > > The Mistral workflow needs to notify > > > (e.g. via Zaqar) or get polled for a result of the Workflow > > > execution. > > I think I misread your request here. For now polling is required. I'd > just create a common Python and/or Javascript function to do this for > python-tripleoclient and the UI. That way you can call a workflow and > wait on the result in a synchronous manner if you want a convenient way > to do it. > > Lets do proceed with looking at Zaqar integration and see if that > helps. That seems like the best long term solution. I don't think it > would be very hard to integrate with Zaqar within our workflows...
As a first step, I should point out that heat already supports Zaqar for event notification, via a simple addition to the heat environment: http://docs.openstack.org/developer/heat/template_guide/environment.html#retrieving-events So, it might make sense as an interim step to not poll for events from the heat stack while the deployment workflow is executing - we could just get events via Zaqar instead (which will probably be a lot more efficient). For example, here's some test code showing waiting for a completion event via websocket and Zaqar (thanks therve!): https://gist.github.com/therve/9c7c762d3eb81b076be4 Sorry, this is a bit OT from this thread, but I wanted to point out we do have some Zaqar integration already (just not yet directly in Mistral AIUI) Steve __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
