On 01/14/2016 12:54 PM, Steven Hardy wrote:
On Wed, Jan 13, 2016 at 04:41:28AM -0500, Tzu-Mainn Chen wrote:
Hey all,
I realize now from the title of the other TripleO/Mistral thread [1] that
the discussion there may have gotten confused. I think using Mistral for
TripleO processes that are obviously workflows - stack deployment, node
registration - makes perfect sense. That thread is exploring practicalities
for doing that, and I think that's great work.
What I inappropriately started to address in that thread was a somewhat
orthogonal point that Dan asked in his original email, namely:
"what it might look like if we were to use Mistral as a replacement for the
TripleO API entirely"
I'd like to create this thread to talk about that; more of a 'should we'
than 'can we'. And to do that, I want to indulge in a thought exercise
stemming from an IRC discussion with Dan and others. All, please correct me
if I've misstated anything.
The IRC discussion revolved around one use case: deploying a Heat stack
directly from a Swift container. With an updated patch, the Heat CLI can
support this functionality natively. Then we don't need a TripleO API; we
can use Mistral to access that functionality, and we're done, with no need
for additional code within TripleO. And, as I understand it, that's the
true motivation for using Mistral instead of a TripleO API: avoiding custom
code within TripleO.
That's definitely a worthy goal... except from my perspective, the story
doesn't quite end there. A GUI needs additional functionality, which boils
down to: understanding the Heat deployment templates in order to provide
options for a user; and persisting those options within a Heat environment
file.
Right away I think we hit a problem. Where does the code for 'understanding
options' go? Much of that understanding comes from the capabilities map
in tripleo-heat-templates [2]; it would make sense to me that responsibility
for that would fall to a TripleO library.
Still, perhaps we can limit the amount of TripleO code. So to give API
access to 'getDeploymentOptions', we can create a Mistral workflow.
Retrieve Heat templates from Swift -> Parse capabilities map
Which is fine-ish, except from an architectural perspective
'getDeploymentOptions' violates the abstraction layer between storage and
business logic, a problem that is compounded because 'getDeploymentOptions'
is not the only functionality that accesses the Heat templates and needs
exposure through an API. And, as has been discussed on a separate TripleO
thread, we're not even sure Swift is sufficient for our needs; one possible
consideration right now is allowing deployment from templates stored in
multiple places, such as the file system or git.
Actually, that whole capabilities map thing is a workaround for a missing
feature in Heat, which I have proposed, but am having a hard time reaching
consensus on within the Heat community:
https://review.openstack.org/#/c/196656/
Given that is a large part of what's anticipated to be provided by the
proposed TripleO API, I'd welcome feedback and collaboration so we can move
that forward, vs solving only for TripleO.
Yes, the original intent was to provide user with a means to safely
construct the deployment template tree which iiuc is what this proposed
feature provides. Then in the process of figuring out how to provide
user with a reasonable choices for designing the deployment, I realized
that more natural way is to provide the choices on the environments
level (which is what we already do in THT - provide the list of
alternative/complementary environments) The capabilities map provides a
list of environments which user is able to choose from when designing
the deployment supporting the choice with environment description and
information about whether the environment is required to use, if it
depends on the use of other environment or if the environment is one of
the mutually exclusive choices.
So in addition to what the mentioned spec defines, to implement a
replacement for capabilities_map in Heat, we'd need a way to specify
environment description, other environments dependency, mutual
exclusivity to other environments etc from within environment itself.
Heat then could return a response which resembles to what
capabilities_map provides now.
Maybe the current spec is sufficient and based on the information that
this spec brings Heat would already be able to identify the
inter-environment behavior based on parsing the resource_registry in the
environments? Only thing we'd need to add is a 'description' to
environment file.
-- Jirka
Are we going to have duplicate 'getDeploymentOptions' workflows for each
storage mechanism? If we consolidate the storage code within a TripleO
library, do we really need a *workflow* to call a single function? Is a
thin TripleO API that contains no additional business logic really so bad
at that point?
Actually, this is an argument for making the validation part of the
deployment a workflow - then the interface with the storage mechanism
becomes more easily pluggable vs baked into an opaque-to-operators API.
E.g, in the long term, imagine the capabilities feature exists in Heat, you
then have a pre-deployment workflow that looks something like:
1. Retrieve golden templates from a template store
2. Pass templates to Heat, get capabilities map which defines features user
must/may select.
3. Prompt user for input to select required capabilites
4. Pass user input to Heat, validate the configuration, get a mapping of
required options for the selected capabilities (nested validation)
5. Push the validated pieces ("plan" in TripleO API terminology) to a
template store
This is a pre-deployment validation workflow, and it's a superset of the
getDeploymentOptions feature you refer to.
Historically, TripleO has had a major gap wrt workflow, meaning that we've
always implemented it either via shell scripts (tripleo-incubator) or
python code (tripleo-common/tripleo-client, potentially TripleO API).
So I think what Dan is exploring is, how do we avoid reimplementing a
workflow engine, when a project exists which already does that.
My gut reaction is to say that proposing Mistral in place of a TripleO API
is to look at the engineering concerns from the wrong direction. The
Mistral alternative comes from a desire to limit custom TripleO code at all
costs. I think that is an extremely dangerous attitude that leads to
compromises and workarounds that will quickly lead to a shaky code base
full of design flaws that make it difficult to implement or extend any
functionality cleanly.
I think it's not about limiting TripleO code at all costs, it's about
learning from past mistakes, where long-term TripleO specific workarounds
for gaps in other projects have become serious technical debt.
For example, the old merge.py approach to template composition was a
workaround for missing heat features, then Tuskar was another workaround
(arguably) for missing heat features, and now we're again proposing a
long-term workaround for some missing heat features, some of which are
already proposed (referring to the API for capabilities resolution).
I think the correct attitude is to simply look at the problem we're
trying to solve and find the correct architecture. For these get/set
methods that the API needs, it's pretty simple: storage -> some logic ->
a REST API. Adding a workflow engine on top of that is unneeded, and I
believe that means it's an incorrect solution.
What may help is if we can work through the proposed API spec, and
identify which calls can reasonably be considered workflows vs those where
it's really just proxying an API call with some logic?
When we have a defined list of "not workflow" API requirements, it'll
probably be much easier to rationalize over the value of a bespoke API vs
mistral?
Steve
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
