On 12/07/14 06:41, Zane Bitter wrote: > On 11/07/14 09:37, Tomas Sedovic wrote: >> Hi all, >> >> This is a follow-up to Clint Byrum's suggestion to add the `Map` >> intrinsic function[0], Zane Bitter's response[1] and Randall Burt's >> addendum[2]. >> >> Sorry for bringing it up again, but I'd love to reach consensus on this. >> The summary of the previous conversation: > > Please keep bringing it up until you get a straight answer ;) > >> 1. TripleO is using some functionality currently not supported by Heat >> around scaled-out resources >> 2. Clint proposed a `map` intrinsic function that would solve it >> 3. Zane said Heat have historically been against a for-loop functionality >> 4. Randall suggested ResourceGroup's attribute passthrough may do what >> we need >> >> I've looked at the ResourceGroup code and experimented a bit. It does do >> some of what TripleO needs but not all. > > Many thanks for putting this together Tomas, this is exactly the kind of > information that is _incredibly_ helpful in knowing what sort of > features we need in HOT. Fantastic work :) > >> Here's what we're doing with our scaled-out resources (what we'd like to >> wrap in a ResourceGroup or similar in the future): >> >> >> 1. Building a coma-separated list of RabbitMQ nodes: >> >> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L642 >> >> >> This one is easy with ResourceGroup's inner attribute support: >> >> list_join: >> - ", " >> - {get_attr: [controller_group, name]} >> >> (controller_group is a ResourceGroup of Nova servers) >> >> >> 2. Get the name of the first Controller node: >> >> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L339 >> >> >> Possible today: >> >> {get_attr: [controller_group, resource.0.name]} >> >> >> 3. List of IP addresses of all controllers: >> >> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L405 >> >> >> We cannot do this, because resource group doesn't support extended >> attributes. >> >> Would need something like: >> >> {get_attr: [controller_group, networks, ctlplane, 0]} >> >> (ctlplane is the network controller_group servers are on) > > I was going to give an explanation of how we could implement this, but > then I realised a patch was going to be easier: > > https://review.openstack.org/#/c/106541/ > https://review.openstack.org/#/c/106542/
Thanks, that looks great. > >> 4. IP address of the first node in the resource group: >> >> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/swift-deploy.yaml#L29 >> >> >> Can't do: extended attributes are not supported for the n-th node for >> the group either. > > I believe this is possible today using: > > {get_attr: [controller_group, resource.0.networks, ctlplane, 0]} Yeah, I've missed this. I have actually checked the ResourceGroup's GetAtt method but didn't realise the connection with the GetAtt function so I hadn't tried it before. > >> This can be solved by `get_resource` working with resource IDs: >> >> get_attr: >> - {get_attr: [controller_group, resource.0]} >> - [networks, ctlplane, 0] >> >> (i.e. we get the server's ID from the ResourceGroup and change >> `get_attr` to work with the ID's too. Would also work if `get_resource` >> understood IDs). > > This is never going to happen. > > Think of get_resource as returning an object whose string representation > is the UUID of the named resource (get_attr is similar, but returning > attributes instead). It doesn't mean that having the UUID of a resource > is the same as having the resource itself; the UUID could have come from > anywhere. What you're talking about is a radical departure from the > existing, very simple but extremely effective, model toward something > that's extremely difficult to analyse with lots of nasty edge cases. > It's common for people to think they want this, but it always turns out > there's a better way to achieve their goal within the existing data model. Right, that makes sense. I don't think I fully grasped the existing model so this felt like a nice quick fix. > >> Alternatively, we could extend the ResourceGroup's get_attr behaviour: >> >> {get_attr: [controller_group, resource.0.networks.ctlplane.0]} >> >> but the former is a bit cleaner and more generic. > > I wrote a patch that implements this (and also handles (3) above in a > similar manner), but in the end I decided that this: > > {get_attr: [controller_group, resource.0, networks, ctlplane, 0]} > > would be better than either that or the current syntax (which was > obviously obscure enough that you didn't discover it). My only > reservation was that it might make things a little weird when we have an > autoscaling API to get attributes from compared with the dotted syntax > that you suggest, but I soon got over it ;) So now that I understand how this works, I'm not against keeping things the way we are. There is a consistency there, we just need to document it and perhaps show some examples. > >> --- >> >> >> That was the easy stuff, where we can get by with the current >> functionality (plus a few fixes). >> >> What follows are examples that really need new intrinsic functions (or >> seriously complicating the ResourceGroup attribute code and syntax). >> >> >> 5. Building a list of {ip: ..., name: ...} dictionaries to configure >> haproxy: >> >> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L478 >> >> >> This really calls for a mapping/for-each kind of functionality. Trying >> to invent a ResourceGroup syntax for this would be perverse. >> >> Here's what it could look like under Clint's `map` proposal: >> >> map: >> - ip: {get_attr: [{get_resource: "$1"}, networks, ctlplane, 0] >> name: {get_attr: [{get_resource: "$1"}, name]} >> - {get_attr: [compute_group, refs]} > > This has always been the tricky one :D > > IMHO the real problem here is that we're trying to collate the data at > the point where it is consumed, not the point where it is produced. It's > not like we were just given a big blob of data and now have to somehow > extract the useful parts; we produced it ourselves by combining data > from the scaled units. If we didn't get the right data, we have only > ourselves to blame ;) > > So if the provider template that defines e.g. a compute node contains > the section: > > outputs: > host_entry: > value: > ip: {get_attr: [compute_server, networks, ctlplane, 0]} > name: {get_attr: [compute_server, name]} > > Then in your main template all you need to do is: > > {getattr: [compute_group, host_entry]} Oh this is absolutely wonderful. I've had all the pieces in my head but I didn't make the connection. You're completely right about using a provider template here anyway -- that's what I planned to do, but I didn't fully appreciate the connection between outputs and attributes (even though I knew about it). > > to get the list of {ip: ..., name: ...} dicts. (As a bonus, this is > about as straightforward to read as I can imagine it ever getting.) > > Note that you *will* want to be using a provider template as the scaled > unit _anyway_, because each compute_server will have associated software > deployments and quite possibly a bunch of other resources that need to > be in the scaled unit. Yep, exactly. > > There is one aspect of this that probably doesn't work yet: originally > outputs and attributes were only allowed to be strings. We changed that > for attributes, but probably not yet for outputs (even though outputs of > provider templates become attributes of the facade resource). But that > should be easy to fix. (And if your data can be returned as a string, it > should already work.) Unless I misunderstood what you're saying, it seems to be working now: controller.yaml: outputs: hosts_entry: description: An IP address and a hostname of the server value: ip: {get_attr: [controller_server, networks, private, 0]} name: {get_attr: [controller_server, name]} environment.yaml: resource_registry: OS::TripleO::Controller: controller.yaml test-resource-group.yaml: resources: servers: type: OS::Heat::ResourceGroup properties: count: 3 resource_def: type: OS::TripleO::Controller properties: key_name: {get_param: key_name} image: {get_param: image_id} outputs: hosts: description: "/etc/hosts entries for each server" value: {get_attr: [servers, hosts_entry]} Heat stack-show test-resource-group: { "output_value": [ "{u'ip': u'10.0.0.4', u'name': u'rg-7heh-0-tweejsvubaht-controller_server-mscy33sbtirn'}", "{u'ip': u'10.0.0.3', u'name': u'rg-7heh-1-o4szl7lry27d-controller_server-sxpkalgi27ii'}", "{u'ip': u'10.0.0.2', u'name': u'rg-7heh-2-l2y6rqxml2fi-controller_server-u4jcjacjdrea'}" ], "description": "/etc/hosts entries for each server", "output_key": "hosts" }, > >> (this relies on `get_resource` working with resource IDs. Alternatively, > > That's a show-stopper right out of the gate. If we implemented the map > thing you would have to use something like: > > map: > - ip:$1 > name: $2 > - {get_attr: [compute_group, networks, ctlplane, 0]} > - {get_attr: [compute_group, name]} > > (which is IMHO about 50 times easier to read anyway.) > >> we could have a `resources` attribute for ResourceGroup that returns >> objects that can be used with get_attr. > > Sounds increasingly hacky. > >> 6. Building the /etc/hosts file >> >> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L585 >> >> >> Same as above, but also joining two lists together. >> >> We can use nested {list_join: ["\n", [...]} just as we're doing now, but >> having a `concat_list` function would make this and some other cases >> shorter and clearer. > > I'm not strongly opposed to this one. We were trying to keep the number > of functions in HOT as small as possible... and it does look like it is > possible to do what you need already... so I would treat this as a low > priority. Agreed. > >> 7. Building the list of Swift devices: >> >> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/swift-deploy.yaml#L23 >> >> >> In addition to the abowe, we're adding a single element at the beginning >> of a list. >> >> Asking for a `cons` support is pushing it, right? ;-) > > lol, Greenspun's Tenth Law in action right here :D I *have* been saying we should switch to s-expressions all along O:-). > >> We could just wrap that in a list and use `concat_list` or keep using >> nested `list_join`s as in the /etc/hosts case. > > +1 > >> ---- >> >> >> So this boils down to 4 features proposals: >> >> 1. Support extended attributes in ResourceGroup's members > > Sorted. Yep > >> 2. Allow a way to use a Resource ID (e.g. what you get by {get_attr: >> [ResourceGroup, refs]} or {get_attr: [ResourceGroup, resource.0]}) with >> existing intrinsic functions (get_resource, get_attr) > > No dice, but (1) solves the problem anyway. Agreed > >> 3. A `map` intrinsic function that turns a list of items to another list >> by doing operations on each item > > There may be a better solution available to us already, so IMO > investigate that first. If that turns out not to be the case then we'll > need to reach a consensus on whether map is something we want. You're right. I no longer think map (or anything like it) is necessary. > >> 4. A `concat_list` intrinsic function that joins multiple lists into one. > > Low priority. Yeah. > >> I think the first two are not controversial. What about the other two? >> I've shown you some examples where we would find a good use in the >> TripleO templates. The lack of `map` actually blocks us from going >> all-Heat. > > Hopefully that's not actually the case. > >> The alternative would be to say that this sort of stuff to be done >> inside the instance by os-apply-config et al. It would complicate things >> for TripleO, but oh well. > > It seems to me that the alternative is not necessarily to modify > os-apply-config, but rather to provide a software config with a script > that converts the data from whatever format Heat can supply to whatever > format is needed by the application. Although I don't think it's even > required in the specific case you mention, I find that a much better > answer for the general case than turning the template format into the > JSON equivalent of XSLT ;) That's what I thought -- not rewriting os-apply-config but simply passing the raw data to it and changing the tripleo image elements to output the configuration format required by the applications. +1 on the XSLT sentiment, I just didn't realise we pretty much solve everything with the existing model (hat off to the design btw, I love how it works together -- we just maybe need to communicate it a bit more). > >> Either way, can we come to a clear answer? I would love to convert our >> templates to 100% Natural Heat during Juno and I'm willing to put my >> time towards coding the necessary features. > > +1 that is a worthy goal and we appreciate your contributions toward > reaching it. Thanks so much for your answers, Zane. They were most helpful. > > cheers, > Zane. > >> Tomas >> >> >> [0]: >> http://lists.openstack.org/pipermail/openstack-dev/2014-February/027536.html >> >> [1]: >> http://lists.openstack.org/pipermail/openstack-dev/2014-April/031944.html >> [2]: >> http://lists.openstack.org/pipermail/openstack-dev/2014-April/032074.html >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
