Tagging kolla so the kolla community also sees it. Pardon the top posting. -Ryan
----- Original Message ----- From: "Dan Prince" <dpri...@redhat.com> To: "openstack-dev" <openstack-dev@lists.openstack.org> Sent: Wednesday, August 5, 2015 2:29:13 PM Subject: [openstack-dev] [TripleO] [Puppet] Deploying OpenStack with Puppet modules on Docker with Heat Hi, There is a lot of interest in getting support for container based deployment within TripleO and many different ideas and opinions on how to go about doing that. One idea on the table is to use Heat to help orchestrate the deployment of docker containers. This would work similar to our tripleo-heat -templates implementation except that when using docker you would swap in a nested stack template that would configure containers on baremetal. We've even got a nice example that shows what a containerized TripleO overcloud might look like here [1]. The approach outlines how you might use kolla docker containers alongside of the tripleo-heat-templates to do this sort of deployment. This is all cool stuff but one area of concern is how we do the actual configuration of the containers. The above implementation relies on passing environment variables into kolla built docker containers which then self configure all the required config files and start the service. This sounds like a start... but creating (and maintaining) another from scratch OpenStack configuration tool isn't high on my list of things to spend time on. Sure there is already a kolla community helping to build and maintain this configuration tooling (mostly thinking config files here) but this sounds a bit like what tripleo -image-elements initially tried to do and it turns out there are much more capable configuration tools out there. Since we are already using a good bit of Puppet in tripleo-heat -templates the idea came up that we would try to configure Docker containers using Puppet. Again, here there are several ideas in the Puppet community with regards to how docker might best be configured with Puppet. Keeping those in mind we've been throwing some ideas out on an etherpad here [2] that describes using Heat for orchestration, Puppet for configuration, and Kolla docker images for containers. A quick outline of the approach is: -Extend the heat-container-agent [3] that runs os-collect-config and all the required hooks we require for deployment. This includes docker -compute, bash scripts, and Puppet. NOTE: As described in the etherpad I've taken to using DIB to build this container. I found this to be faster from a TripleO development baseline. -To create config files the heat-container-agent would run a puppet manifest for a given role and generate a directory tree of config files (/var/lib/etc-data for example). -We then run a docker-compose software deployment that mounts those configuration file(s) into a read only volume and uses them to start the containerized service. The approach could look something like this [4]. This nice thing about this is that it requires no modification to OpenStack Puppet modules. We can use those today, as-is. Additionally, although Puppet runs in the agent container we've created a mechanism to set all the resources to noop mode except for those that generate config files. And lastly, we can use exactly the same role manifest for docker that we do for baremetal. Lots of re-use here... and although we are disabling a lot of Puppet functionality in setting all the non-config resources to noop the Kolla containers already do some of that stuff for us (starting services, etc.). ---- All that said (and trying to keep this short) we've still got a bit of work to do around wiring up externally created config files to kolla build docker containers. A couple of issues are: -The external config file mechanism for Kolla containers only seems to support a single config file. Some services (Neutron) can have multiple files. Could we extend the external config support to use multiple files? -If a service has multiple files kolla may need to adjust its service startup script to use multiple files. Perhaps a conf.d approach would work here? -We are missing published version of some key kolla containers. Namely openvswitch and the neutron-openvswitch-agent for starters but I'd also like to have a Ceilometer agent and SNMP agent container as well so we have feature parity with the non-docker compute role. Once we have solutions for the above I think we'll be very close to a fully dockerized compute role with TripleO heat templates. From there we can expand the idea to cover other roles within the tripleo-heat -templates too. I'll stop there for now. Any ideas and thoughts appreciated. Dan ----- [1] https://review.openstack.org/#/c/178840/ (Containerized TripleO Overcloud.) [2] https://etherpad.openstack.org/p/tripleo-docker-puppet [3] http://git.openstack.org/cgit/openstack/heat -templates/log/hot/software-config/heat-container-agent [4] https://review.openstack.org/#/c/209505/ (Docker compute role configured via Puppet) __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
