We do support some upstream charts but we started mariadb/rabbit before some of the upstream charts were written, so we duplicate a little bit of functionality at the moment. You can mix and match though. If an upstream chart doesn't work with kolla-kubernetes, I consider that a bug we should fix. Likewise, you should be able to run noncontainerized stuff mixed in too. If it doesn't work, its likewise a bug. You should be able to run kolla-kubernetes with a baremetal db.
Some known working stuff: prometheus/grafana upstream charts start collecting data from the containers as soon as they are launched. I have also tested a bit with the upstream fluent-bit chart and have a ps in the works to make it work much better. Thanks, Kevin ________________________________________ From: Emilien Macchi [emil...@redhat.com] Sent: Monday, July 17, 2017 10:13 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [TripleO] Let's use Ansible to deploy OpenStack services on Kubernetes On Mon, Jul 17, 2017 at 5:32 AM, Flavio Percoco <fla...@redhat.com> wrote: > On 14/07/17 08:08 -0700, Emilien Macchi wrote: >> >> On Fri, Jul 14, 2017 at 2:17 AM, Flavio Percoco <fla...@redhat.com> wrote: >>> >>> >>> Greetings, >>> >>> As some of you know, I've been working on the second phase of TripleO's >>> containerization effort. This phase if about migrating the docker based >>> deployment onto Kubernetes. >>> >>> These phase requires work on several areas: Kubernetes deployment, >>> OpenStack >>> deployment on Kubernetes, configuration management, etc. While I've been >>> diving >>> into all of these areas, this email is about the second point, OpenStack >>> deployment on Kubernetes. >>> >>> There are several tools we could use for this task. kolla-kubernetes, >>> openstack-helm, ansible roles, among others. I've looked into these tools >>> and >>> I've come to the conclusion that TripleO would be better of by having >>> ansible >>> roles that would allow for deploying OpenStack services on Kubernetes. >>> >>> The existing solutions in the OpenStack community require using Helm. >>> While >>> I >>> like Helm and both, kolla-kubernetes and openstack-helm OpenStack >>> projects, >>> I >>> believe using any of them would add an extra layer of complexity to >>> TripleO, >>> which is something the team has been fighting for years years - >>> especially >>> now >>> that the snowball is being chopped off. >>> >>> Adopting any of the existing projects in the OpenStack communty would >>> require >>> TripleO to also write the logic to manage those projects. For example, in >>> the >>> case of openstack-helm, the TripleO team would have to write either >>> ansible >>> roles or heat templates to manage - install, remove, upgrade - the charts >>> (I'm >>> happy to discuss this point further but I'm keepping it at a high-level >>> on >>> purpose for the sake of not writing a 10k-words-long email). >>> >>> James Slagle sent an email[0], a couple of days ago, to form TripleO >>> plans >>> around ansible. One take-away from this thread is that TripleO is >>> adopting >>> ansible more and more, which is great and it fits perfectly with the >>> conclusion >>> I reached. >>> >>> Now, what this work means is that we would have to write an ansible role >>> for >>> each service that will deploy the service on a Kubernetes cluster. >>> Ideally >>> these >>> roles will also generate the configuration files (removing the need of >>> puppet >>> entirely) and they would manage the lifecycle. The roles would be >>> isolated >>> and >>> this will reduce the need of TripleO Heat templates. Doing this would >>> give >>> TripleO full control on the deployment process too. >>> >>> In addition, we could also write Ansible Playbook Bundles to contain >>> these >>> roles >>> and run them using the existing docker-cmd implementation that is coming >>> out >>> in >>> Pike (you can find a PoC/example of this in this repo[1]). >>> >>> Now, I do realize the amount of work this implies and that this is my >>> opinion/conclusion. I'm sending this email out to kick-off the discussion >>> and >>> gather thoughts and opinions from the rest of the community. >>> >>> Finally, what I really like about writing pure ansible roles is that >>> ansible >>> is >>> a known, powerfull, tool that has been adopted by many operators already. >>> It'll >>> provide the flexibility needed and, if structured correctly, it'll allow >>> for >>> operators (and other teams) to just use the parts they need/want without >>> depending on the full-stack. I like the idea of being able to separate >>> concerns >>> in the deployment workflow and the idea of making it simple for users of >>> TripleO >>> to do the same at runtime. Unfortunately, going down this road means that >>> my >>> hope of creating a field where we could collaborate even more with other >>> deployment tools will be a bit limited but I'm confident the result would >>> also >>> be useful for others and that we all will benefit from it... My hopes >>> might >>> be a >>> bit naive *shrugs* >> >> >> Of course I'm biased since I've been (a little) involved in that work >> but I like the idea of : >> >> - Moving forward with our containerization. docker-cmd will help us >> for sure for this transition (I insist on the fact TripleO is a >> product that you can upgrade and we try to make it smooth for our >> operators), so we can't just trash everything and switch to a new >> tool. I think the approach that we're taking is great and made of baby >> steps where we try to solve different problems. >> - Using more Ansible - the right way - when it makes sense : with the >> TripleO containerization, we only use Puppet for Configuration >> Management, managing a few resources but not for orchestration (or not >> all the features that Puppet provide) and for Data Binding (Hiera). To >> me, it doesn't make sense for us to keep investing much in Puppet >> modules if we go k8s & Ansible. That said, see the next point. >> - Having a transition path between TripleO with Puppet and TripleO >> with apbs and have some sort of binding between previous hieradata >> generated by TripleO & a similar data binding within Ansible playbooks >> would help. I saw your PoC Flavio, I found it great and I think we >> should make >> https://github.com/tripleo-apb/ansible-role-k8s-keystone/blob/331f405bd3f7ad346d99e964538b5b27447a0ebf/provision-keystone-apb/tasks/hiera.yaml >> optional when running apbs, and allow to provide another format (more >> Ansiblish) to let folks not using TripleO to use it. We also should >> target this new format and switch service by service in TripleO to use >> this new format, as long as apbs support both. I think that way we can >> step by step migrate to use Ansible for configuration management. >> >> There are some things to figure out: >> - We kind of found out solutions for OpenStack services - great - now >> what do we do for services like MySQL, Apache, etc. Do we have >> "standard" and "community-supported" apbs? Do we need to create some? > > > The question should be whether we have community maintained roles that > deploy > third-party services on k8s. I'm sorry for nitpicking but I just want to > make > sure we all keep in mind that the APB wrap is optional (although convenient > for > us). > > The answer is not that I'm aware of. There are roles to deploy some of these > services on baremetal. *If* we're going to use Helm, I found the answer of my own question: https://github.com/kubernetes/charts/tree/master/stable/mariadb (though it's missing Galera support AFIK). And some other services that we currently use in TripleO: https://github.com/kubernetes/charts/tree/master/stable I tried to find if whether or not kolla-kubernetes was using these Helms, and unless I fail to find them I think they don't. I would be curious to know the reason if that's the case. (Please correct me with links). >> - Where the apbs should live? IMO in OpenStack and IMO not in big tent >> for now, under no umbrella. > > > Again, roles! > > I feel like this question is forcing us to get a bit ahead of ourselves but, > for > the sake of discussion, I think they could just exist in the openstack org > under > no team for now. But the answer really depends on where the discussion about > hosted repos goes. > > There are different ways we could structure this and I would be lying if I > said > I haven't thought about it. We can talk about this point when the right time > comes. I don't see ourselves creating repos under openstack just yet. > >> - Since we use Puppet modules which don't only make configuration >> management but also some orchestration (like creating keystone >> endpoints, etc) - where should we put this logic? +1 for apbs using >> clean Ansible code (and not bash templating). > > > Yeah, this logic would go into the ansible roles. > >> - How we can help our vendors to whom we asked them to write Puppet >> modules to deploy their software (Contrail, Nuage, etc). We still >> might need some sort or "running puppet from ansible" for some >> software we wouldn't have apbs as quickly as we would need. > > > Yeah, this would be one of the hard problems to solve. Running puppet from > ansible is possible and I know openstack-infra does that. This is one option > and > probably the one we would have to support in the transition period. > >> I hope I didn't divert the discussion but here's my feedback and food >> for thoughts. > > > Not at all, thanks for throwing all these questions out there. These are > valid > concerns and we'll have to address them. It's better to document them now. > > > Flavio > > -- > @flaper87 > Flavio Percoco > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
