On Wed, 10 Oct 2018 at 08:08, Florian Engelmann < florian.engelm...@everyware.ch> wrote:
> Am 10/9/18 um 1:47 PM schrieb Mark Goddard: > > > > > > On Tue, 9 Oct 2018 at 12:03, Florian Engelmann > > <florian.engelm...@everyware.ch <mailto:florian.engelm...@everyware.ch>> > > > wrote: > > > > Am 10/9/18 um 11:04 AM schrieb Mark Goddard: > > > Thanks for these suggestions Florian, there are some interesting > > ideas > > > in here. I'm a little concerned about the maintenance overhead of > > adding > > > support for all of these things, and wonder if some of them could > be > > > done without explicit support in kolla and kolla-ansible. The > kolla > > > projects have been able to move quickly by providing a flexible > > > configuration mechanism that avoids the need to maintain support > for > > > every OpenStack feature. Other thoughts inline. > > > > > > > I do understand your apprehensions Mark. For some of the suggested > > changes/additions I agree. But adding those components without > > kolla/kolla-ansible integration feels not right. > > > > > > I'm not entirely against adding some of these things, if enough people > > in the community want them. I'd just like to make sure that if they can > > be done in a sane way without changes, then we do that and document how > > to do it instead. > > > > Yes I agree and that is a very important role/job. > > > > > > > > > On Mon, 8 Oct 2018 at 11:15, Florian Engelmann > > > <florian.engelm...@everyware.ch > > <mailto:florian.engelm...@everyware.ch> > > <mailto:florian.engelm...@everyware.ch > > <mailto:florian.engelm...@everyware.ch>>> > > > wrote: > > > > > > Hi, > > > > > > I would like to start a discussion about some changes and > > additions I > > > would like to see in in kolla and kolla-ansible. > > > > > > 1. Keepalived is a problem in layer3 spine leaf networks as > any > > > floating > > > IP can only exist in one leaf (and VRRP is a problem in > > layer3). I > > > would > > > like to use consul and registrar to get rid of the "internal" > > floating > > > IP and use consuls DNS service discovery to connect all > > services with > > > each other. > > > > > > > > > Without reading up, I'm not sure exactly how this fits together. > If > > > kolla-ansible made the API host configurable for each service > rather > > > than globally, would that be a step in the right direction? > > > > No that would not help. The problem is HA. Right now there is a > > "central" floating IP (kolla_internal_vip_address) that is used for > all > > services to connect to (each other). Keepalived is failing that IP > over > > if the "active" host fails. In a layer3 (CLOS/Spine-Leaf) network > this > > IP is only available in one leaf/rack. So that rack is a "SPOF". > > Using service discovery fits perfect in a CLOS network and scales > much > > better as a HA solution. > > > > Right, but what I'm saying as a thought experiment is, if we gave you > > the required variables in kolla-ansible (e.g. nova_internal_fqdn) to > > make this possible with an externally managed consul service, could that > > work? > > Now I get you. I would say all configuration templates need to be > changed to allow, eg. > > $ grep http /etc/kolla/cinder-volume/cinder.conf > glance_api_servers = http://10.10.10.5:9292 > auth_url = http://internal.somedomain.tld:35357 > www_authenticate_uri = http://internal.somedomain.tld:5000 > auth_url = http://internal.somedomain.tld:35357 > auth_endpoint = http://internal.somedomain.tld:5000 > > to look like: > > glance_api_servers = http://glance.service.somedomain.consul:9292 > auth_url = http://keystone.service.somedomain.consul:35357 > www_authenticate_uri = http://keystone.service.somedomain.consul:5000 > auth_url = http://keystone.service.somedomain.consul:35357 > auth_endpoint = http://keystone.service.somedomain.consul:5000 Those values are generally formed using the internal FQDN (kolla_internal_fqdn), that is supposed to resolve to the internal VIP. If we had something like this in group_vars/all.yml, we could make the host configurable on a per-service basis, while still retaining the default. This is a common pattern in kolla-ansible. For example: nova_external_fqdn: "{{ kolla_external_fqdn }}" nova_internal_fqdn: "{{ kolla_internal_fqdn }}" > > > > > > > > > > > 2. Using "ports" for external API (endpoint) access is a > > major headache > > > if a firewall is involved. I would like to configure the > > HAProxy (or > > > fabio) for the external access to use "Host:" like, eg. "Host: > > > keystone.somedomain.tld", "Host: nova.somedomain.tld", ... > > with HTTPS. > > > Any customer would just need HTTPS access and not have to > > open all > > > those > > > ports in his firewall. For some enterprise customers it is > > not possible > > > to request FW changes like that. > > > > > > 3. HAProxy is not capable to handle "read/write" split with > > Galera. I > > > would like to introduce ProxySQL to be able to scale Galera. > > > > > > > > > It's now possible to use an external database server with > > kolla-ansible, > > > instead of deploying a mariadb/galera cluster. This could be > > implemented > > > how you like, see > > > > > > https://docs.openstack.org/kolla-ansible/latest/reference/external-mariadb-guide.html > . > > > > Yes I agree. And this is what we will do in our first production > > deployment. But I would love to see ProxySQL in Kolla as well. > > As a side note: Kolla-ansible does use: > > > > option mysql-check user haproxy post-41 > > > > to check Galera, but that check does not fail if the node is out of > > sync > > with the other nodes! > > > > > http://galeracluster.com/documentation-webpages/monitoringthecluster.html > > > > That's good to know. Could you raise a bug in kolla-ansible on > > launchpad, and offer advice on how to improve this check if you have any? > > > > done: https://bugs.launchpad.net/kolla-ansible/+bug/1796930 > > Thanks > > > > > > > > 4. HAProxy is fine but fabio integrates well with consul, > > statsd and > > > could be connected to a vault cluster to manage secure > > certificate > > > access. > > > > > > As above. > > > > > > 5. I would like to add vault as Barbican backend. > > > > > > Does this need explicit support in kolla and kolla-ansible, or > > could it > > > be done through configuration of barbican.conf? Are there > additional > > > packages required in the barbican container? If so, see > > > > > > https://docs.openstack.org/kolla/latest/admin/image-building.html#package-customisation > . > > > > True but the vault (and consul) containers could be deployed and > > managed > > by kolla-ansible. > > > > I'd like to see if anyone else is interested in this. Kolla ansible > > already deploys a large number of services, which is great. As with many > > other projects I'm seeing the resources of core contributors fall off a > > little, and I think we need to consider how to ensure the project is > > maintainable long term. In my view a good way of doing that is to enable > > integration with existing services, rather than deploying them. We need > > to decide where the line is as a community. We have an IRC meeting at > > 3pm UTC if you'd like to bring it up then. > > > Sorry I didn't manage to attend the IRC meeting. But again, I aggree, it > would be great to extend the (already great) felxibility of > kolla-ansible to be "add" "external" services. > > Sorry, I missed a key word here - tomorrow. It is now today, i.e. Wednesdays at 3pm UTC. > > > > > > > > > 6. I would like to add an option to enable tokenless > > authentication for > > > all services with each other to get rid of all the openstack > > service > > > passwords (security issue). > > > > > > Again, could this be done without explicit support? > > > > We did not investigate here. Changes to the apache configuration are > > needed. I guess we will have to change the kolla container itself to > do > > so? Is it possible to "inject" files in a container using > kolla-ansible? > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > < > http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
