Christophe, Thank you, we really appreciate you looking into this, and I will try to help you as much as I can, because we really need to have this software working, soon.
So here's something that, to me, is very telling # printenv |grep OS_CACERT OS_CACERT=/etc/openldap/cacerts/gpcprod_root_ca.pem ^^^ here you can see that my self-signed CA cert is loaded into my environment, having sourced my openrc file Now I'm going to invoke the cloudkitty client with debug, and grep for 'curl' to see what it's actually doing: # openstack --debug rating info-config-get 2>&1 |grep -b1 curl 9774-Get auth_ref 9787:REQ: curl -g -i --cacert "/etc/openldap/cacerts/gpcprod_root_ca.pem" -X GET https://keystone.gpcprod:5000/v3 -H "Accept: application/json" -H "User-Agent: osc-lib/1.9.0 keystoneauth1/3.4.0 python-requests/2.14.2 CPython/2.7.5" 10014-Starting new HTTPS connection (1): keystone.gpcprod -- 16319-run(Namespace()) 16336:REQ: curl -g -i -X GET https://keystone.gpcprod:5000/v3 -H "Accept: application/json" -H "User-Agent: python-keystoneclient" 16461-Starting new HTTPS connection (1): keystone.gpcprod ^^^ you can see that the first time, it correctly forms the curl, and that works fine. But the second time (and the User-Agent has changed), it never even passes the --cacert option to curl at all. The results then are predictable: Starting new HTTPS connection (1): keystone.gpcprod SSL exception connecting to https://keystone.gpcprod:5000/v3: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python2.7/site-packages/cloudkittyclient/v1/shell_cli.py", line 78, in take_action shell.do_info_config_get(ckclient, parsed_args) File "/usr/lib/python2.7/site-packages/cloudkittyclient/v1/shell.py", line 93, in do_info_config_get utils.print_dict(cc.config.get_config(), dict_property="Section") File "/usr/lib/python2.7/site-packages/cloudkittyclient/v1/core.py", line 88, in get_config out = self.api.get(self.base_url).json() File "/usr/lib/python2.7/site-packages/cloudkittyclient/apiclient/client.py", line 359, in get return self.client_request("GET", url, **kwargs) File "/usr/lib/python2.7/site-packages/cloudkittyclient/apiclient/client.py", line 349, in client_request self, method, url, **kwargs) File "/usr/lib/python2.7/site-packages/cloudkittyclient/apiclient/client.py", line 248, in client_request self.authenticate() File "/usr/lib/python2.7/site-packages/cloudkittyclient/apiclient/client.py", line 319, in authenticate self.auth_plugin.authenticate(self) File "/usr/lib/python2.7/site-packages/cloudkittyclient/apiclient/auth.py", line 201, in authenticate self._do_authenticate(http_client) File "/usr/lib/python2.7/site-packages/cloudkittyclient/client.py", line 191, in _do_authenticate ks_session = _get_keystone_session(**ks_kwargs) File "/usr/lib/python2.7/site-packages/cloudkittyclient/client.py", line 87, in _get_keystone_session v2_auth_url, v3_auth_url = _discover_auth_versions(ks_session, auth_url) File "/usr/lib/python2.7/site-packages/cloudkittyclient/client.py", line 38, in _discover_auth_versions ks_discover = discover.Discover(session=session, auth_url=auth_url) File "/usr/lib/python2.7/site-packages/keystoneclient/discover.py", line 178, in __init__ authenticated=authenticated) File "/usr/lib/python2.7/site-packages/keystoneclient/_discover.py", line 143, in __init__ authenticated=authenticated) File "/usr/lib/python2.7/site-packages/keystoneclient/_discover.py", line 38, in get_version_data resp = session.get(url, headers=headers, authenticated=authenticated) File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 535, in get return self.request(url, 'GET', **kwargs) File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 428, in request resp = send(**kwargs) File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 466, in _send_request raise exceptions.SSLError(msg) SSLError: SSL exception connecting to https://keystone.gpcprod:5000/v3: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) clean_up CliInfoGetConfig: SSL exception connecting to https://keystone.gpcprod:5000/v3: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) Jonathan On Tue, Sep 4, 2018 at 5:50 AM Christophe Sauthier < [email protected]> wrote: > Hello > > Thanks for those elements. > > It is really surprising because as you can imagine this is something we > set up many times... > I'll take care to set up the same environment than you and I'll let you > know if I am facing the same issues... I am trying to do that quickly... > > Regards > > Christophe > > ---- > Christophe Sauthier > CEO > > Objectif Libre : Au service de votre Cloud > > +33 (0) 6 16 98 63 96 | [email protected] > > https://www.objectif-libre.com | @objectiflibre > Recevez la Pause Cloud Et DevOps : https://olib.re/abo-pause > > Le 2018-08-31 23:40, [email protected] a écrit : > > On Fri, 2018-08-31 at 23:20 +0200, Christophe Sauthier wrote: > >> Hello Jonathan > >> > >> Can you describe a little more your setup (release/method of > >> installation/linux distribution) /issues that you are facing ? > > > > > > It is OpenStack Queens, on CentOS 7.5, using the packages from the > > centos-cloud repo (which I suppose is the same is RDO). > > > > # uname -msr > > Linux 3.10.0-862.3.2.el7.x86_64 x86_64 > > > > # rpm -qa |grep cloudkitty |sort > > openstack-cloudkitty-api-7.0.0-1.el7.noarch > > openstack-cloudkitty-common-7.0.0-1.el7.noarch > > openstack-cloudkitty-processor-7.0.0-1.el7.noarch > > openstack-cloudkitty-ui-7.0.0-1.el7.noarch > > python2-cloudkittyclient-1.2.0-1.el7.noarch > > > > It is 'deployed' with custom puppet code only. I follow exactly the > > installation guides posted here: > > https://docs.openstack.org/cloudkitty/queens/index.html > > > > I'd prefer not to post full config files, but my [keystone_authtoken] > > section of cloudkitty.conf is identical (aside from service > > credentials) to the ones found in my glance, nova, cinder, neutron, > > gnocchi, ceilometer, etc, all of those services are working perfectly. > > > > > > My processor.log file is full of > > > > 2018-08-31 16:38:04.086 30471 WARNING cloudkitty.orchestrator [-] > > Error > > while collecting service network.floating: SSL exception connecting to > > https://keystone.gpcprod:5000/v3/auth/tokens: ("bad handshake: > > Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate > > verify failed')],)",): SSLError: SSL exception connecting to > > https://keystone.gpcprod:5000/v3/auth/tokens: ("bad handshake: > > Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate > > verify failed')],)",) > > 2018-08-31 16:38:04.094 30471 WARNING cloudkitty.orchestrator [-] > > Error > > while collecting service image: SSL exception connecting to > > https://keystone.gpcprod:5000/v3/auth/tokens: ("bad handshake: > > Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate > > verify failed')],)",): SSLError: SSL exception connecting to > > https://keystone.gpcprod:5000/v3/auth/tokens: ("bad handshake: > > Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate > > verify failed')],)",) > > > > and so on > > > > > > But, I mean, there's other little things too. I can see from running > > > > 'openstack --debug rating info-config-get' > > > > that it never even loads the cacert from my env, so it fails talking > > to > > keystone trying to get a token; the request never even gets to the > > cloudkitty api endpoint. > > > > > > > >> > >> Because we have deployed it/used it many times with SSL without > >> issue... > >> > >> It could be great also that you step up on #cloudkitty to discuss it. > >> > >> Christophe > >> > >> ---- > >> Christophe Sauthier > >> CEO > >> > >> Objectif Libre : Au service de votre Cloud > >> > >> +33 (0) 6 16 98 63 96 | [email protected] > >> > >> https://www.objectif-libre.com | @objectiflibre > >> Recevez la Pause Cloud Et DevOps : https://olib.re/abo-pause > >> > >> Le 2018-08-31 23:15, [email protected] a écrit : > >>> Anyone out there have Cloudkitty successfully working with SSL? By > >>> which I mean that Cloudkitty is able to talk to keystone over https > >>> without cert errors, and also talk to SSL'd rabbitmq? Oh, and the > >>> client tools also? > >>> > >>> Asking for a friend... > >>> > >>> > >>> > >>> Jonathan > >>> > >>> > >>> _______________________________________________ > >>> OpenStack-operators mailing list > >>> [email protected] > >>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators >
_______________________________________________ OpenStack-operators mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
