On 02/04/2018 10:44 PM, Qiming Teng wrote: > Starting about 24 hours ago, we have been notified CI gate failure > although we haven't changed anything to our project locally. Before that > we have spent quite some time making the out-of-tree tempest plugins > work on gate. > > After checking the log again and again ... we found the following logs > from Keystone: > > Feb 05 03:31:12.609492 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-dfcbf106-fbf5-41bd-9012-3c65d1de5f9a None admin] Could not find > project: service.: ProjectNotFound: Could not find project: service. > > Feb 05 03:31:13.845694 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-50feed46-7c15-425d-bec7-1b4a7ccf6859 None admin] Could not find > service: clustering.: ServiceNotFound: Could not find service: > clustering. > > Feb 05 03:31:12.552647 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-0a5e660f-dad6-4779-aea4-dd6969c728e6 None admin] Could not find > domain: Default.: DomainNotFound: Could not find domain: Default. > > Feb 05 03:31:12.441128 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-7eb9ed90-28fc-40aa-8a41-d560f7a156c9 None admin] Could not find > user: senlin.: UserNotFound: Could not find user: senlin. > > Feb 05 03:31:12.336572 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-19e52d02-5471-49a2-8acd-360199d8c6e0 None admin] Could not find > role: admin.: RoleNotFound: Could not find role: admin. > > Feb 05 03:28:33.797665 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-544cd822-18a4-4f7b-913d-297716418239 None admin] Could not find > user: glance.: UserNotFound: Could not find user: glance. > > Feb 05 03:28:29.993214 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING py.warnings [None > req-dc411d9c-6ab9-44e3-9afb-20e5e7034f12 None admin] > /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:865: > UserWarning: Policy identity:create_endpoint failed scope check. The > token used to make the request was project scoped but the policy > requires ['system'] scope. This behavior may change in the future where > using the intended scope is required > > Feb 05 03:28:29.920892 ubuntu-xenial-ovh-gra1-0002362092 > devstack@keystone.service[24845]: WARNING keystone.common.wsgi [None > req-32a4a378-d6d3-411e-9842-2178e577af27 None admin] Could not find > service: compute.: ServiceNotFound: Could not find service: compute. These are all WARNINGS messages. If this is a tempest run, these are probably from negative testing [0], in which case keystone is doing the correct thing. The warnings you've pasted are also present in successful tempest runs [1]. Can you provide a link to a patch that's failing? What project do you work on?
[0] https://github.com/openstack/tempest/blob/master/tempest/api/identity/admin/v3/test_projects_negative.py [1] http://logs.openstack.org/57/540557/2/check/tempest-full/bbd7cdd/controller/logs/screen-keystone.txt.gz?level=WARNING > > .... > > ---------------------- > > So I'm wondering what the hack happened? Keystone version bump? > Devstack changed? Tempest settings changed? > Why are we merging these changes near the end of a cycle when people are > focusing on stabilizing things? The original feature freeze date was 10 days ago [2] and with the condition of the gate during that time, there were several projects trailing with merging feature. Keystone was one of them and we issued feature freeze exceptions for those efforts [3] [4] [5]. Based on the warnings you've reported, I'm not convinced any of those efforts are affecting CI in a negative way, especially since we're still getting support into tempest to test those features. [2] https://releases.openstack.org/queens/schedule.html#q-keystone-ffreeze [3] http://lists.openstack.org/pipermail/openstack-dev/2018-January/126587.html [4] http://lists.openstack.org/pipermail/openstack-dev/2018-January/126588.html [5] http://lists.openstack.org/pipermail/openstack-dev/2018-January/126589.html > Any hints on these are highly appreciated. > > - Qiming > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
signature.asc
Description: OpenPGP digital signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
