On Fri, Dec 08, 2017 at 08:56:58PM +1100, Ian Wienand wrote: > On 12/08/2017 08:38 PM, Ian Wienand wrote: > > However, the gate did not become healthy. Upon further investigation, > > the executors are very frequently failing jobs with > > > > 2017-12-08 06:41:10,412 ERROR zuul.AnsibleJob: [build: > > 11062f1cca144052afb733813cdb16d8] Exception while executing job > > Traceback (most recent call last): > > File "/usr/local/lib/python3.5/dist-packages/zuul/executor/server.py", > > line 588, in execute > > str(self.job.unique)) > > File "/usr/local/lib/python3.5/dist-packages/zuul/executor/server.py", > > line 702, in _execute > > File "/usr/local/lib/python3.5/dist-packages/zuul/executor/server.py", > > line 1157, in prepareAnsibleFiles > > File "/usr/local/lib/python3.5/dist-packages/zuul/executor/server.py", > > line 500, in make_inventory_dict > > for name in node['name']: > > TypeError: unhashable type: 'list' > > > > This is leading to the very high "retry_limit" failures. > > > > We suspect change [3] as this did some changes in the node area. > > [3] https://review.openstack.org/521324 > > It was quickly pointed out by frickler that jobs to ze04 were working, > which made it clear that actually the executors just needed to be > restarted to pick up these changes too. I've done that and things are > looking better. > Thanks and yah, we needed to restart all of zuul for changes in 521324, sorry for not over communicating that.
> -i > > _______________________________________________ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra _______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
