Hi, According to the Yardstick console log, it seems that Yardstick is failed to login to the cfg01 [email protected]<mailto:[email protected]> via SSH. What TC046 do is logining to the target node and terminating the apache2 processes and check if the terminated processes will recover and the keystone service is not affected. If the process recovery time is shorter than 10s and the keystone service outage is within 5s, this test case will be considered as Pass.
The SSH info that Yardstick uses to login to the target node is written in '/etc/yardstick/pod.yaml' Can you checkout this file and make sure the info is correct. Regards, Jing Lu From: [email protected] [mailto:[email protected]] On Behalf Of Ting Wu Sent: Tuesday, January 09, 2018 9:48 PM To: [email protected] Subject: [opnfv-tech-discuss] yardstick tc046 failure HA test case: Control node Openstack service down - keystone Hi, We are running yardstick in our arm-pod with fuel installation, there are few failed testcases. One of them is "opnfv_yardstick_tc046.yaml" The testcase is to check if the keystone service is alive in all controllers in HA scenario. When we manually check on the nodes, the keystone is alive but the test case reported failure. Does someone know what the pass criteria of this test case? //login to cfg and manually check service status, according to https://docs.openstack.org/fuel-docs/latest/userdocs/fuel-user-guide/troubleshooting/service-status.html ubuntu@cfg01:~$ sudo salt "ctl*" service.status apache2 ctl02.baremetal-mcp-pike-ovs-ha.local: True ctl01.baremetal-mcp-pike-ovs-ha.local: True ctl03.baremetal-mcp-pike-ovs-ha.local: True ubuntu@cfg01:~$ sudo salt "ctl*" cmd.run "netstat -nltp | egrep '5000|35357'" ctl03.baremetal-mcp-pike-ovs-ha.local: tcp 0 0 172.16.10.13:5000 0.0.0.0:* LISTEN 1051/apache2 tcp 0 0 172.16.10.10:5000 0.0.0.0:* LISTEN 14059/haproxy tcp 0 0 172.16.10.13:35357 0.0.0.0:* LISTEN 1051/apache2 tcp 0 0 172.16.10.10:35357 0.0.0.0:* LISTEN 14059/haproxy ctl02.baremetal-mcp-pike-ovs-ha.local: tcp 0 0 172.16.10.12:5000 0.0.0.0:* LISTEN 1164/apache2 tcp 0 0 172.16.10.10:5000 0.0.0.0:* LISTEN 14095/haproxy tcp 0 0 172.16.10.12:35357 0.0.0.0:* LISTEN 1164/apache2 tcp 0 0 172.16.10.10:35357 0.0.0.0:* LISTEN 14095/haproxy ctl01.baremetal-mcp-pike-ovs-ha.local: tcp 0 0 172.16.10.11:5000 0.0.0.0:* LISTEN 2124/apache2 tcp 0 0 172.16.10.10:5000 0.0.0.0:* LISTEN 14105/haproxy tcp 0 0 172.16.10.11:35357 0.0.0.0:* LISTEN 2124/apache2 tcp 0 0 172.16.10.10:35357 0.0.0.0:* LISTEN 14105/haproxy //yardstick printout 2018-01-09 10:08:29,150 [INFO] yardstick.benchmark.core.task task.py:483 Parsing task config: /home/opnfv/repos/yardstick/tests/opnfv/test_cases/opnfv_yardstick_tc046.yaml 2018-01-09 10:08:29,218 [INFO] yardstick.benchmark.contexts.node node.py:57 Parsing pod file: etc/yardstick/nodes/fuel_baremetal/pod.yaml 2018-01-09 10:08:29,257 [INFO] yardstick.benchmark.core.task task.py:373 Starting runner of type 'Duration' Background task ended Background task ended Background task ended 2018-01-09 10:08:29,274 [INFO] yardstick.benchmark.runners.duration duration.py:46 Worker START, duration is 1s Process Duration-ServiceHA-7429: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/home/opnfv/repos/yardstick/yardstick/benchmark/runners/duration.py", line 52, in _worker_process benchmark.setup() File "/home/opnfv/repos/yardstick/yardstick/benchmark/scenarios/availability/serviceha.py", line 47, in setup attacker_ins.setup() File "/home/opnfv/repos/yardstick/yardstick/benchmark/scenarios/availability/attacker/attacker_process.py", line 28, in setup self.connection.wait(timeout=600) File "/home/opnfv/repos/yardstick/yardstick/ssh.py", line 374, in wait raise SSHTimeout("Timeout waiting for '%s'", self.host) SSHTimeout: ("Timeout waiting for '%s'", '172.16.10.12') 2018-01-09 10:18:30,210 [ERROR] yardstick.benchmark.core.task task.py:276 Scenario NO.1: "ServiceHA" ERROR! 2018-01-09 10:18:30,212 [ERROR] yardstick.benchmark.core.task task.py:127 Testcase: "opnfv_yardstick_tc046" FAILED!!! Traceback (most recent call last): File "/home/opnfv/repos/yardstick/yardstick/benchmark/core/task.py", line 123, in start data = self._run(scenarios, run_in_parallel, args.output_file) File "/home/opnfv/repos/yardstick/yardstick/benchmark/core/task.py", line 278, in _run "{0} runner status {1}".format(runner.__execution_type__, status)) RuntimeError: Duration runner status 1 2018-01-09 10:18:30,214 [INFO] yardstick.benchmark.core.task task.py:143 Task /home/opnfv/repos/yardstick/tests/opnfv/test_cases/opnfv_yardstick_tc046.yaml finished in 601 secs Best Regards Ting Wu NFV Engineer, ARMBand Team ENEA R&D Cloud Technologies Email [email protected]<mailto:[email protected]> Phone +46 709714003 Enea Software AB Jan Stenbecks Torg 17 Box 1033, SE-164 21 Kista, Sweden www.enea.com<http://www.enea.com/> [cid:[email protected]] This message, including attachments, is CONFIDENTIAL. It may also be privileged or otherwise protected by law. If you received this email by mistake please let us know by reply and then delete it from your system; you should not copy it or disclose its contents to anyone. All messages sent to and from Enea may be monitored to ensure compliance with internal policies and to protect our business. Emails are not secure and cannot be guaranteed to be error free as they can be intercepted, a mended, lost or destroyed, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of email transmission. Anyone who communicates with us by email accepts these risks.
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
