Hi, SSHTimeout may be caused by lots of reasons. Maybe there is something wrong with network, router, security group, so it is hard to locate the problem for user. You can ping and ssh the ip when the yardstick ssh client try to ssh the instance. And see the return message of it.
Does this problem reproduce every time? Or maybe there is something wrong with the environment. ( Since every time after redeploying the environment, the problem is different, so there is a little possibility for the environment) So maybe you can try to redeploying the environment to see if it reproduce. Can you tell me the version of the devstack and what openstack version you’re deploying. I would deploy one locally and then try to figure it out. Regards, Jack Chan From: Marco Varlese [mailto:marco.varl...@suse.com] Sent: Thursday, March 09, 2017 6:29 PM To: Tallgren, Tapio; chenjiankun; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] [yardstick] Yardstick run-time issue Hi, So after redeploying my devstack environment, it looks like yardstick managed to deploy the two VMs but the ping test fails.... Context 'demo-2a3523ca' deployed Starting runner of type 'Duration' 2017-03-09 11:14:56,003 yardstick.benchmark.runners.duration duration.py:43 INFO worker START, duration 60 sec, class <class 'yardstick.benchmark.scenarios.networking.ping.Ping'> 2017-03-09 11:14:56,004 yardstick.benchmark.scenarios.networking.ping ping.py:56 INFO Log in via key, user:cirros, host:10.0.0.70, key_filename:/home/stack/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/resources/files/yardstick_key-0e70545c Process Process-2: Traceback (most recent call last): File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/home/stack/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/benchmark/runners/duration.py", line 47, in _worker_process benchmark = cls(scenario_cfg, context_cfg) File "/home/stack/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/benchmark/scenarios/networking/ping.py", line 60, in __init__ self.connection.wait(timeout=600) File "/home/stack/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/ssh.py", line 301, in wait raise SSHTimeout("Timeout waiting for '%s'", self.host) SSHTimeout: ("Timeout waiting for '%s'", u'10.0.0.70') Runner failed Undeploying all contexts Undeploying context 'demo-2a3523ca' Do I need to configure anything (still) for yardstick? Thanks, Marco On Wed, 2017-03-08 at 14:36 +0100, Marco Varlese wrote: On Wed, 2017-03-08 at 15:31 +0200, Tallgren, Tapio wrote: Does the nova scheduler log tell anything useful about why there are not enough hosts available? Since you are running in a VM, it could be that you really do not have enough hosts. I moved everything to a physical host since last time it was suggested that VM could be problematic. So, eveything is running on physical machine right now. -Tapio On 03/08/2017 12:54 PM, Marco Varlese wrote: Hi, I can create manually an instance with the "openstack server create". However, when yardstick does its magic I get the following error: ==== Writing /home/stack/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/resources/files/yardstick_key-e9b9c7c2 ... Deploying context 'demo-86b44547' 2017-03-08 11:33:12,907 yardstick.orchestrator.heat heat.py:450 INFO Creating stack 'demo-86b44547' error: failed to deploy stack: 'Resource CREATE failed: ResourceInError: resources.athena.demo-86b44547: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"' Undeploying all contexts ==== And then - if I check the status: (yardstick_venv) stack@wingenfelder<mailto:stack@wingenfelder>:~/yardstick> openstack server list +--------------------------------------+----------------------+--------+------------------------------+--------------+ | ID | Name | Status | Networks | Image Name | +--------------------------------------+----------------------+--------+------------------------------+--------------+ | c91fba90-500b-4c8e-86c3-f08b67a4f5e0 | athena.demo-86b44547 | ERROR | | cirros-0.3.3 | | 14a415e9-adaa-4d8a-b465-e9ccff85cbd8 | ares.demo-86b44547 | ACTIVE | demo-86b44547-test=10.0.1.11 | cirros-0.3.3 | +--------------------------------------+----------------------+--------+------------------------------+--------------+ Thanks, Marco On Wed, 2017-03-08 at 06:11 +0000, chenjiankun wrote: Hi, Can you create an instance using openstack command manually? Using the follow command: openstack server create --image cirros-0.3.3 --flavor m1.small --nic net-id=net04_ext server_name (you need to replace the flavor and nic value with your own. Using the follow command to get all flavors: openstack flavor list Using the follow command to get all network: openstack network list ) If you can create a instance, I think maybe there is something wrong with yardstick. If failed to create an instance, maybe there is something wrong with the environment. Regards, Jack Chan From: opnfv-tech-discuss-boun...@lists.opnfv.org<mailto:opnfv-tech-discuss-boun...@lists.opnfv.org> [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Marco Varlese Sent: Thursday, March 02, 2017 9:34 PM To: Tapio Tallgren; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Subject: Re: [opnfv-tech-discuss] [yardstick] Yardstick run-time issue On Thu, 2017-03-02 at 14:34 +0200, Tapio Tallgren wrote: On 03/02/2017 02:15 PM, Marco Varlese wrote: On Thu, 2017-03-02 at 13:41 +0200, Tapio Tallgren wrote: On 03/02/2017 10:45 AM, Marco Varlese wrote: Hi, In my previous email I did not mention the use of the docker image, I was trying to install it on the system following the online Yardstick documentation. Why do I need to use the docker image necessarily? I have been using a kvm VM for Yardstick. Docker is easier to use since you do not need to set up the virtenv etc. Also, are there any particular reasons/dependencies why it has to be Ubuntu 14.04? It's quite "old" and the new LTS (16.04) has been out for a while. I just used the Xenial Ubuntu and it worked equally well. I downloaded and booted the Xenial image, git cloned Yardstick to it, and then installed Yardstick on it. For reporting, I used a Docker image. How did you setup the Openstack environment? In my case I cloned DevStack (master) and installed it on the box. My setup has everything installed onto one single VM: 1) DevStack (with Heat) 2) Yardstick with its virtual-env And when I try launch the ping.yaml test case I get the error mentioned in my prev email... Any thoughts/ideas? I had a physical deployment. Since you are running everything in a VM, it is best to use the virtenv and not launch another VM. Can you use the "openstack server list" command from the Yardstick virtenv? Do you see cirros-0.3.3 when you do "openstack image list"? I think there was some confusion about cirros-0.3.3 and cirros-0.3.4, but I cannot find the reference right now. Yes I can see it... I downloaded the image from http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img and installed it via glance... +--------------------------------------+--------------------------+--------+ | ID | Name | Status | +--------------------------------------+--------------------------+--------+ | 03e72a29-88fc-4be7-90e4-70173fa80ce2 | cirros-0.3.3 | active | +--------------------------------------+--------------------------+--------+ ==== I just redeployed everything on a physical machine... just following the same steps... the error I get now is different Writing /root/yardstick_venv/lib/python2.7/site-packages/yardstick-0.1.dev0-py2.7.egg/yardstick/resources/files/yardstick_key-1d2a644a ... Deploying context 'demo-d3c047d1' 2017-03-02 14:22:26,407 yardstick.orchestrator.heat heat.py:450 INFO Creating stack 'demo-d3c047d1' error: failed to deploy stack: 'ERROR: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>504 Gateway Timeout</title> </head><body> <h1>Gateway Timeout</h1> <p>The gateway did not receive a timely response from the upstream server or application.</p> </body></html> ' Undeploying all contexts ====== These are my ENV variables: OS_PROJECT_DOMAIN_ID=default OS_REGION_NAME=RegionOne OS_USER_DOMAIN_ID=default OS_PROJECT_NAME=demo OS_IDENTITY_API_VERSION=3 OS_PASSWORD=secret OS_AUTH_URL=http://AA.BBB.CC.DDD:5000/v2.0<http://10.161.8.252:5000/v2.0> OS_USERNAME=demo OS_TENANT_NAME=demo OS_VOLUME_API_VERSION=2 EXTERNAL_NETWORK=net04_ext Interestingly, the openrc file (which I have to source) by default uses v3 for the OS_AUTH_URL but I had to change it to v2.0 because Yardstick was complaining with the following error: error: failed to deploy stack: 'Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-dff1cdff-1eeb-40a5-94a4-b250444ed32b)' -Tapio Thanks, Marco -Tapio Thanks, Marco On Thu, 2017-03-02 at 08:35 +0000, chenjiankun wrote: Hi Marco, What’s the latest status of this issue? Does my solution be OK for you? I’m looking forward to your reply. Regards, Jack Chan From: chenjiankun Sent: Thursday, February 23, 2017 5:25 PM To: 'Marco Varlese' Cc: opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Subject: RE: [opnfv-tech-discuss] [yardstick] Yardstick run-time issue Hi Marco, I am Jack from yardstick team. I haven’t experienced this issue before. But it seems there are some errors with the cirros image. I think you can try to run ping test case in the follow steps: 1. docker run -itd --privileged --name yardstick opnfv/yardstick:latest 2. docker exec -it yardstick bash 3. config and source the openstack openrc file(include EXTERNAL_NETWORK) 4. load images: you can execute the follow command in this directory: cd /home/opnfv/repos/yardstick bash tests/ci/load_image.sh 5. run test case yardstick -d task start sample/ping.yaml BTW, can you tell me: 1. what’s your Openstack version in your SUT? 2. Which yardstick docker are you using? If you have any other questions, please feel free to let me know. Thanks. Best Regards, Jack Chan -----Original Message----- From: opnfv-tech-discuss-boun...@lists.opnfv.org<mailto:opnfv-tech-discuss-boun...@lists.opnfv.org> [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Marco Varlese Sent: Tuesday, February 21, 2017 7:37 PM To: opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Subject: [opnfv-tech-discuss] Yardstick run-time issue Hi, I'm trying to setup an environment to deploy Yardstick; to start with I'm doing everything in one single Virtual machine. The Virtual Machine basically represent both the Jumphost and the SUT. I understand this is not the best to measure performance but my real lab setup is being currently worked on and I'd like to get ahead of the game with learning. I have an Openstack environment installed via Devstack (master branch) including Heat. When I try and launch the ping.yaml test case I get the following error: 2017-02-21 12:18:29,515 yardstick.orchestrator.heat heat.py:440 INFO Creating stack 'demo-054b1f93' error: failed to deploy stack: 'ERROR: Property error: : resources.ares.demo- 054b1f93.properties.image: : "cirros-0.3.3" does not validate glance.image (constraint not found)' Has anybody experienced this issue before? Any help or input would be much appreciated. Thanks, Marco _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
