Sowmya Krishnan created CLOUDSTACK-4974:
-------------------------------------------
Summary: [Automation] Handle exceptions for every API call in the
test and verify the output
Key: CLOUDSTACK-4974
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4974
Project: CloudStack
Issue Type: Test
Security Level: Public (Anyone can view this level - this is the default.)
Components: Automation, marvin
Affects Versions: 4.2.0
Environment: Test scripts written using Marvin
Reporter: Sowmya Krishnan
Present design of test cases assumes that the different APIs triggered as a
pre-requisite of the test would all pass. For example, if we creating a VPC, we
assume that the listVPCOffering API would pass (or that it is tested as part of
some other test). Or after creating a Network we assume that listNetwork on the
created network id would pass and return the expected network object.
Also for most tests, not every single API call is enclosed within a try block.
Given this, it is likely that if an API not directly related to the test fails,
the test script can still fail and throws exceptions. While running huge number
of tests this could lead to difficulty in debugging.
One way to deal with this is to handle exceptions for every API call, but this
would bloat the test scripts to unmanageable size. Other option that came up
was to include a common decorator for the entire test which would take care of
handling exception for the test.
We probably have a better way to handle this in the re-factored Marvin, but at
some point we probably need to fix existing scripts as well which have not been
re-written using the "new" Marvin.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
