[
https://issues.apache.org/jira/browse/CLOUDSTACK-8975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970612#comment-14970612
]
ASF GitHub Bot commented on CLOUDSTACK-8975:
--------------------------------------------
Github user nitt10prashant commented on the pull request:
https://github.com/apache/cloudstack/pull/960#issuecomment-150501158
@runseb
"I personally fail to see the benefit of skipped tests versus a failed test"
If i may ,
skip is a type of failure where you know in advance about failure ,
since you know about the failure you just Handle them in order to
differentiate from a general failure(may be product issue).So its all about
categorizing the failure like expected failure and unexpected failure
Its about how you see reports
If you accept skip as known failure then 100% success on BVT ~ 20% of
skipped =80% success with 20% of known failure ,at least not a product issue
which you can say without looking at particular script .
Without skip you have to analyze each and every failure then you may come
up with % of product issue and % of failure due to setup an all
"While @nitt10prashant and @pvr9711 you seem to want 100% success on BVT
even if it includes some tests that are actually not run."
My comment:
1-Not run test cases are the one which are not picked up by runner may be
coz of different tags and those test cases will not appear in your report
2-skipped test case are the one which are picked up by runner but due to
non availability of configurations or some other reasons could not run
completely and they will appear in your report .
3-I want 100% success on BVT without any skip if all the setup config is
available .incase they are not i would prefer to skip them in order to find out
real failure which does not include any failure because of setup/configuration
just by looking at my report .
> [Automation]skip test case if niciraNvp settings are not available in config
> file
> ---------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-8975
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8975
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Automation
> Affects Versions: 4.7.0
> Reporter: prashant kumar mishra
> Assignee: prashant kumar mishra
> Priority: Minor
> Fix For: 4.7.0
>
>
> /root/cloudstack/test/integration/smoke/test_nicira_controller.py is
> failing if niciraNvp settings are not available in config file. Modifying
> script to skip test case if settings are not available
> log
> =======
> File "/root/cloudstack/test/integration/smoke/test_nicira_controller.py",
> line 46, in setUpClass
> cls.nicira_hosts = cls.config.niciraNvp.hosts
> AttributeError: 'NoneType' object has no attribute 'hosts'
> -------------------- >> begin captured stdout << ---------------------
> === TestName: None | Status : EXCEPTION ===
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
