Hi,
Please find attached a brief report of the Tempest test cases that are failing
for each installer. I hope this helps as a feedback to the installer and
feature owners to improve the quality of the deployments and reach the release
date without errors. Now it's the time to solve as many bugs as we can.
Remember, if you know that a specific test case fails because of a known bug
that can't be fixed before the Colorado release date, we can always add that
test to the blacklist so that it's not executed.
@David: it would be nice to track the progress of this. We have a Dashboard
where we show the status (weather icons) of the test cases per scenario and per
installer.
http://testresults.opnfv.org/reporting/functest/release/master/index-status-compass.html
It's basically the same thing that I did in those txt's but in an automated
manner.
Regards,
Jose
RALLY All scenarios:
Rally fails in heat, 75.00% success in that scenario
TEMPEST:
os-odl_l2-nofeature-ha, os-odl_l3-nofeature-ha, os-onos-sfc-ha
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern
odl-nosdn-nofeature-ha
timeout. Last test executed:
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
os-odl_l2-sfc-ha
Deployment failure?
No tests run lately
os-odl_l2-bgpvpn-ha
No failures, because the tests that normally fail are
blacklisted due to lacking support for floating IP in VPN service in ODL.
RALLY All scenarios:
83.89% success: 0% in cinder, 75% heat, 80% nova
TEMPEST:
All scenarios:
setUpClass
(tempest.api.volume.test_volumes_actions.VolumesV1ActionsTest)
setUpClass
(tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest)
tempest.api.volume.test_volumes_get.VolumesV1GetTest.test_volume_create_get_update_delete
tempest.api.volume.test_volumes_get.VolumesV1GetTest.test_volume_create_get_update_delete_from_image
tempest.api.volume.test_volumes_get.VolumesV2GetTest.test_volume_create_get_update_delete
tempest.api.volume.test_volumes_get.VolumesV2GetTest.test_volume_create_get_update_delete_from_image
setUpClass
(tempest.api.volume.test_volumes_list.VolumesV1ListTestJSON)
setUpClass
(tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON)
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern
os-onos-sfc-ha
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
RALLY All scenarios:
100% Success
TEMPEST:
os-nosdn-nofeature-ha, os-odl_l2-nofeature-ha
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
os-odl_l3-nofeature-ha
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern
os-odl_l2-bgpvpn-ha
no failures
os-odl_l2-sfc-ha
no failures
os-onos-nofeature-ha
No tests run --> Healthcheck fails: ERROR - The instance
opnfv-instance4 couldn't get an IP from the DHCP agent.
os-onos-sfc-ha
tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops
RALLY:
Not executed so far
TEMPEST:
os-odl_l2-nofeature-ha
setUpClass
(tempest.api.object_storage.test_account_quotas.AccountQuotasTest)
setUpClass
(tempest.api.object_storage.test_account_services.AccountTest)
tempest.api.object_storage.test_container_quotas.ContainerQuotasTest.test_upload_large_object
tempest.api.object_storage.test_container_quotas.ContainerQuotasTest.test_upload_too_many_objects
tempest.api.object_storage.test_container_quotas.ContainerQuotasTest.test_upload_valid_object
tempest.api.object_storage.test_container_services.ContainerTest.test_create_container
tempest.api.object_storage.test_container_services.ContainerTest.test_list_container_contents
tempest.api.object_storage.test_container_services.ContainerTest.test_list_container_metadata
setUpClass
(tempest.api.object_storage.test_object_services.ObjectTest)
_______________________________________________
opnfv-tech-discuss mailing list
[email protected]
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
