On 20:49 Thu 26 Mar , Ryan Hsu wrote: > Hi Mike, > > We (VMware CI) run "testr run tempest.api.volume" for our Cinder CI and this > runs about ~240 tests for us. I'm guessing that the rest of the ~60 tests are > not being run due to skips and disabled features. For example, here is > a sampling of tests that are skipped in a recent run (note that this is using > tempest.conf with no explicit disabling of Cinder services): > > tempest.api.compute.test_live_block_migration.LiveBlockMigrationTestJSON.test_iscsi_volume > ... SKIPPED: Block Live migration not available > setUpClass > (tempest.api.orchestration.stacks.test_volumes.CinderResourcesTest) ... > SKIPPED: Heat support is required > setUpClass > (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV2Test) ... > SKIPPED: Cinder multi-backend feature disabled > tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume > ... SKIPPED: SSH required for this test > setUpClass > (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV1Test) ... > SKIPPED: Cinder backup feature disabled > setUpClass > (tempest.api.volume.admin.test_volumes_backup.VolumesBackupsV2Test) ... > SKIPPED: Cinder backup feature disabled > setUpClass > (tempest.api.volume.admin.test_multi_backend.VolumeMultiBackendV1Test) ... > SKIPPED: Cinder multi-backend feature disabled > tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern > ... SKIPPED: Skipped until Bug: 1373513 is resolved. > tempest.scenario.test_stamp_pattern.TestStampPattern.test_stamp_pattern ... > SKIPPED: Skipped until Bug: 1205344 is resolved. > setUpClass (tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest) > ... SKIPPED: The EC2 API is not available > setUpClass (tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest) ... > SKIPPED: The EC2 API is not available > tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern > ... SKIPPED: Skipped until Bug: 1373513 is resolved. > > As we are actually running the volume suite according to the FAQ and the > above skipped tests are documented by our CI, would it be possible to add an > exception to the rule? I'm sure these numbers will be different for all CIs > and as long as people are not abusing and hiding skipped tests, I don't see > this as a problem.
There will be no exceptions. Everyone must pass the same tests or you're not an approved volume driver for OpenStack Cinder. You should also take this bug [1] Vmware hit as a lesson of doing any excluding in your CI. The driver would've been seriously broken in Kilo if this wasn't caught. [1] - https://bugs.launchpad.net/cinder/+bug/1436603 -- Mike Perez __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
