Hi All,
As Cloudstack is becoming more modularized, we can start thinking about
how to test these components.
The organization of maven projects in cloudstack. At lowest level, it's a
maven project. The modularized component, means a group of maven projects. The
subsystem, means a logical group of components. A system, will be build on a
lot of subsystems. For example, there will be a system called, cloud-engine,
which is a server, provides restful API service to other systems(e.g API
server). Cloud-engine will have a set of subsystems: computing, storage,
network etc. For each subsystem, will have a group of components: e.g. storage
subsystem will include volume/image/snapshot .. components. For each component,
will have a set of maven projects, for example, volume component will have a
maven project, called storage-volume, and a lot of plugin projects, such as
(storage-volume-plugin-solidfire, storage-volume-plugin-netapp etc).
Based on above hierarchy, the integration test will happen at different
levels: at component level, at subsystem level, at system level etc. I want to
bring up the discuss about how to organize integration tests, and which
integration test framework should we use.
First, how to organize integration tests? The organization will be
affected/constrained by the dependency between maven projects you want to test.
1. Will the integration test at each level have its own maven
project? Or we put all the integration test cases into one giant maven
project? I'd prefer one maven project at one level, as it looks like more
cleaner. For example, there will be a storage-integration-test maven project,
which depends on storage-volume-integration-test,
storage-image-integration-test, and so on, while
storage-volume-integration-test will depend on storage-volume,
storage-volume-plugin-solidfire, and storage-volume-plugin-netapp and so on.
When you write test case at each level, you only focus on the limited area you
want to test: the design of test case and the configuration file of each test
case, will/can only be specific to the area. Hope it will make writing test
case easier.
2. Which test framework we should use? Junit or testNG? I was sold to
testNG by its feature sets: parametrized test case, group test cases, running
test case in parallel at different level(method, class etc), test case
dependency etc. You can find out more information about testNG at
http://kaczanowscy.pl/tomek/sites/default/files/testng_vs_junit.txt.slidy_.html#(1).
But in order to integrate testNG with Spring, need a little bit coding.
For example, may need to add a listener in testNG, to inject @DB annotation,
and need to make sure mockito will work. I created a base testNG class:
CloudStackTestNGBase, which can read configuration parameters from a testNG
config xml file, setup @DB context for each test case etc.
Comments/feedback are welcome!
