On Fri, Nov 9, 2012 at 3:24 PM, Alex Huang <alex.hu...@citrix.com> wrote: > > >> -----Original Message----- >> From: Alex Huang [mailto:alex.hu...@citrix.com] >> Sent: Friday, November 09, 2012 12:19 PM >> To: cloudstack-dev@incubator.apache.org >> Subject: RE: Unit tests >> >> > >> 2 - Do we want to get stricter about what exactly is considered to be >> > >> a unit test (vs an automated test that may or may not be based on the >> > >> junit framework)? >> > > >> > > I actually consider db tests to be unit tests as long as the db is setup >> > > and >> > cleaned up by the unit test itself. >> > >> > Having a real DB as a requirement for low level unit tests does two >> > things that I don't like. >> > >> > First, it requires the configuration of a database (in our case, >> > MySQL) as part of running the test. That's a build dependency that >> > (IMO) shouldn't be needed. >> > >> > Second, configuring and using a real database significantly slows down >> > the unit test process. For unit tests to be useful, they have to be >> > as fast as possible. We simply don't have enough unit tests for this >> > to be a major problem right now, but as the coverage increases (true >> > unit tests, not integration tests) this will get much worse. >> > >> > In my ideal world, unit tests are limited in their test scope to only >> > cover code within a particular object (inherited or directly >> > accessed). Imported objects should usually be mocked. We might be >> > arguing about nuance of definition here, but I've always tried to keep >> > a bright red line between unit and integration tests. Both matter, >> > but they test different things. >> > >> >> Actually, you and I are not that far apart. I'm thinking mainly in terms of >> testing code that are actually accessing database. For example, the unit >> test >> is to ensure a search query returns the results that are intended. You can't >> really mock those up. >> >> For code paths that are not testing search queries but are just dependent on >> search results, I agree with you. Just mock up the DAOs. > > And as for it being slow, perhaps we can split unit tests into two levels. > One level is for testing access to db and one level is for actual business > logic. > > --Alex >
I think that's the right answer. If we can have them broken out as functional tests, which are run by the CI server or a developer that wants to run them on demand, then that would be best. If the unit tests are in the compile path, and are fast, then we always know that each build will force a unit test run. That really requires the separation of unit from functional.
