Can any tests that require external components be added to the functional regressions that are enabled now? This way you can keep unit and functional tests clearly separated.
-----Original Message----- From: Alex Huang [mailto:alex.hu...@citrix.com] Sent: Friday, November 09, 2012 12:24 PM To: cloudstack-dev@incubator.apache.org Subject: RE: Unit tests > -----Original Message----- > From: Alex Huang [mailto:alex.hu...@citrix.com] > Sent: Friday, November 09, 2012 12:19 PM > To: cloudstack-dev@incubator.apache.org > Subject: RE: Unit tests > > > >> 2 - Do we want to get stricter about what exactly is considered > > >> to be a unit test (vs an automated test that may or may not be > > >> based on the junit framework)? > > > > > > I actually consider db tests to be unit tests as long as the db is > > > setup and > > cleaned up by the unit test itself. > > > > Having a real DB as a requirement for low level unit tests does two > > things that I don't like. > > > > First, it requires the configuration of a database (in our case, > > MySQL) as part of running the test. That's a build dependency that > > (IMO) shouldn't be needed. > > > > Second, configuring and using a real database significantly slows > > down the unit test process. For unit tests to be useful, they have > > to be as fast as possible. We simply don't have enough unit tests > > for this to be a major problem right now, but as the coverage > > increases (true unit tests, not integration tests) this will get much worse. > > > > In my ideal world, unit tests are limited in their test scope to > > only cover code within a particular object (inherited or directly > > accessed). Imported objects should usually be mocked. We might be > > arguing about nuance of definition here, but I've always tried to > > keep a bright red line between unit and integration tests. Both > > matter, but they test different things. > > > > Actually, you and I are not that far apart. I'm thinking mainly in > terms of testing code that are actually accessing database. For > example, the unit test is to ensure a search query returns the results > that are intended. You can't really mock those up. > > For code paths that are not testing search queries but are just > dependent on search results, I agree with you. Just mock up the DAOs. And as for it being slow, perhaps we can split unit tests into two levels. One level is for testing access to db and one level is for actual business logic. --Alex
