I too like to use random data in test cases but you also have to use edge cases that random data tend to avoid. It is painful to recall the millions of successful tests of my Dictionary implementation that failed to detect that "Dictionary new: 0" created a non-functional object. ((self pvtHash: key) \\ capacity) + 1" did not work well when capacity was 0. 0 size, 0 capacity, 0 room left, any edge case. Conversely, you have to avoid special cases sometimes: when testing x ./ y and y \ x for Quaternions, you want to make sure y isZero not.
One thing I would like some advice on is testing traits. I understand about testing classes and about testing methods in classes (though I am never thorough enough), but how do you test a method that isn't bound to a particular context, where the thing that reveals a mistake may be a class it will be combined with in the future that doesn't exist yet? What is considered good practice? On Tue, 4 Feb 2025 at 05:25, Russ Whaley <whaley.r...@gmail.com> wrote: > > Folks, I really appreciate (and learn from) these types of conversations - > especially on testing where I struggle for consistency and completeness. > > I tend to agree that my testing focuses on methods, however the majority of > my ‘failures’ revolve around insufficient object values that eventually have > my tests failing - I’ve defaulted to generating ‘data’ as randomly as I can > early on, however non-realistic, and running tests a few thousand times to > point out assumptions I make in my methods. > > Thanks for discussions! > > Russ Whaley > whaley.r...@gmail.com > > > On Thu, Jan 23, 2025 at 5:02 AM Richard O'Keefe <rao...@gmail.com> wrote: >> >> This makes the assumption that there is, most of the time, a single, >> unique, object that the test is clearly about. >> I hasten to admit that my unit testing practice needs a lot to be desired. >> But it has not been my experience that there *is* any such object much >> of the time. >> For example, to test a method, I will sometimes create two equal >> objects, manipulate one of them in the >> old trusted way, manipulate the other in the new way, and then see if >> the objects are equal again. >> Which then is *the* subject? >> The answer is neither, because what the test is *about* is not an >> object, but a method. >> Or again, I open a stream to a new file, create a complex object, >> persist it on the stream, close >> the stream, open another stream reading from the file, reconstitute an >> object from it, and then >> check that the two objects are isomorphic. Which is *the* subject? >> Neither, because what I am testing is the encoding and decoding >> methods, not the objects. >> >> The key paragraph is this one: >> <quote> >> One could argue that there is not always a single object under test. >> In some situations that is correct, for instance when testing the >> interaction between objects. But in my experience, most unit tests >> have a single object under test. >> </quote> >> >> And it is *this* that we really need a blog post on. I am very ready >> to believe that you are a much better tester than I am. >> (That wouldn't be hard.) But *something* interesting is going on when >> I think in terms of testing classes and methods to >> the point where I find the idea of testing a particular object almost >> incomprehensible, yet you think this is the norm. >> I take some comfort from the fact that the Gherkin language in >> Cucumber (and why oh which has nobody bothered to >> write down a proper grammar, let alone even an informal semantics, for >> it? How am I supposed to know what is and what >> is not legal Gherkin?) goes out of its way to conceal the existence >> of subjects, should they exist, from test reviewers, and >> calls what a test is about a "feature". >> >> Here are two tests for the Quaternion class: >> testMultiplicationByScalarCommutes >> sampleExactScalars do: [:x | >> sampleExactQuaternions do: [:y | >> self assert: (x*y) = (y*x)]]. >> >> testMultiplicationOfPowersCommutes >> sampleExactQuaternions do: [:x | >> |y z| >> y := x squared. >> z := x cubed. >> self assert: (y*z) = (z*y)]. >> >> These two methods test important properties of the #* method. >> >> Here's another one, part of testing the test data.. >> >> testExactScalarsAreExactScalars >> sampleExactScalars do: [:each | >> self assert: each isScalar. >> self assert: each isExact]. >> >> This tests the *methods* #isScalar and #isExact, and indirectly the >> code that generates the test data. >> It's not about any particular object. Nor is it about *one* method. >> It's very hard to write a test case that >> tests only one method. In fact my own rather embarrassing experience >> is that when a test case of mine >> failed it quite often wasn't the method I thought I was testing that >> was wrong. Returning to the >> persistence example, it's not testing #saveOn: (object saveOn: >> byteStream) or #restore (byteStream restore). >> It's testing that the two methods are properly related. It would be >> wrong to call it #testSaveOn: and it >> would be wrong to call it #testRestore. There's a distinction here: >> it *is* possible to write methods to test >> #saveOn: by itself, by writing various objects and checking that you >> get the right bytes. But if the external >> representation is changed (and I need to change it), the tests for >> #saveOn: and #restore as such will have >> to be rewritten, while the tests for #saveOn: and #restore *together* will >> not. >> >> Cucumber offers an insight here, I think. A scenario in Gherkin is >> not a test case for an object. >> Nor is it a test case for a method. It is a test case for a composite >> *behaviour* of a system, >> at some contextually appropriate level of detail. >> >> Brian Marick had some lovely stickers. One of them reads "An example >> would be handy about now." >> >> I note that your blog post *did* provide an example of testing Date. >> In which none of your test cases used 'subject'. >> I entirely agree with you that the three methods have good >> intention-revealing names. >> What we actually need is some examples where 'subject' pays off. >> >> >> >> On Mon, 20 Jan 2025 at 04:04, Koen De Hondt >> <k...@all-objects-all-the-time.st> wrote: >> > >> > Dear Pharo users and developers, >> > >> > In my series of blog posts on testing, I added a follow-up post about a >> > dedicated name for the object under test in SUnit tests, the “subject". >> > >> > Happy reading! >> > >> > Ciao, >> > Koen >> >
