I do not think Clirr is enough as exemplified by the change of behavior in Base64 encoding with CRLFs between version 1.3 and 1.4 (see CODEC-94.)
There is no substitute for running unit tests. As we change code in a component, unit tests are created or changed. It is possible to create compatibility problems when code and unit tests change. You have to consciously and carefully leave some tests as-is from a previous versions to ensure backward compatibility. Sometimes of course, you are fixing bugs, so unit tests must change too. If you run the previous version's test against the current code base, you are simulating what users do when they drop in a new jar in their application. Since we do not have previous version's tests next to the current tests, how can we test backward compatibility? If we produce a test jar file for each release and keep it in SVN and deliver it, that is a snapshot of old "clients" for a given version. The test phase of the build can then run all of the tests in each test jar, including the test jar for the current tests, against the component jar. Gary > -----Original Message----- > From: Matt Benson [mailto:gudnabr...@gmail.com] > Sent: Monday, April 05, 2010 10:00 > To: Commons Developers List > Subject: Re: [codec][lang] Provide a test jar plus [daemon] > > Is it not sufficient to simply run clirr reports before a release? > > -Matt > > On Apr 5, 2010, at 11:16 AM, Gary Gregory wrote: > > > Seeing the discussion about [daemon] and not releasing made me > > think of another use for a test jar file. > > > > What I would like to know when evaluating an RC for releasing a > > maintenance of a commons component (from x.y.n to x.y.n+1) is that > > there is 100% binary compatibility. > > > > As part of the build I would run (at least) the 1.0.2 unit tests > > against the 1.0.3 RC. If 100% pass all is well, if not, it is > > either a bug or a known acceptable failure fixing a bug and should > > be documented somehow, probably in a ticket. > > > > This would mean that a release 1.0.3 RC would include foo- > > test-1.0.2.jar. And maybe also foo-test-1.0.0.jar and foo- > > test-1.0.1.jar, hm... > > > > Thoughts? > > > > Gary Gregory > > Senior Software Engineer > > Seagull Software > > email: ggreg...@seagullsoftware.com > > email: ggreg...@apache.org > > www.seagullsoftware.com > > > > > > From: Gary Gregory > > Sent: Tuesday, March 30, 2010 16:58 > > To: Commons Developers List > > Subject: [codec][lang] Provide a test jar > > > > I am starting with codec and lang since it what I am most > > interested in ATM... > > > > I would like to run commons.xxx unit tests as part of my build as a > > sanity check when I try out a new combo of JVM, OS, jars, etc. > > > > Right now, I would have to compile the unit tests as part of my > > build which is not great. > > > > So, what about providing a commons-codec-1.5-test.jar file like we > > provide a sources and javadoc file? > > > > Same for any commons project really... > > > > Thoughts? > > > > Gary Gregory > > Senior Software Engineer > > Seagull Software > > email: ggreg...@seagullsoftware.com > > email: ggreg...@apache.org > > www.seagullsoftware.com > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
