I tried searching bugzilla to see if this had been noticed before but
couldn't find anything. My project is using Cruise Control to rebuild our
development database every night and send out unit test results in the
morning to the developers so in the morning they know whether the latest SQL
scripts they checked in worked properly, whether our test data loaded
correctly, etc. I was noticing that some of the test failures shown in the
emails were missing the actual reason for the test failure -- they just said
"at <class name, line number etc>".
I've tracked down the issue to the Ant junit task XML formatter output which
is used by the CruiseControl email. CruiseControl looks for the failure
message in the first line of the stacktrace in the text node child of the
<failure> tag, which normally looks something like
"junit.framework.AssertionFailedError: This data is more than 24 hours
old!." However, the Ant junit XML formatter strips off the first line with
the failure message in any test failure where the failure message contains
the word " more", resulting in just the "at ...." line remaining, which is
what shows up in CruiseControl's email.
Looking at the Ant 1.8.2 source code I tracked that down to the
org.apache.tools.ant.taskdefs.optional.junit.JUnitTaskRunner.getFilteredTrace
method. It removes lines from the stacktrace if they contain any of the
following values:
private static final String[] DEFAULT_TRACE_FILTERS = new String[] {
"junit.framework.TestCase",
"junit.framework.TestResult",
"junit.framework.TestSuite",
"junit.framework.Assert.", // don't filter AssertionFailure
"junit.swingui.TestRunner",
"junit.awtui.TestRunner",
"junit.textui.TestRunner",
"java.lang.reflect.Method.invoke(",
"sun.reflect.",
"org.apache.tools.ant.",
// JUnit 4 support:
"org.junit.",
"junit.framework.JUnit4TestAdapter",
" more",
};
As you can see, the last item is " more". Bingo! Presumably this was added
to remove the "... 48 more" line you see at the end of some stacktraces?
Anyway, I was wondering whether this qualifies as something worth fixing,
since:
- anybody using CruiseControl with Ant-run JUnit tests is fairly likely to
run into it (it should be fairly common to fail a test when there are "more
of X than there should be")
- it's unlikely that anybody running into this problem will suspect the word
"more" as the culprit
- and even if they do know, it's hard to implement a policy of "Okay
everybody, remember not to use the word 'more' in your failure messages."
How to fix it is another question entirely... maybe make it a regex and test
for "\.\.\. \d+ more"?
Bob
