Hi, while working on [CSV-58] - Escape Handling needs rethinking [1] I have come to the conclusion that the test environment needs some clean up. The problem that I see is, that some of the tests test several things at once. That makes it hard to identify what exactly is broken, whenever a test fails. I'd like to do some changes but want to have some feedback first.
Starting with CSVLexerTest I would like to split up test methods into simpler tests that only test one (or a few things). I hope that this way the testNextTokenX methods become obsolete. After cleaning up the lexer test I want to have a close look at CSVParserTest. To me it looks like, that some of the tests in the parser really belong to the lexer test. The Parser test should mainly be concerned with Records and not to much with the actual contents of tokens (the parsing of tokens is tested in the lexer test). I'm also thinking of implementing matchers for the Token class that can do the following: assertThat(token, hasType(EOF)); assertThat(token, hasContent("expected Content")); This will make the assertTokenEquals method obsolete. Extending a TypeSafeDiagnostingMatcher [2] yields the possibility of adding useful information like: Expected token that has type EOF but was EORECORD I have already added some tests to the CSVLexerTest that may serve as an example of what I'm thinking about [3]. If nobody object's I'll probably start tomorrow or on monday with this. Regards, Benedikt [1] https://issues.apache.org/jira/browse/CSV-58 [2] https://github.com/hamcrest/JavaHamcrest/blob/master/hamcrest-core/src/main/java/org/hamcrest/TypeSafeDiagnosingMatcher.java [3] http://svn.apache.org/viewvc?view=revision&revision=r1460364 -- http://people.apache.org/~britter/ http://www.systemoutprintln.de/ http://twitter.com/BenediktRitter http://github.com/britter