I think we can document what we have now and what our possible road map is.
Let's cut an RC and see what happens. Are you up for RM'ing? Gary On Thu, Jul 10, 2014 at 4:12 AM, Benedikt Ritter <brit...@apache.org> wrote: > 2014-07-09 4:15 GMT+02:00 Gary Gregory <garydgreg...@gmail.com>: > > > We do have a discrepancy between our format class and lexer (which is > > hardwired with CR & LF). > > > > Ideally, it seems the lexer should pickup it's set of EOL Strings from > the > > format. > > > > I recall reading worries of performance issues changing this but either > we > > support all of the EOL strings including some of the odd ball ones like > > Unicode, or we do not. Perhaps we can have an alternate Lexer that takes > a > > set of EOL strings if performance is really that much worse. > > > > Sounds reasonable, but seems to be a lot of work. Maybe we can just > document that 1.0 can only handle CR & LF and add the ability for more > exotic record separators in 1.1. I'm hoping for higher adoption and more > patches once we have a release on maven central. > > Benedikt > > > > > > Gary > > > > > > On Mon, Jul 7, 2014 at 1:47 PM, Benedikt Ritter <brit...@apache.org> > > wrote: > > > > > Any thoughts about this fix? Could be a solution to push out 1.0. If we > > > come up with a more generic solution afterwards, we can still deprecate > > > escapeCRLFOnce. > > > > > > Benedikt > > > > > > ---------- Forwarded message ---------- > > > From: Tillmann Gaida (JIRA) <j...@apache.org> > > > Date: 2014-06-30 10:36 GMT+02:00 > > > Subject: [jira] [Comment Edited] (CSV-35) Escaped line separators are > not > > > supported > > > To: brit...@apache.org > > > > > > > > > > > > [ > > > > > > > > > https://issues.apache.org/jira/browse/CSV-35?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047460#comment-14047460 > > > ] > > > > > > Tillmann Gaida edited comment on CSV-35 at 6/30/14 8:34 AM: > > > ------------------------------------------------------------ > > > > > > I added a patch "commons-csv CSV-35 escapeCRLFOnce[ test].patch", which > > > introduces a CSVFormat setting "escapeCRLFOnce", which enables the > > desired > > > behaviour in Lexer. It is false by default and I did not change > > > CSVFormat.MYSQL, which might be approprate. I am not exactly happy with > > the > > > naming of the setting. Consider renaming it if you happen to build upon > > the > > > patch. > > > > > > EDIT: clarity > > > > > > EDIT: This is a very specific setting. A cleaner solution would > probably > > be > > > to allow escaping of record separators by a single escape char. However > > it > > > appears that the MYSQL format uses LF as a record separator, so we > would > > > need to have multiple record separators, which in this case would not > be > > > actual record separators. > > > > > > I'd argue that CRLF is special enough to have an individual setting, > but > > I > > > would also agree with having a cleaner CSVFormat. The only real > > alternative > > > would be having a way to individually specify character sequences and a > > > replacement if they are preceded by the escape char. > > > > > > > > > was (Author: tillmann gaida): > > > I added a patch "commons-csv CSV-35 escapeCRLFOnce[ test].patch", which > > > introduces a CSVFormat setting "escapeCRLFOnce", which enables the > > desired > > > behaviour in Lexer. It is false by default and I did not change > > > CSVFormat.MYSQL, which might be approprate. I am not exactly happy with > > the > > > naming of the setting. Consider renaming it if you happen to build upon > > the > > > patch. > > > > > > EDIT: clarity > > > > > > > Escaped line separators are not supported > > > > ----------------------------------------- > > > > > > > > Key: CSV-35 > > > > URL: https://issues.apache.org/jira/browse/CSV-35 > > > > Project: Commons CSV > > > > Issue Type: Bug > > > > Reporter: Emmanuel Bourg > > > > Fix For: 1.0 > > > > > > > > Attachments: CSV-35.patch, commons-csv CSV-35 escapeCRLFOnce > > > test.patch, commons-csv CSV-35 escapeCRLFOnce.patch, > > > mysql-export-line-terminated-by-crlf.csv, > > > mysql-export-line-terminated-by-lf.csv > > > > > > > > > > > > Commons CSV doesn't handle escaped line separators, for example: > > > > {code} > > > > value1;value2;value3a\ > > > > value3b > > > > {code} > > > > In this case the expected result is: > > > > {code}["value1", "value2", "value3a\nvalue3b"]{code} > > > > This kind of escaping is produced by MySQL, whether the field > enclosing > > > is enabled or not. It's possible to see enclosing quotes and escaped > line > > > separators like this: > > > > {code} > > > > "value1";"value2";"value3a\ > > > > value3b" > > > > {code} > > > > > > > > > > > > -- > > > This message was sent by Atlassian JIRA > > > (v6.2#6252) > > > > > > > > > > > > -- > > > http://people.apache.org/~britter/ > > > http://www.systemoutprintln.de/ > > > http://twitter.com/BenediktRitter > > > http://github.com/britter > > > > > > > > > > > -- > > E-Mail: garydgreg...@gmail.com | ggreg...@apache.org > > Java Persistence with Hibernate, Second Edition > > <http://www.manning.com/bauer3/> > > JUnit in Action, Second Edition <http://www.manning.com/tahchiev/> > > Spring Batch in Action <http://www.manning.com/templier/> > > Blog: http://garygregory.wordpress.com > > Home: http://garygregory.com/ > > Tweet! http://twitter.com/GaryGregory > > > > > > -- > http://people.apache.org/~britter/ > http://www.systemoutprintln.de/ > http://twitter.com/BenediktRitter > http://github.com/britter > -- E-Mail: garydgreg...@gmail.com | ggreg...@apache.org Java Persistence with Hibernate, Second Edition <http://www.manning.com/bauer3/> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/> Spring Batch in Action <http://www.manning.com/templier/> Blog: http://garygregory.wordpress.com Home: http://garygregory.com/ Tweet! http://twitter.com/GaryGregory
