We do have a discrepancy between our format class and lexer (which is
hardwired with CR & LF).
Ideally, it seems the lexer should pickup it's set of EOL Strings from the
format.
I recall reading worries of performance issues changing this but either we
support all of the EOL strings including some of the odd ball ones like
Unicode, or we do not. Perhaps we can have an alternate Lexer that takes a
set of EOL strings if performance is really that much worse.
Gary
On Mon, Jul 7, 2014 at 1:47 PM, Benedikt Ritter <brit...@apache.org>
wrote:
Any thoughts about this fix? Could be a solution to push out 1.0. If we
come up with a more generic solution afterwards, we can still deprecate
escapeCRLFOnce.
Benedikt
---------- Forwarded message ----------
From: Tillmann Gaida (JIRA) <j...@apache.org>
Date: 2014-06-30 10:36 GMT+02:00
Subject: [jira] [Comment Edited] (CSV-35) Escaped line separators are not
supported
To: brit...@apache.org
[
https://issues.apache.org/jira/browse/CSV-35?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047460#comment-14047460
]
Tillmann Gaida edited comment on CSV-35 at 6/30/14 8:34 AM:
------------------------------------------------------------
I added a patch "commons-csv CSV-35 escapeCRLFOnce[ test].patch", which
introduces a CSVFormat setting "escapeCRLFOnce", which enables the
desired
behaviour in Lexer. It is false by default and I did not change
CSVFormat.MYSQL, which might be approprate. I am not exactly happy with
the
naming of the setting. Consider renaming it if you happen to build upon
the
patch.
EDIT: clarity
EDIT: This is a very specific setting. A cleaner solution would probably
be
to allow escaping of record separators by a single escape char. However
it
appears that the MYSQL format uses LF as a record separator, so we would
need to have multiple record separators, which in this case would not be
actual record separators.
I'd argue that CRLF is special enough to have an individual setting, but
I
would also agree with having a cleaner CSVFormat. The only real
alternative
would be having a way to individually specify character sequences and a
replacement if they are preceded by the escape char.
was (Author: tillmann gaida):
I added a patch "commons-csv CSV-35 escapeCRLFOnce[ test].patch", which
introduces a CSVFormat setting "escapeCRLFOnce", which enables the
desired
behaviour in Lexer. It is false by default and I did not change
CSVFormat.MYSQL, which might be approprate. I am not exactly happy with
the
naming of the setting. Consider renaming it if you happen to build upon
the
patch.
EDIT: clarity
Escaped line separators are not supported
-----------------------------------------
Key: CSV-35
URL: https://issues.apache.org/jira/browse/CSV-35
Project: Commons CSV
Issue Type: Bug
Reporter: Emmanuel Bourg
Fix For: 1.0
Attachments: CSV-35.patch, commons-csv CSV-35 escapeCRLFOnce
test.patch, commons-csv CSV-35 escapeCRLFOnce.patch,
mysql-export-line-terminated-by-crlf.csv,
mysql-export-line-terminated-by-lf.csv
Commons CSV doesn't handle escaped line separators, for example:
{code}
value1;value2;value3a\
value3b
{code}
In this case the expected result is:
{code}["value1", "value2", "value3a\nvalue3b"]{code}
This kind of escaping is produced by MySQL, whether the field enclosing
is enabled or not. It's possible to see enclosing quotes and escaped line
separators like this:
{code}
"value1";"value2";"value3a\
value3b"
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
--
http://people.apache.org/~britter/
http://www.systemoutprintln.de/
http://twitter.com/BenediktRitter
http://github.com/britter
--
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
