[
https://issues.apache.org/jira/browse/FLINK-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876735#comment-16876735
]
chaiyongqiang commented on FLINK-12979:
---------------------------------------
When i check com.fasterxml.jackson.dataformat.csv.CsvSchema, there's no limit
for setLineSeparator. The default is '\n', So, For flink, we need set the
LineSeparator to ''.
Even we set a custom property, we also need set the LineSeparator to '' when
we handle the custom property. So just make '' as an available charactor is
a simple but effective way to solve this.
Just like following check. we could allow user set an ''. It has the same
result.
if (!delimiter.equals("\n") && !delimiter.equals("\r") &&
!delimiter.equals("\r\n")) {
throw new IllegalArgumentException(
"Unsupported new line delimiter. Only
\\n, \\r, or \\r\\n are supported.");
}
Do you think the way of opening the limit of setting '' is ok ?
> Sending data to kafka with CsvRowSerializationSchema always adding a "\n",
> "\r","\r\n" at the end of the message
> ----------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-12979
> URL: https://issues.apache.org/jira/browse/FLINK-12979
> Project: Flink
> Issue Type: Bug
> Components: API / Type Serialization System
> Affects Versions: 1.8.0, 1.8.1, 1.8.2, 1.9.0, 2.0.0
> Reporter: chaiyongqiang
> Assignee: Hugo Louro
> Priority: Major
>
> When sending data to kafka using CsvRowSerializationSchema, the
> CsvRowSerializationSchema#serialize method helps generating value for
> KafkaRecord, which will call CsvEncoder#endRow and in which a
> _cfgLineSeparator will be added at the end of KafkaRecord.value.
> But For CsvRowSerializationSchema#Builder , when you calling the mothod
> setLineDelimiter only "\n","\r","\r\n" could be used as the parameter.
> It's not friendly when you want to send a message "123,pingpong,21:00" to
> kafka but kafka receives a message "123,pingpong,21:00\r\n".
> I'm not sure about the reason for limitting the lineDelimiter to
> "\n","\r","\r\n" ？ In previous version and jackson-databind, there's no
> limits on lineDelimiter.
> But at least it should let the application developer to set LineDelimiter
> with "".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
