sharkdtu commented on a change in pull request #14813:
URL: https://github.com/apache/flink/pull/14813#discussion_r567551672
##########
File path:
flink-formats/flink-csv/src/test/java/org/apache/flink/formats/csv/CsvFormatFactoryTest.java
##########
@@ -206,6 +206,29 @@ public void testDeserializeWithEscapedFieldDelimiter()
throws IOException {
assertEquals(expected, actual);
}
+ @Test
+ public void testDeserializeWithDisableQuoteCharacter() throws IOException {
+ // test deserialization schema
+ final Map<String, String> options =
+ getModifiedOptions(
+ opts -> {
+ opts.put("csv.disable-quote-character", "true");
+ opts.remove("csv.quote-character");
+ });
+
+ final DynamicTableSource actualSource = createTableSource(options);
+ assert actualSource instanceof
TestDynamicTableFactory.DynamicTableSourceMock;
+ TestDynamicTableFactory.DynamicTableSourceMock sourceMock =
+ (TestDynamicTableFactory.DynamicTableSourceMock) actualSource;
+
+ DeserializationSchema<RowData> deserializationSchema =
+ sourceMock.valueFormat.createRuntimeDecoder(
+ ScanRuntimeProviderContext.INSTANCE,
SCHEMA.toRowDataType());
+ RowData expected = GenericRowData.of(fromString("\"abc"), 123, false);
+ RowData actual =
deserializationSchema.deserialize("\"abc;123;false".getBytes());
Review comment:
It can throw exception when 'csv.disable-quote-character' is set false
(default value), because it does not find closed quotation
of '\"abc'. But when 'csv.disable-quote-character' is set ture, it
shouldn't throw excetpion as it will ignore quote character. Actually, this
didn't take effect. So this pr fix this bug.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
