Re: Dataset read csv file problem

2017-11-27 Thread ebru
Thank you Fabian, we’ve implemented a custom CsvInputFormat. > On 24 Nov 2017, at 15:35, Fabian Hueske wrote: > > Hi Ebru, > > this case is not supported by Flink's CsvInputFormat. The problem is that > such a file could not be read in parallel because it is not possible to > identify record

Re: Dataset read csv file problem

2017-11-24 Thread Fabian Hueske
Hi Ebru, this case is not supported by Flink's CsvInputFormat. The problem is that such a file could not be read in parallel because it is not possible to identify record boundaries if you start reading in the middle of the file. We have a new CsvInputFormat under development that follows the RFC

Dataset read csv file problem

2017-11-24 Thread ebru
Hello all, We are trying to read csv files which contains fields containing \n character, also \n character is line delimiter. We used parseQuotedStrings('\"') Method but, it ignores only field delimiters so we couldn’t parse the fields that contains \n character. How can we solve this problem