[
https://issues.apache.org/jira/browse/FLINK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552591#comment-16552591
]
Fabian Hueske commented on FLINK-9814:
--------------------------------------
We could certainly read the first line when generating input splits.
But as I said before, I would only do this for a single file and not every file
that is found. Otherwise, a single process (split generation) would block the
processing of the whole job.
Another issue is the formatting of the header. According to
[RFC4180|https://tools.ietf.org/html/rfc4180]:
{quote}
There maybe an optional header line appearing as the first line
of the file with the same format as normal record lines. This
header will contain names corresponding to the fields in the file
and should contain the same number of fields as the records in
the rest of the file (the presence or absence of the header line
should be indicated via the optional "header" parameter of this
MIME type). For example:
field_name,field_name,field_name CRLF
aaa,bbb,ccc CRLF
zzz,yyy,xxx CRLF
{quote}
only the field names may be encoded in the header.
> CsvTableSource "lack of column" warning
> ---------------------------------------
>
> Key: FLINK-9814
> URL: https://issues.apache.org/jira/browse/FLINK-9814
> Project: Flink
> Issue Type: Wish
> Components: Table API & SQL
> Affects Versions: 1.5.0
> Reporter: François Lacombe
> Assignee: vinoyang
> Priority: Minor
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> The CsvTableSource class is built by defining expected columns to be find in
> the corresponding csv file.
>
> It would be great to throw an Exception when the csv file doesn't have the
> same structure as defined in the source. For retro-compatibility sake,
> developers should explicitly set the builder to define columns stricly and
> expect Exception to be thrown in case of structure difference.
> It can be easilly checked with file header if it exists.
> Is this possible ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
