[
https://issues.apache.org/jira/browse/FLINK-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15724956#comment-15724956
]
ASF GitHub Bot commented on FLINK-3848:
---------------------------------------
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/2810
@tonycox, great. Thanks for the notice! Is the PR ready to review from your
point of view or are you still working on it?
@KurtYoung Thanks for reaching out to this PR. I'd like to propose the
following. I'll have a look at both PRs and see how we can maybe split the
effort into two commits. So, everybody gets a contribution out of this effort.
Regarding making `CsvTableSource` a `ProjectableTableSource`, it is true
that `CsvTableSource` will not benefit as much as other sources. Even with
pushed-down projections, it will still read all data and split it into rows.
However, we can save some time in field parsing (esp. tailing fields are not
parsed at all) and object creation (and later garbage collection). So, making
`CsvTableSource` a `ProjectableTableSource` will mainly safe some CPU time and
not IO. In addition, this is currently the only `BatchTableSource` and rather
easy to test.
> Add ProjectableTableSource interface and translation rule
> ---------------------------------------------------------
>
> Key: FLINK-3848
> URL: https://issues.apache.org/jira/browse/FLINK-3848
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
> Reporter: Fabian Hueske
> Assignee: Anton Solovev
>
> Add a {{ProjectableTableSource}} interface for {{TableSource}} implementation
> that support projection push-down.
> The interface could look as follows
> {code}
> def trait ProjectableTableSource {
> def setProjection(fields: Array[String]): Unit
> }
> {code}
> In addition we need Calcite rules to push a projection into a TableScan that
> refers to a {{ProjectableTableSource}}. We might need to tweak the cost model
> as well to push the optimizer in the right direction.
> Moreover, the {{CsvTableSource}} could be extended to implement
> {{ProjectableTableSource}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
