Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/2762
Hi @mushketyk, sorry for the concise description before. The problem is the
following:
A `TableSource` provides schema information for the Table it produces.
However, the methods `TableSource.getFieldNames()` and
`TableSouce.getFieldTypes()` return flat arrays which are interpreted as flat
schema (the second field in the array represents the second table attribute)
without any nesting.
Avro and many other storage formats (JSON, Parquet, ...) support nested
data structures. With the current limitation of the `TableSource` interface, we
would need to convert the nested data into a flat schema. However, the Table
API and SQL support processing of nested data and it would be a much better
integration to pass Avro objects in their original structure into the Table API
/ SQL query (see my updated proposal for
[FLINK-3871](https://issues.apache.org/jira/browse/FLINK-3871).
In order to be able to create a Table of nested Avro data, we need to
improve the `TableSource` interface first. Once this is done, we can continue
with this PR.
I'm very sorry, that I did not think about this earlier and the effort you
already put into this issue.
Please let me know if you have any questions.
Best, Fabian
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
