Thanks for the feedback Fabian. I updated the document and addressed
your comments.
I agree that tables which are stored in different systems need more
discussion. I would suggest to deprecate the field mapping interfaces in
this release and remove it in the next release.
Regards,
Timo
Am 02.10.18 um 11:06 schrieb Fabian Hueske:
Thanks for the proposal Timo!
I've done a pass and added some comments (mostly asking for clarification,
details).
Overall, this is going into a very good direction.
I think the tables which are stored in different systems and using a format
definition to define other formats require some more discussions.
However, these are also not the features that we would start with.
>From a compatibility point of view, an important question to answer would
be whether we can drop the support for field mapping, i.e., do we have
users who take advantage of mapping format fields to fields with a
different name in the schema.
Besides that, all existing functionality is preserved although the syntax
changes a bit.
Best,
Fabian
Am Mo., 1. Okt. 2018 um 10:53 Uhr schrieb Timo Walther <twal...@apache.org>:
Hi everyone,
as some of you might have noticed, in the last two releases we aimed to
unify SQL connectors and make them more modular. The first connectors
and formats have been implemented and are usable via the SQL Client and
Java/Scala/SQL APIs.
However, after writing more connectors/example programs and talking to
users, there are still a couple of improvements that should be applied
to unified SQL connector API.
I wrote a design document [1] that discusses limitations that I have
observed and consideres feedback that I have collected over the last
months. I don't know whether we will implement all of these
improvements, but it would be great to get feedback for a satisfactory
API and for future priorization.
The general goal should be to connect to external systems as convenient
and type-safe as possible. Any feedback is highly appreciated.
Thanks,
Timo
[1]
https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit?usp=sharing
