Thanks for taking this up, Jarek. I left a few comments on the pull request already.
Thanks & Regards, Amogh Desai On Mon, Dec 4, 2023 at 5:24 AM Jarek Potiuk <ja...@potiuk.com> wrote: > Hey here, > > In parallel to the serialisation discussion, I also wanted to make sure we. > raise the topic of common.io. It's been - similarly as serialization - > kind > of "low profile" - i.e. we've implemented it, discussed in the PRs and > added a new provider, but we never - I think - in the devlist discussed the > properties of it - why we are doing it, what is the context of it and what > are the perceived benefits of it.. > > I attempted to recover from all the past discussion the reasoning we have > it and what decisions have been made then: > > https://github.com/apache/airflow/pull/36015 > > Any comments are welcome. In short the ADR record there descibses: > > * using Common SQL operators (main goal) > * standardizing the common.sql DBApiHook API (needed to achieve the goal) > * describing the expected properties of data returned by the DBAPIHook > (clarifying how it will be used in operators) > * applicability of it for (future?) Open Lineage column-level lineage of > Hooks (possible future usage) > > Looking forward to comments - either in the PR (particularly) or in general > - here. > > J. >
