Thanks for the context on the Flink side! I think it sounds reasonable to
keep up to date with the latest supported Flink version. If we want, we
could later go with something similar to what we do for Spark but we’ll see
how it goes and what the Flink community needs. We should probably add a
sect
I think this is the right direction for plugging in other formats. This has
been suggested a few times and I fully support going this direction if
there is someone that wants to build it.
That said, I think there are some significant challenges to doing this that
I want to highlight:
1. For Pa