Re: [DISCUSS] Spark version support strategy

2021-10-03 Thread Ryan Blue
Thanks for the context on the Flink side! I think it sounds reasonable to keep up to date with the latest supported Flink version. If we want, we could later go with something similar to what we do for Spark but we’ll see how it goes and what the Flink community needs. We should probably add a sect

Re: support of RCFile

2021-10-03 Thread Ryan Blue
I think this is the right direction for plugging in other formats. This has been suggested a few times and I fully support going this direction if there is someone that wants to build it. That said, I think there are some significant challenges to doing this that I want to highlight: 1. For Pa