LinkedIn is still on Spark 3.1. I am guessing a number of other companies could be in the same boat. I feel the argument for Spark 2.4 is different from that of Spark 3.1 and it would be great if we can continue to support 3.1 for some time.
On Wed, Apr 19, 2023 at 11:06 AM Ryan Blue <b...@tabular.io> wrote: > +1 > > As we said in the 2.4 discussion, the format itself should provide forward > compatibility with tables and it is more clear that we aren't adding new > features if you have to use older versions for Spark 3.1. > > On Wed, Apr 19, 2023 at 10:08 AM Anton Okolnychyi > <aokolnyc...@apple.com.invalid> wrote: > >> Hey folks, >> >> What does everybody think about Spark 3.1 support after we add Spark 3.4 >> support? Our initial plan was to release jars for the last 3 versions. Are >> there any blockers for dropping 3.1? >> >> - Anton > > > > -- > Ryan Blue > Tabular >
