+1 on dropping Java 8 in Spark 4.0, saying this as a fan of the fast-paced (positive) updates to Arrow, eh?!
On Tue, Jun 6, 2023 at 4:02 PM Sean Owen <sro...@gmail.com> wrote: > I haven't followed this discussion closely, but I think we could/should > drop Java 8 in Spark 4.0, which is up next after 3.5? > > On Tue, Jun 6, 2023 at 2:44 PM David Li <lidav...@apache.org> wrote: > >> Hello Spark developers, >> >> I'm from the Apache Arrow project. We've discussed Java version support >> [1], and crucially, whether to continue supporting Java 8 or not. As Spark >> is a big user of Arrow in Java, I was curious what Spark's policy here was. >> >> If Spark intends to stay on Java 8, for instance, we may also want to >> stay on Java 8 or otherwise provide some supported version of Arrow for >> Java 8. >> >> We've seen dependencies dropping or planning to drop support. gRPC may >> drop Java 8 at any time [2], possibly this September [3], which may affect >> Spark (due to Spark Connect). And today we saw that Arrow had issues >> running tests with Mockito on Java 20, but we couldn't update Mockito since >> it had dropped Java 8 support. (We pinned the JDK version in that CI >> pipeline for now.) >> >> So at least, I am curious if Arrow could start the long process of >> migrating Java versions without impacting Spark, or if we should continue >> to cooperate. Arrow Java doesn't see quite so much activity these days, so >> it's not quite critical, but it's possible that these dependency issues >> will start to affect us more soon. And looking forward, Java is working on >> APIs that should also allow us to ditch the --add-opens flag requirement >> too. >> >> [1]: https://lists.apache.org/thread/phpgpydtt3yrgnncdyv4qdq1gf02s0yj >> [2]: >> https://github.com/grpc/proposal/blob/master/P5-jdk-version-support.md >> [3]: https://github.com/grpc/grpc-java/issues/9386 >> >
