Hi Laurent, These are good ideas worth evaluating. I am also on board with waiting for user feedback after the Java 8 deprecation is released. It is important to strike the right balance between supporting the broader community and allowing developers to improve the project with newer technologies to stay relevant. The quick push to Java 17 mainly originates from the general consensus among other related Java projects (e.g. Spark, Iceberg, Avro) that have discussed migrating directly from Java 8 to Java 17. Perhaps we revisit this discussion in 3-6 months to see how the Java ecosystem has taken to these changes?
Thanks, Dane On Wed, Aug 21, 2024 at 6:31 PM Laurent Goujon <laur...@dremio.com.invalid> wrote: > There's another path which is to have arrow core still targeting Java 11 > but having the arrow orc module targeting Java 17 for example. Or like I > also mentioned in the other thread, leverage MRJAR capability to offer > support for recent java APIs while maintaining compatibility for those > users who cannot migrate out of Java 11 yet. I even contributed some of the > work to support it but we postponed it due to the lack of immediate use for > it (if I interpret correctly the feedback I received). > > I guess I'm a bit surprised by this push to drop support for older java > versions which are still supported and used (even if less and less) while > we also trying very hard to not ditch support for RHEL7/CentOS 7 for > example, something i discovered when we realized that grpc proto plugin was > not compatible with this OS anymore (discussed at > https://github.com/apache/arrow/pull/43264, and mentioned in > https://github.com/apache/arrow/issues/40735, but at the same time we > upgraded grpc in https://github.com/apache/arrow/pull/43657?). In > comparison, even if it's kind of true that Java 8, 11 and soon 17 may not > see new features, they are still receiving fixes and security updates from > multiple vendors, including Adoptium which are planning new releases for > Java 8 and 11 in october (and availability until 2026 for java 8 and 2027 > for java 11). > I don't want to tune down the desire of try and adopt latest technology, > but I'm kindly asking for the whole community to be considered and to > discuss possible alternatives to achieve the same goals? > > Laurent > > On Wed, Aug 21, 2024 at 9:43 AM Dane Pitkin <dpit...@apache.org> wrote: > > > I realized I'm repeating myself a bit here with my reasoning, my > apologies. > > I'll emphasize that the dependencies are the strongest factor, in that > over > > time we have to pin more and more dependencies to older versions. I think > > code/stack modernization would be nice and the biggest feature I'm > looking > > forward to is Arrow integration with the new FFM APIs in Java 22. > However, > > modernization hasn't been the biggest priority yet mainly due to > > contributor capacity. I imagine if we start filing GitHub issues with > > improvement ideas, contributors might start picking them up over time. > > > > On Wed, Aug 21, 2024 at 12:01 PM Dane Pitkin <dpit...@apache.org> wrote: > > > > > Hi Laurent, > > > > > > Kudos again for doing the migration from Java 8 -> 11 in Arrow. That > was > > a > > > large contribution, thank you! > > > > > > I see a few reasons to drop Java 11: > > > - Java 11 official support ended September 2023, but it will receive > > > extended support until 2032. > > > - Dependencies are upgrading at a faster pace. For example, ORC v2.0 > > > supports Java 17+. > > > > > > Thanks, > > > Dane > > > > > > > > > On Fri, Aug 16, 2024 at 11:51 AM Laurent Goujon > > <laur...@dremio.com.invalid> > > > wrote: > > > > > >> I think like last time we should see where the community is as a whole > > on > > >> JDK version support? Just looking at Spark may not be enough to get a > > >> sense > > >> that it is okay to drop Java 11. Overall I think we should at least > > give 2 > > >> major versions before removing support. > > >> > > >> When I did the migration from Java 8 to Java 11, most of the updates > > were > > >> on the toolchain but I haven't seen a lot of code changes which would > > not > > >> compile with Java 8. And so I'm not sure what we expect from Java 17 > in > > >> terms of code changes and or stack/modernization. @Dane, could you > > >> elaborate maybe? > > >> > > >> Cheers, > > >> > > >> Laurent > > >> > > >> On Tue, Aug 6, 2024 at 5:30 PM Vibhatha Abeykoon <vibha...@gmail.com> > > >> wrote: > > >> > > >> > Thanks Dane for once again pushing the topic on Java language > support. > > >> > In terms of project maintenance and long term growth, I am happy > with > > >> this > > >> > change. > > >> > > > >> > Regarding the usage of `--add-opens`, it would still be okay, given > > the > > >> > fact that we provided that option either way. > > >> > But in future, I think we should figure out a way to do this better. > > >> > > > >> > Although as you suggested, it would be best to gather feedback from > > the > > >> > community, > > >> > specifically that we have already done a minimum version upgrade > very > > >> > recently. > > >> > Also, it would be better to do this around v20 or v21 if we agree to > > >> move > > >> > forward. The reason > > >> > is it would at least give enough time for some users to get ready > for > > a > > >> > change. Again this may require > > >> > consensus in the community. Also we can take another consensus from > > the > > >> > github dependabot PRs, > > >> > it provides a hint on how much technical burden and vulnerabilities > we > > >> have > > >> > to keep up when we don't > > >> > upgrade the minimal supported Java version. I think we had a good > > >> > experience in gathering those details for Java 8 [1] (citing once > > more). > > >> > > > >> > +1 for this proposed change. > > >> > > > >> > [1] https://github.com/apache/arrow/issues/38051 > > >> > > > >> > On Thu, Aug 1, 2024 at 2:17 AM Jacob Wujciak <assignu...@apache.org > > > > >> > wrote: > > >> > > > >> > > Thanks Dane for starting the discussion! > > >> > > I would be +1 but I am neither a Java user nor familiar with the > > space > > >> > but > > >> > > seeing spark go 17+ is encouraging. > > >> > > > > >> > > Also worth mentioning is that people that can't drop 11 can always > > >> > continue > > >> > > using the versions that still support it. > > >> > > > > >> > > Best > > >> > > Jacob > > >> > > > > >> > > Am Mi., 31. Juli 2024 um 19:00 Uhr schrieb Dane Pitkin < > > >> > dpit...@apache.org > > >> > > >: > > >> > > > > >> > > > Hi all, > > >> > > > > > >> > > > I'd like to bring up for discussion dropping Java 11 and > > supporting > > >> > Java > > >> > > 17 > > >> > > > as the minimum version[1]. Earlier this year we agreed to drop > > Java > > >> 8 > > >> > and > > >> > > > support Java 11 as the min version[2]. That has now been > completed > > >> and > > >> > > will > > >> > > > be released in Arrow v18 [3]. > > >> > > > > > >> > > > My suggestion would be to drop Java 11 in Arrow v19 (~Jan 2025). > > If > > >> we > > >> > > want > > >> > > > to wait for feedback from users after we release removal of Java > > 8, > > >> > then > > >> > > > perhaps Arrow v20 (~Apr 2025). > > >> > > > > > >> > > > > > >> > > > Some reasonings: > > >> > > > - Java 11 is now in Extended Support for the remainder of its > > >> lifecycle > > >> > > > - Apache Spark only supports Java 17+ in v4.X > > >> > > > > > >> > > > Some drawbacks: > > >> > > > - Users will be required to add java command line arguments > > >> > > > (--add-opens)[4]. > > >> > > > > > >> > > > > > >> > > > Overall, this could be a big step towards modernizing the Arrow > > Java > > >> > > > project. > > >> > > > > > >> > > > Thanks, > > >> > > > Dane > > >> > > > > > >> > > > > > >> > > > [1]https://github.com/apache/arrow/issues/43307 > > >> > > > [2] > > https://lists.apache.org/thread/65vqpmrrtpshxo53572zcv91j1lb2y8g > > >> > > > [3]https://github.com/apache/arrow/issues/38051 > > >> > > > [4]https://arrow.apache.org/docs/java/install.html#id3 > > >> > > > > > >> > > > > >> > > > >> > > > > > >
