Hi all, Thanks to everyone for the discussion and for the detailed replies.
I agree that Java 21 as the minimum with full Java 25 support is a fair balance, and I am happy to go with Dongjoon's proposal. On Manu's concern: > downstream projects (e.g., Iceberg) and users on Spark 4.x would lack a clear > upgrade path without Java 21 support. I don't see Java support as a blocker for Iceberg. Trino has required Java 25 since Java 25 GA, and the Iceberg core module behind the Trino Iceberg connector has run on Java 25 without issue. Spark has supported Java 25 since 4.2.0 (SPARK-51167), so adapting the Iceberg Spark connector to Java 25 should be straightforward. Please let me know if you see a concrete issue. Thanks, Cheng Pan > On Nov 1, 2026, at 20:38, Yang Jie via dev <[email protected]> wrote: > > > +1 to Dongjoon's proposal (Java 21 as the minimum, with full Java 25 > support). > > On Cheng Pan's proposal, I tried to find solid reasons to require Java 25 and > couldn't find enough. > > I looked for benefits that only come with a Java 25 minimum, rather than just > running on Java 25. We could move datasketches-java to 9.x, which is built > for Java 25, and drop the SPARK-53327 patch. NativeLibrary.java could call > the FFM API directly instead of through method handles. And we could use > virtual threads more widely in Spark's own code: on Java 21 a virtual thread > that blocks inside synchronized pins its carrier thread, which JEP 491 fixed > in JDK 24. The first two are mostly about code maintenance, and for the last > one I can't give a measurable number yet. So far I don't see them as strong > enough reasons to require Java 25 for 5.0.0, and we can revisit the minimum > later. > > Thanks, > Jie Yang > > On 2026/10/30 08:05:41 Peter Toth wrote: >> Yes, my +1 was for the original proposal to drop Java 17 support. I >> think dropping Java 21 support in Spark 5 next year would be too >> early. >> Also, given the new Spark versioning policy, Spark 6 should be >> released in Q1 2028, so we'll have more frequent opportunities to >> reconsider dropping support for Java versions. >> >> On Thu, Oct 29, 2026 at 9:17 PM Mark Hamstra <[email protected]> wrote: >>> >>> Could we please get a little more than just naked +1s? >>> >>> I think Peter Toth and Hyukjin Kwon were giving their +1s to just >>> Dongjoon Hyun's proposal, but I'm not sure because we actually have >>> two proposals in play. I think that Cheng Pan makes a good argument >>> for considering Java 25 to be the minimum version for Spark 5.0, but >>> I'd like to see more discussion of that. We could have two separate >>> discussion threads on these separable proposals, but I think that they >>> can both be efficiently addressed within this one DISCUSS thread if >>> posters make it clear exactly which proposal they are commenting on. >>> >>> For the record, at this point I'm not committing to either proposal, >>> I'm closer to doing so for Dongjoon Hyun's, but I'm also leaning >>> toward Cheng Pan's. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: [email protected] >>> >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: [email protected] >> >> > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] >
