Dear community, I'd like to initiate a discussion on the future release and support strategy for Hadoop. This includes a few important topics that I believe we should align on: 1. Ongoing Support for the Hadoop 3.4 Branch
As it stands, Hadoop 3.4 is the last release line that supports Java 8. It is expected that Java 8 compilation support will be officially removed from the trunk branch after a community vote. At the same time, the trunk branch is increasingly diverging from the 3.4 line due to the adoption of JUnit 5 and new features based on JDK17 syntax. As a result, future bug fixes and backports will become significantly more difficult. Given this, I’d like to ask: *after the release of hadoop-3.4.2, does the community plan to continue maintaining and releasing hadoop-3.4.3 or other 3.4-series versions?* 2. Preparation for Hadoop 3.5 Release With the JDK17 support work nearing completion, it’s time to start planning for the Hadoop 3.5 release. Specific questions for discussion include: - Should we set up a new CI pipeline for JDK17 on the trunk branch? - Should we also add support for JDK21 in CI? - If JDK21 support is added, can we remove the existing JDK11 pipeline? 3. Operating System Support Strategy Several major OS distributions—CentOS 7, CentOS 8, and Debian 10—have already reached or are approaching end-of-life (EOL). Should we adjust our support policy for these systems? For example: - Officially deprecate EOL systems; - Provide documentation-only support for compilation; - Remove outdated base images from Dockerfiles. 4. Release Strategy Under Multi-JDK Builds As we prepare to support JDK11, JDK17, and JDK21, we also need a clear strategy for future releases. Key questions include: - Should we publish multiple build variants like: hadoop-3.5.0-jdk11, hadoop-3.5.0-jdk17, hadoop-3.5.0-jdk21? - What naming convention should we adopt for these variants? - For Maven Central artifacts, should we distinguish by JDK version or consider a unified approach? Additional Context and Suggestions from My Side: 1. The JDK17 compatibility work is being tracked under HADOOP-17177 <https://issues.apache.org/jira/browse/HADOOP-17177>. I am in the process of reviewing and linking all relevant JIRAs—both completed and pending—to this umbrella JIRA, and will share an overview in the coming week. 2. Regarding the JUnit 5 migration: Based on our experience, JUnit 5 introduces significant differences compared to JUnit 4. It fundamentally changes how tests are written and run. My proposed upgrade path is as follows: - Complete the migration to *JUnit 5.8.2* (we are still working on HDFS and Hadoop-Azure and aim to finish in the next two weeks); - Remove JUnit 4 usage module by module (e.g., Yarn, MapReduce, HDFS, Common, etc.); - Upgrade to *JUnit 5.13.2* (as Steve suggested); - Upgrade the *Maven Surefire Plugin*. Earlier issues were due to the mixed use of JUnit 4 and 5, which will no longer be a concern once JUnit 5 is fully adopted; - Upgrade the *Maven version* to 3.9.0 for better compatibility and plugin support. 3. Once the JUnit 5 migration is complete, we will conduct a thorough test audit to ensure everything runs correctly and mark unstable tests for future attention. 4. Regarding legacy OS support, I recommend a "lightweight" approach: - Remove unsupported base images from our Dockerfiles or upgrade them to newer versions; - Provide documentation to help users build on older systems; - But *formally discontinue* full support for EOL systems. I’d appreciate hearing your thoughts on these topics. Looking forward to a fruitful discussion! Best regards, Shilun Fan. On Wed, Jun 4, 2025 at 7:23 AM slfan1989 <slfan1...@apache.org> wrote: > Hi, Chris > > Thank you very much for your efforts in driving the JDK17 upgrade. I will > continue to follow up on the JUnit5 migration, and I hope we can > successfully complete this upgrade together. > > Best Regards, > Shilun Fan. > > On Mon, Mar 24, 2025 at 1:16 PM slfan1989 <slfan1...@apache.org> wrote: > > > Hello everyone, > > > I've mentioned in other forums that I'd volunteer as release manager for > > Apache Hadoop 3.5.0, with a goal of full support for Java 17. I'd like to > > get more specific about release planning and a potential target date. > > > It can be easy to get lost in the nested historical JIRAs for Java 17 > > > support [1]. I'd like to start by summarizing my understanding of work done > > and work remaining for Java 17 as well as other significant features the > > community wants to ship in 3.5.0: > > > - We've completed the Jersey upgrade [2]. > > - We've made significant progress on the JUnit 5 upgrade [3], and we > > have a list of several PRs remaining to be reviewed and committed [4]. > > - There is some sun.misc > usage that needs alternate implementations [5] > > [6]. There are PRs, but they haven't had any recent activity. > > - Once these issues are settled, we need to (re-)introduce a Java 17 > > build profile [7]. > > - I and a few others are also interested in trying to target the GCS > > FileSystem implementation for 3.5.0 [8]. > > > Does this look accurate? Did I miss anything important for 3.5.0, either > > for Java 17 or other features? > > > Chris Nauroth > > > [1] https://issues.apache.org/jira/browse/HADOOP-17177 > > [2] https://issues.apache.org/jira/browse/HADOOP-15984 > > [3] https://issues.apache.org/jira/browse/HADOOP-14693 > > [4] https://github.com/apache/hadoop/pull/7337#issuecomment-2857464756 > > [5] https://issues.apache.org/jira/browse/HADOOP-19329 > > [6] https://issues.apache.org/jira/browse/HADOOP-19334 > > [7] https://issues.apache.org/jira/browse/HADOOP-19298 > > [8] https://issues.apache.org/jira/browse/HADOOP-19343 >
