Thanks, everyone, for starting this discussion. This sounds like a good plan to start with. As I was working on the JUnit 4 to 5 upgrade, I paused for a while as I got occupied with other stuff. But I would be happy to complete it as part of the Java switch process.
On Tue, Jul 30, 2024 at 4:21 AM slfan1989 <slfan1...@apache.org> wrote: > Thank you very much for initiating this discussion! I am also very much > looking forward to JDK 17. I have observed that using --add-opens= is quite > common in other Apache projects. One thing I know is that it is necessary > to upgrade JUnit 4 to JUnit 5. There are some JIRA issues currently being > worked on for this. For YARN MapReduce, some pull requests have already > been completed for this feature. Upgrading Jersey is a very challenging > project. Currently, our upgrade strategy is to perform it all at once. If > we can find a way to break Jersey down into individual modules for > upgrading, it might be much easier. > > I have successfully debugged most of the unit tests for YARN's > ResourceManager and NodeManager.(Upgrade Jersey 2.4.1) Additionally, we > are using Hadoop compiled with JDK 17 internally (mainly on the current > trunk branch with --add-opens=). We have replaced over 600 NM instances, > and this part is functioning normally. Therefore, I believe we are > essentially ready to upgrade to JDK 17. > > My idea is to create a separate branch specifically for upgrading to JDK > 17. After debugging and ensuring everything works, we can then merge this > branch into the trunk branch. > > By the way, there's a JIRA that we have had to revert twice > (HADOOP-19071)[1], which involves upgrading the maven-surefire-plugin. > > [1] https://issues.apache.org/jira/browse/HADOOP-19071 > > - Shilun Fan > > On Tue, 30 Jul 2024 at 06:37, Shilun Fan wrote: > > > Depends on what we mean by switching to JDK-17, Compile time support > > or Runtime support, We don't have compile time support for JDK-11 too, > > it is just runtime, We have a daily build as well for JDK-11, which > > has now some genuine failures now, need to check [1], as of now the > > version is hardcoded here [2], unless we change here it but just > > change our local JAVA_HOME, it will compile with JDK-17 as well I > > believe > > > So, If we are to chase the runtime support we can setup a regular CI > > with JDK-17 like we have one for JDK-11, there were some test failures > > reported as part of HADOOP-18716, most of them should be fixable by > > some dependency upgrades or some hacks like adding one of these > > ``` > > --add-opens=java.base/java.net=ALL-UNNAMED > > --add-opens=java.base/java.util=ALL-UNNAMED > > --add-opens=java.base/java.util.concurrent=ALL-UNNAMED > > --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED > > --add-opens=java.base/java.lang=ALL-UNNAMED > > --add-opens=java.base/java.util.regex=ALL-UNNAMED > > --add-opens=java.base/java.io=ALL-UNNAMED > > ``` > > > That is some typical JDK-17 hack.... > > > For the compile support, that is we change the value at [2] to 17, I > > think then some bigger problems will surface, One I know is Jersey > > creates some issues for sure, We need to upgrade Jersey, As per this > > [3] to Jersey 2.35+ or do some hack to use a patched jersey version > > with minimal change which can unblock the JDK-17 initiative > > > -Ayush > > > > [1] > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/709/ > > [2] > > https://github.com/apache/hadoop/blob/038636a1b5250e06622cac7ee11b12965c91111e/hadoop-project/pom.xml#L160 > > [3] https://github.com/eclipse-ee4j/jersey/wiki/Road-Map > > On Tue, 30 Jul 2024 at 00:53, PJ Fanning wrote: > > > > I think this is worth considering. I think it would require a minor > > release like 3.5.0 as opposed to considering it for future 3.4.x patch > > releases. > > I tend to build locally with Java 11, by default and I haven't hit > > major issues building Hadoop. There may be some gotcha somewhere but > > it is likely to be easy enough to fix. > > > > It's probably only a matter of time before some important dependency > > jars for Hadoop require Java 11 or 17. > > > > On Mon, 29 Jul 2024 at 20:04, Steve Loughran > > wrote: > > > > > > A lot of projects are moving off java8. making java17 the new baseline > > > > > > what do we need to there that is blocker rather than just "nice"'? > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org >
