I’m following up on this topic.
David has a PR from last year that’s done much of the heavy lifting for
refactoring the codebase to be package-friendly.
https://github.com/apache/arrow/pull/13072
What’s changed since and what’s left:
* New components have been added (Flight SQL for example) that will need to
be updated for modules.
* There wasn’t a clear solution on how to do this without breaking JDK 8
support. Compiling module-info.java files require using JDK9, but using JDK9
breaks using JDK8 methods of accessing sun.misc.Unsafe.
* There is a Gradle plugin that can compile module-info.java files
purely syntactically that we can adapt to maven. It has limitations (the one I
see is that it can’t iterate through classloaders to handle annotations), but
using this might be a good stopgap until we JDK 8 support is deprecated.
* Some plugins need to be updated:
* maven-dependency-plugin 3.0.1 can’t parse module-info.class files.
* checkstyle 3.1.0 can’t parse module-info.java files. Our existing
checkstyle rules file can’t be loaded with newer versions. We can exclude
module-info.java for now and have a separate Issue for updating checkstyle
itself and the rules file.
* grpc-java could not be modularized when the PR above was written.
* Grpc 1.57 now can be modularized
(grpc/grpc-java#3522<https://github.com/grpc/grpc-java/issues/3522>)
From: David Dali Susanibar Arce <davi.sar...@gmail.com>
Date: Wednesday, May 25, 2022 at 5:02 AM
To: dev@arrow.apache.org <dev@arrow.apache.org>
Subject: [DISC][Java]: Migrate Arrow Java to JPMS Java Platform Module System
Hi All,
This email's purpose is a request for comments to migrate Arrow Java to JPMS
Java Platform Module System <https://openjdk.java.net/projects/jigsaw/spec/>
JSE 9+ (1).
Current status:
- Arrow Java use JSE1.8 specification
- Arrow Java works with JSE1.8/9/11/17
- This is possible because Java offers “legacy mode”
Proposal:
Migrate to JPMS Java Platform Module System. This Draft PR
<https://github.com/apache/arrow/pull/13072>(2<https://github.com/apache/arrow/pull/13072%3e(2>)
contains an initial port of
the modules: Format / Memory Core / Memory Netty / Memory Unsafe / Vector
for evaluation.
Main Reason to migrate:
- JPMS offer Strong encapsulation, Well-defined interfaces
<https://github.com/nipafx/demo-jigsaw-reflection>, Explicit dependencies.
<https://nipafx.dev/java-modules-reflection-vs-encapsulation/> (3)(4)
- JPMS offer reliable configuration and security to hide platform internals.
- JPMS offers a partial solution to solve problems about read (80%) /write
(20%) code.
- JPMS offer optimization for readability about read/write ratio (90/10)
thru module-info.java.
- Consistency logs, JPMS implement consistency logs to really use that to
solve the current problem.
- Be able to customize JRE needed with only modules needed (not
java.desktop for example and others) thru JLink.
- Modules have also been implemented by other languages such as Javascript
(ES2015), C++(C++20), Net (Nuget/NetCore)..
- Consider taking a look at this discussion about pros/cons
<https://www.reddit.com/r/java/comments/okt3j3/do_you_use_jigsaw_modules_in_your_java_projects/>
(5).
- Eventual migration to JPMS is a practical necessity as more projects
migrate.
Effort:
- First of all we need to decide to move from JSE1.8 to JSE9+ or be able to
offer support for both jar components JSE1.8 and JSE9+ included.
- Go bottom up for JPMS.
- Packages need to be unique (i.e. org.apache.arrow.memory /
io.netty.buffer). Review Draft PR with initial proposal.
- Dependencies also need to be modularized. If some of our current
dependencies are not able to be used as a module this will be a blocker for
our modules (we could patch that but this is an extra effort).
Killers:
- FIXME! I need your support to identify killer reasons to be able to push
this implementation.
Please let us know if Arrow Java to JPMS Java Platform Module System is
needed and should be implemented.
Please use this file for any comments
https://docs.google.com/document/d/1qcJ8LPm33UICuGjRnsGBcm8dLI08MyiL8BO5JVzTutA/edit?usp=sharing
Resources used:
(1): https://openjdk.java.net/projects/jigsaw/spec/
(2): https://github.com/apache/arrow/pull/13072
(3): https://nipafx.dev/java-modules-reflection-vs-encapsulation/
(4): https://github.com/nipafx/demo-jigsaw-reflection
(5):
https://www.reddit.com/r/java/comments/okt3j3/do_you_use_jigsaw_modules_in_your_java_projects/
Best regards,
--
David
