[
https://issues.apache.org/jira/browse/FLINK-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-25002:
-------------------------------------
Description:
Java 17 actually enforces the encapsulation of the JDK (opposed to Java 11
which just printed warnings), requiring us to explicitly open/export any
package that we access illegally.
The following is a list of opens/exports that I needed to get most tests to
pass, also with some comments which component needed them. Overall the
ClosureCleaner and FieldSerializer result in the most offenses, as they try to
access private fields.
These properties need to be set _for all JVMs in which we run Flink_, including
surefire forks, other tests processes (TestJvmProcess/TestProcessBuilder/Yarn)
and the distribution.
This needs some thought on how we can share this list across poms (surefire),
code (test processes / yarn) and the configuration (distribution).
{code:xml}
<surefire.module.config> <!--
-->--add-exports java.base/sun.net.util=ALL-UNNAMED <!--
required by JmxServer
-->--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
<!--
-->--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED <!--
HadoopUtilsTest
-->--add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED <!--
-->--add-opens java.base/java.lang=ALL-UNNAMED <!--
RocksDBAsyncSnapshotTest Whitebox SerializedThrowable
-->--add-opens java.base/java.lang.invoke=ALL-UNNAMED <!--
RoundingMode
-->--add-opens java.base/java.math=ALL-UNNAMED <!--
-->--add-opens java.base/java.net=ALL-UNNAMED <!--
InitOutputPathTest
-->--add-opens java.base/java.io=ALL-UNNAMED <!--
InitOutputPathTest
-->--add-opens java.base/java.lang.reflect=ALL-UNNAMED <!--
ClosureCleaner Timestamp
-->--add-opens java.sql/java.sql=ALL-UNNAMED <!--
lz4 Buffer
-->--add-opens java.base/java.nio=ALL-UNNAMED <!--
ClosureCleaner SimpleDateFormat
-->--add-opens java.base/java.text=ALL-UNNAMED <!--
FieldSerializer LocalDate
-->--add-opens java.base/java.time=ALL-UNNAMED <!--
ExpressionKeysTest
-->--add-opens java.base/java.util=ALL-UNNAMED <!--
AsynchronousFileIOChannelTest
-->--add-opens java.base/java.util.concurrent=ALL-UNNAMED <!--
Kryo FieldSerializer
-->--add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED <!--
ClosureCleaner ReentrantLock
-->--add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED <!--
Whitebox ReferencePipeline$Head
-->--add-opens java.base/java.util.stream=ALL-UNNAMED <!--
ClosureCleaner ZoneInfo
-->--add-opens java.base/sun.util.calendar=ALL-UNNAMED
</surefire.module.config>
{code}
Additionally, the following JVM arguments must be supplied when running Maven:
{code}
export MAVEN_OPTS="\
--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
--add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED"
{code}
was:
Java 17 actually enforces the encapsulation of the JDK (opposed to Java 11
which just printed warnings), requiring us to explicitly open/export any
package that we access illegally.
The following is a list of opens/exports that I needed to get most tests to
pass, also with some comments which component needed them. Overall the
ClosureCleaner and FieldSerializer result in the most offenses, as they try to
access private fields.
These properties need to be set _for all JVMs in which we run Flink_, including
surefire forks, other tests processes (TestJvmProcess/TestProcessBuilder/Yarn)
and the distribution.
This needs some though on how we can share this list across poms (surefire),
code (test processes / yarn) and the configuration (distribution).
{code:xml}
<surefire.module.config> <!--
-->--add-exports java.base/sun.net.util=ALL-UNNAMED <!--
required by JmxServer
-->--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
<!--
-->--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED <!--
-->--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED <!--
HadoopUtilsTest
-->--add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED <!--
-->--add-opens java.base/java.lang=ALL-UNNAMED <!--
RocksDBAsyncSnapshotTest Whitebox SerializedThrowable
-->--add-opens java.base/java.lang.invoke=ALL-UNNAMED <!--
RoundingMode
-->--add-opens java.base/java.math=ALL-UNNAMED <!--
-->--add-opens java.base/java.net=ALL-UNNAMED <!--
InitOutputPathTest
-->--add-opens java.base/java.io=ALL-UNNAMED <!--
InitOutputPathTest
-->--add-opens java.base/java.lang.reflect=ALL-UNNAMED <!--
ClosureCleaner Timestamp
-->--add-opens java.sql/java.sql=ALL-UNNAMED <!--
lz4 Buffer
-->--add-opens java.base/java.nio=ALL-UNNAMED <!--
ClosureCleaner SimpleDateFormat
-->--add-opens java.base/java.text=ALL-UNNAMED <!--
FieldSerializer LocalDate
-->--add-opens java.base/java.time=ALL-UNNAMED <!--
ExpressionKeysTest
-->--add-opens java.base/java.util=ALL-UNNAMED <!--
AsynchronousFileIOChannelTest
-->--add-opens java.base/java.util.concurrent=ALL-UNNAMED <!--
Kryo FieldSerializer
-->--add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED <!--
ClosureCleaner ReentrantLock
-->--add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED <!--
Whitebox ReferencePipeline$Head
-->--add-opens java.base/java.util.stream=ALL-UNNAMED <!--
ClosureCleaner ZoneInfo
-->--add-opens java.base/sun.util.calendar=ALL-UNNAMED
</surefire.module.config>
{code}
Additionally, the following JVM arguments must be supplied when running Maven:
{code}
export MAVEN_OPTS="\
--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
--add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED"
{code}
> Setup required --add-opens/--add-exports
> ----------------------------------------
>
> Key: FLINK-25002
> URL: https://issues.apache.org/jira/browse/FLINK-25002
> Project: Flink
> Issue Type: Sub-task
> Components: Build System, Build System / CI, Documentation, Tests
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Major
>
> Java 17 actually enforces the encapsulation of the JDK (opposed to Java 11
> which just printed warnings), requiring us to explicitly open/export any
> package that we access illegally.
> The following is a list of opens/exports that I needed to get most tests to
> pass, also with some comments which component needed them. Overall the
> ClosureCleaner and FieldSerializer result in the most offenses, as they try
> to access private fields.
> These properties need to be set _for all JVMs in which we run Flink_,
> including surefire forks, other tests processes
> (TestJvmProcess/TestProcessBuilder/Yarn) and the distribution.
> This needs some thought on how we can share this list across poms (surefire),
> code (test processes / yarn) and the configuration (distribution).
> {code:xml}
> <surefire.module.config> <!--
> -->--add-exports java.base/sun.net.util=ALL-UNNAMED <!--
> required by JmxServer
> -->--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED <!--
> -->--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED <!--
> -->--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED <!--
> -->--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
> <!--
> -->--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED <!--
> -->--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED <!--
> HadoopUtilsTest
> -->--add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED <!--
> -->--add-opens java.base/java.lang=ALL-UNNAMED <!--
> RocksDBAsyncSnapshotTest Whitebox SerializedThrowable
> -->--add-opens java.base/java.lang.invoke=ALL-UNNAMED <!--
> RoundingMode
> -->--add-opens java.base/java.math=ALL-UNNAMED <!--
> -->--add-opens java.base/java.net=ALL-UNNAMED <!--
> InitOutputPathTest
> -->--add-opens java.base/java.io=ALL-UNNAMED <!--
> InitOutputPathTest
> -->--add-opens java.base/java.lang.reflect=ALL-UNNAMED <!--
> ClosureCleaner Timestamp
> -->--add-opens java.sql/java.sql=ALL-UNNAMED <!--
> lz4 Buffer
> -->--add-opens java.base/java.nio=ALL-UNNAMED <!--
> ClosureCleaner SimpleDateFormat
> -->--add-opens java.base/java.text=ALL-UNNAMED <!--
> FieldSerializer LocalDate
> -->--add-opens java.base/java.time=ALL-UNNAMED <!--
> ExpressionKeysTest
> -->--add-opens java.base/java.util=ALL-UNNAMED <!--
> AsynchronousFileIOChannelTest
> -->--add-opens java.base/java.util.concurrent=ALL-UNNAMED <!--
> Kryo FieldSerializer
> -->--add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED <!--
> ClosureCleaner ReentrantLock
> -->--add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED <!--
> Whitebox ReferencePipeline$Head
> -->--add-opens java.base/java.util.stream=ALL-UNNAMED <!--
> ClosureCleaner ZoneInfo
> -->--add-opens java.base/sun.util.calendar=ALL-UNNAMED
> </surefire.module.config>
> {code}
> Additionally, the following JVM arguments must be supplied when running Maven:
> {code}
> export MAVEN_OPTS="\
> --add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \
> --add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED \
> --add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \
> --add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
> --add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
> --add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED"
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
