Henning Schmiedehausen created FLINK-34581:
----------------------------------------------
Summary: streaming code throws
java.lang.reflect.InaccessibleObjectException
Key: FLINK-34581
URL: https://issues.apache.org/jira/browse/FLINK-34581
Project: Flink
Issue Type: Bug
Components: API / DataStream
Affects Versions: 1.18.1
Reporter: Henning Schmiedehausen
I have a pretty simple test pipeline (read a bunch of tables from Apache Kafka,
join and project them, then write to Apache Iceberg) that I run locally with
Junit5 and the MiniClusterExtension. This works ok with Java 11 and Flink
1.17.2 and Flink 1.18.1.
With Java 17, I see
{{[WARN ] IcebergStreamWriter (1/1)#1
(78d9251dbbab3aae84bf303dfc080d23_626c1e687bcaad0c13c507629a5894a8_0_1)
switched from RUNNING to FAILED with failure cause:}}
{{java.io.IOException: Could not perform checkpoint 2 for operator
IcebergStreamWriter (1/1)#1.}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1275)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:147)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerCheckpoint(SingleCheckpointBarrierHandler.java:287)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.access$100(SingleCheckpointBarrierHandler.java:64)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler$ControllerImpl.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:488)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.triggerGlobalCheckpoint(AbstractAlignedBarrierHandlerState.java:74)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.barrierReceived(AbstractAlignedBarrierHandlerState.java:66)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.lambda$processBarrier$2(SingleCheckpointBarrierHandler.java:234)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.markCheckpointAlignedAndTransformState(SingleCheckpointBarrierHandler.java:262)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:231)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:181)}}
{{at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:159)}}
{{at
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:118)}}
{{at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)}}
{{at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)}}
{{at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)}}
{{at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)}}
{{at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)}}
{{at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)}}
{{at java.base/java.lang.Thread.run(Thread.java:840)}}
{{Caused by: java.lang.RuntimeException:
java.lang.reflect.InaccessibleObjectException: Unable to make field private
final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module
java.base does not "opens java.util" to unnamed module @6af93788}}
{{at
com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:69)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.FlinkChillPackageRegistrar.registerSerializers(FlinkChillPackageRegistrar.java:67)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.getKryoInstance(KryoSerializer.java:513)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.checkKryoInitialized(KryoSerializer.java:522)}}
{{at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:348)}}
{{at
org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:165)}}
{{at
org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:43)}}
{{at
org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:54)}}
{{at
org.apache.flink.runtime.io.network.api.writer.RecordWriter.serializeRecord(RecordWriter.java:141)}}
{{at
org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:107)}}
{{at
org.apache.flink.runtime.io.network.api.writer.ChannelSelectorRecordWriter.emit(ChannelSelectorRecordWriter.java:55)}}
{{at
org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:134)}}
{{at
org.apache.flink.streaming.runtime.io.RecordWriterOutput.collectAndCheckIfChained(RecordWriterOutput.java:114)}}
{{at
org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:95)}}
{{at
org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:48)}}
{{at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:59)}}
{{at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:31)}}
{{at
org.apache.iceberg.flink.sink.IcebergStreamWriter.flush(IcebergStreamWriter.java:113)}}
{{at
org.apache.iceberg.flink.sink.IcebergStreamWriter.prepareSnapshotPreBarrier(IcebergStreamWriter.java:66)}}
{{at
org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.prepareSnapshotPreBarrier(RegularOperatorChain.java:89)}}
{{at
org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:322)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$15(StreamTask.java:1318)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1306)}}
{{at
org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1263)}}
{{... 22 more}}
{{Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
field private final java.lang.Object[] java.util.Arrays$ArrayList.a accessible:
module java.base does not "opens java.util" to unnamed module @6af93788}}
{{at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)}}
{{at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)}}
{{at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)}}
{{at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)}}
{{at
com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:67)}}
{{... 46 more}}
This is blocking us right now from upgrading to Java 17.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
