[
https://issues.apache.org/jira/browse/FLINK-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552412#comment-16552412
]
vinoyang commented on FLINK-9812:
---------------------------------
Hi [~till.rohrmann] I have not analyzed the core problem of this issue. I just
have a look at this. The reason seems that we reused (write data) a closed
spilling channel (FileChannel). the exception point is here :
{code:java}
if (spillingChannel != null) {
// spill to file
ByteBuffer toWrite = segment.wrap(segmentPosition, toCopy);
this.spillingChannel.write(toWrite); //exception point
}{code}
There are two places may close the channel :
{code:java}
spillingChannel.close();
{code}
one is : SpanningWrapper#clear method:
{code:java}
if (spillingChannel != null) {
try {
spillingChannel.close();
}
catch (Throwable t) {
// ignore
}
spillingChannel = null;
}
{code}
this way, I think it's OK, because the close and set method come in pairs.
another is : addNextChunkFromMemorySegment
{code:java}
if (accumulatedRecordBytes == recordLength) {
// we have the full record
if (spillingChannel == null) {
this.serializationReadBuffer.setBuffer(buffer, 0, recordLength);
}
else {
spillingChannel.close();
BufferedInputStream inStream = new BufferedInputStream(new
FileInputStream(spillFile), 2 * 1024 * 1024);
this.spillFileReader = new DataInputViewStreamWrapper(inStream);
}
}
{code}
this way may be a potential problem point, the spillingChannel is closed but is
not been set to null here.
So, if in other place the spillingChannel is not been re-initialized(there are
some condition to call the init method "createSpillingChannel"), it will happen
this exception.
> SpanningRecordSerializationTest fails on travis
> -----------------------------------------------
>
> Key: FLINK-9812
> URL: https://issues.apache.org/jira/browse/FLINK-9812
> Project: Flink
> Issue Type: Bug
> Components: Network, Tests, Type Serialization System
> Affects Versions: 1.6.0
> Reporter: Chesnay Schepler
> Assignee: vinoyang
> Priority: Critical
> Fix For: 1.6.0
>
>
> https://travis-ci.org/zentol/flink/jobs/402744191
> {code}
> testHandleMixedLargeRecords(org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest)
> Time elapsed: 6.113 sec <<< ERROR!
> java.nio.channels.ClosedChannelException: null
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
> at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:199)
> at
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$SpanningWrapper.addNextChunkFromMemorySegment(SpillingAdaptiveSpanningRecordDeserializer.java:529)
> at
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$SpanningWrapper.access$200(SpillingAdaptiveSpanningRecordDeserializer.java:431)
> at
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.setNextBuffer(SpillingAdaptiveSpanningRecordDeserializer.java:76)
> at
> org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest.testSerializationRoundTrip(SpanningRecordSerializationTest.java:149)
> at
> org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest.testSerializationRoundTrip(SpanningRecordSerializationTest.java:115)
> at
> org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest.testHandleMixedLargeRecords(SpanningRecordSerializationTest.java:104)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
