[
https://issues.apache.org/jira/browse/FLINK-36124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Somogyi updated FLINK-36124:
----------------------------------
Summary: S3RecoverableFsDataOutputStream.sync closes the stream and further
write operations throw exception (was: S3RecoverableFsDataOutputStream.sync
closes the stream and further write operation throw exception)
> S3RecoverableFsDataOutputStream.sync closes the stream and further write
> operations throw exception
> ---------------------------------------------------------------------------------------------------
>
> Key: FLINK-36124
> URL: https://issues.apache.org/jira/browse/FLINK-36124
> Project: Flink
> Issue Type: Bug
> Components: Connectors / AWS
> Affects Versions: 2.0.0
> Reporter: Gabor Somogyi
> Assignee: Gabor Somogyi
> Priority: Blocker
>
> This behaviour is introduced in FLINK-28513.
> Rationale why I think it's a bug:
> * `sync` method is defined in `FSDataOutputStream` with the following
> definition:
> {code:java}
> /**
> * Flushes the data all the way to the persistent non-volatile storage
> (for example disks). The
> * method behaves similar to the <i>fsync</i> function, forcing all data
> to be persistent on the
> * devices.
> *
> * @throws IOException Thrown if an I/O error occurs
> */
> {code}
> * In case `sync` method call user of the writer instance is expected to call
> further `write` methods
> * What is actually happening it's blowing up the next write with the
> following exception:
> {code:java}
> java.io.IOException: Stream closed.
> at
> org.apache.flink.core.fs.RefCountedFileWithStream.requireOpened(RefCountedFileWithStream.java:72)
> at
> org.apache.flink.core.fs.RefCountedFileWithStream.write(RefCountedFileWithStream.java:52)
> at
> org.apache.flink.core.fs.RefCountedBufferingFileStream.flush(RefCountedBufferingFileStream.java:104)
> at
> org.apache.flink.core.fs.RefCountedBufferingFileStream.write(RefCountedBufferingFileStream.java:87)
> at
> org.apache.flink.fs.s3.common.writer.S3RecoverableFsDataOutputStream.write(S3RecoverableFsDataOutputStream.java:112)
> at java.base/java.io.OutputStream.write(OutputStream.java:122)
> {code}
> * This can be super easily tested with
> `S3RecoverableFsDataOutputStreamTest.testSync`. Please remove the `expected =
> Exception.class` from the beginning of the test.
> * The following line in the test is testing nothing because never ever
> called:
> https://github.com/apache/flink/blob/56c81995d3b34ed9066b6771755407b93438f5ab/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/S3RecoverableFsDataOutputStreamTest.java#L264
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
