[
https://issues.apache.org/jira/browse/FLINK-37807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski closed FLINK-37807.
----------------------------------
Resolution: Fixed
> Report s5cmd cancellation in a nicer way
> ----------------------------------------
>
> Key: FLINK-37807
> URL: https://issues.apache.org/jira/browse/FLINK-37807
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 2.0.0
> Reporter: Piotr Nowojski
> Assignee: Piotr Nowojski
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.1.0
>
>
> If s5cmd is cancelled it logs for example:
> {noformat}
> java.io.IOException: Failed to cast s5cmd spell [/opt/s5cmd -r 3 --numworkers
> 5 run] [exit code = 1] [cfg: S5CmdConfiguration{path='/opt/s5cmd', args=[-r,
> 3, --numworkers, 5], accessArtifact='null', secretArtifact='null',
> endpoint='null'}] maybe due to:
> at
> org.apache.flink.fs.s3.common.FlinkS3FileSystem.castSpell(FlinkS3FileSystem.java:379)
> at
> org.apache.flink.fs.s3.common.FlinkS3FileSystem.copyFiles(FlinkS3FileSystem.java:296)
> at
> org.apache.flink.core.fs.PluginFileSystemFactory$ClassLoaderFixingFileSystem.copyFiles(PluginFileSystemFactory.java:157)
> at
> org.apache.flink.core.fs.monitoring.MonitoringFileSystem.copyFiles(MonitoringFileSystem.java:86)
> at
> org.apache.flink.core.fs.SafetyNetWrapperFileSystem.copyFiles(SafetyNetWrapperFileSystem.java:56)
> at
> org.apache.flink.core.fs.bandwidth.BandwidthLimitingFileSystem.copyFiles(BandwidthLimitingFileSystem.java:202)
> at
> org.apache.flink.contrib.streaming.state.RocksDBStateDownloader.lambda$createDownloadRunnableUsingCopyFiles$3(RocksDBStateDownloader.java:247)
> at
> org.apache.flink.util.function.ThrowingRunnable.lambda$unchecked$0(ThrowingRunnable.java:49)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: java.io.IOException: Copy process destroyed by CloseableRegistry.
> at
> org.apache.flink.fs.s3.common.FlinkS3FileSystem.lambda$castSpell$0(FlinkS3FileSystem.java:362)
> at org.apache.flink.util.IOUtils.closeQuietly(IOUtils.java:295)
> at org.apache.flink.util.IOUtils.closeAllQuietly(IOUtils.java:282)
> at
> org.apache.flink.core.fs.CloseableRegistry.doClose(CloseableRegistry.java:65)
> at
> org.apache.flink.util.AbstractAutoCloseableRegistry.close(AbstractAutoCloseableRegistry.java:127)
> at org.apache.flink.util.IOUtils.closeQuietly(IOUtils.java:295)
> at org.apache.flink.util.IOUtils.closeAllQuietly(IOUtils.java:282)
> at
> org.apache.flink.core.fs.CloseableRegistry.doClose(CloseableRegistry.java:65)
> at
> org.apache.flink.util.AbstractAutoCloseableRegistry.close(AbstractAutoCloseableRegistry.java:127)
> at org.apache.flink.util.IOUtils.closeQuietly(IOUtils.java:295)
> at org.apache.flink.util.IOUtils.closeAllQuietly(IOUtils.java:282)
> at
> org.apache.flink.core.fs.CloseableRegistry.doClose(CloseableRegistry.java:65)
> at
> org.apache.flink.util.AbstractAutoCloseableRegistry.close(AbstractAutoCloseableRegistry.java:127)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$cancel$13(StreamTask.java:1081)
> at
> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
> at
> java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883)
> at
> java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2257)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.cancel(StreamTask.java:1074)
> at
> org.apache.flink.runtime.taskmanager.Task$TaskCanceler.run(Task.java:1773)
> ... 1 more
> {noformat}
> Which can be miss leading.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
