What filesystem are you using? Is it possible that the source tries to
read a file that is the process of being written to disk?
n 6/3/2021 11:32 PM, Billy Bain wrote:
We are getting this exception occasionally after a job runs for a
month or more.
The files don't seem to be corrupt from our testing, so not sure what
this error means. Task resources & network connectivity seem fine.
How would you debug this?
ink) (1/1)#52423 (595ced3edfe32bb7d826955f1a195a29) switched from
RUNNING to FAILED.
java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(Unknown Source) ~[?:?]
at java.util.zip.InflaterInputStream.read(Unknown Source) ~[?:?]
at java.util.zip.GZIPInputStream.read(Unknown Source) ~[?:?]
at
org.apache.flink.api.common.io.InputStreamFSInputWrapper.read(InputStreamFSInputWrapper.java:80)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.api.common.io.DelimitedInputFormat.fillBuffer(DelimitedInputFormat.java:713)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.api.common.io.DelimitedInputFormat.readLine(DelimitedInputFormat.java:586)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.api.common.io.DelimitedInputFormat.nextRecord(DelimitedInputFormat.java:550)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.readAndCollectRecord(ContinuousFileReaderOperator.java:387)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.processRecord(ContinuousFileReaderOperator.java:349)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.lambda$new$0(ContinuousFileReaderOperator.java:237)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:189)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:609)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:573)
~[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755)
[flink-dist_2.12-1.12.1.jar:1.12.1]
at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:570)
[flink-dist_2.12-1.12.1.jar:1.12.1]
at java.lang.Thread.run(Unknown Source) [?:?]
--
Wayne D. Young
aka Billy Bob Bain
[email protected] <mailto:[email protected]>
