Hi All,
Am currently using Flume 1.7.0.
While processing a set of files, am encountering the below exception and
Flume hangs unpredictably.
18/06/21 05:40:40 ERROR source.SpoolDirectorySource: FATAL: Spool Directory
source src-tpa_idoru: { spoolDir: /tmp/tpa-idoru/ }: Uncaught exception in
SpoolDirectorySource thread. Restart or reconfigure Flume to continue
processing.
java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at
org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:283)
at
org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:132)
at
org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:70)
at
org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:89)
at
org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readDeserializerEvents(ReliableSpoolingFileEventReader.java:343)
at
org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:331)
at
org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:250)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
This issue looks similar to https://jira.apache.org/jira/browse/FLUME-2052
Wouldn't this fix be already included in 1.7? How do I enable flume to skip
these errors and proceed further?
Thanks,
Jyotsna
