>From my understanding, we should copy the file into another folder and move
to source folder after copy is finished, otherwise we will read the
half-copied data or meet the issue as you mentioned above.
On Wed, May 18, 2016 at 8:32 PM, Ted Yu wrote:
> The following should handle the situation y
The following should handle the situation you encountered:
diff --git
a/streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala
b/streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.sca
index ed93058..f79420b 100644
---
a/streaming/src/main/scala
Hi,
I am trying to read the files in a streaming way using Spark
Streaming. For this I am copying files from my local folder to the
source folder from where spark reads the file.
After reading and printing some of the files, it gives the following error:
Caused by: org.apache.hadoop.ipc.RemoteExce
